Display device, screen projection processing method and storage medium

Abstract

The present disclosure relates to a display device, a screen projection processing method and a storage medium, and is applied to the technical field of screen projection. If the type of the current received screen projection data stream is non-audio data after the display device enters a target mode, the display is still in a black screen state, which causes the user to be unable to watch the content of the non-audio data and the experience is poor. The display device comprises a display, and a controller configured to: acquire the type of the current received screen projection data stream and the current playing state of the display device; determine that the type of the current received screen projection data stream is non-audio data and that the current playing state of the display device is in the target mode; control the display switch of the display to be in an open state and control the display to display the non-audio data, wherein when the playing state is in the target mode, the display switch of the display is in a closed state.

Description

[0001] This application claims priority to Chinese patent application No. CN202211477456.8, filed on November 23, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of screen projection technology, and in particular to a display device, screen projection processing method, and storage medium. Background Technology

[0003] With the continuous development of screen mirroring technology, screen mirroring has brought great convenience to people's work and life. Users can send data streams from their mobile devices to display devices for playback via screen mirroring or casting, thus obtaining a better viewing experience.

[0004] When playing a data stream via screen mirroring on a display device, if the user only wants to hear the audio, they can enter target mode on the display device. In target mode, the display device only plays audio data, and the monitor goes black.

[0005] However, if the user casts other data streams again, such as playing videos, the display device will still be in target mode, and the monitor will remain black, preventing the user from watching the video content and resulting in a poor user experience. Summary of the Invention

[0006] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a display device, a screen projection processing method, and a storage medium, which can prevent the display switch from being turned off (i.e., black screen state) when non-audio data is being played due to the target mode, thus ensuring that the user can normally watch the screen projection content during the projection process and improving the user experience.

[0007] In a first aspect, this disclosure provides a display device, the display device comprising: monitor; The controller is configured to: acquire the type of the currently received projection data stream and the current playback status of the display device; The system determines that the currently received screen-casting data stream is non-audio data, and determines that the current playback state of the display device is in target mode. It then controls the display switch of the monitor to be turned on and controls the monitor to display the non-audio data. When the playback state is in target mode, the display switch of the monitor is turned off.

[0008] Secondly, this disclosure provides a screen mirroring processing method, including: Get the type of the currently received screen-casting data stream and the current playback status of the display device; The system determines that the type of the currently received screen-casting data stream is non-audio data, and determines that the current playback state of the display device is in target mode. It then controls the display switch of the display device's monitor to be turned on and controls the monitor to display the non-audio data. When the playback state is in the target mode, the display switch of the monitor is turned off.

[0009] Thirdly, this disclosure provides a computer-readable storage medium, including: storing a computer program on the computer-readable storage medium, wherein when the computer program is executed by a processor, it implements the screen projection processing method as shown in the second aspect.

[0010] Fourthly, this disclosure provides a computer program product, including: when the computer program product is run on a computer, causing the computer to implement the screen projection processing method as shown in the second aspect.

[0011] Compared with the prior art, the technical solution provided in this disclosure has the following advantages: The controller of the display device first obtains the type of the currently received screen-casting data stream and the current playback state of the display device. Then, it determines that the type of the currently received screen-casting data stream is non-audio data and that the current playback state of the display device is in target mode. It then controls the display switch of the monitor to be turned on and controls the monitor to display non-audio data. When the playback state is in target mode, the display switch of the monitor is turned off. In the above technical solution, the display device can automatically control the display switch of the monitor to be turned on and control the monitor to display non-audio data when it determines that the type of the currently received screen-casting data stream is non-audio data and that the current playback state of the display device is in target mode. This avoids the situation where the display switch of the monitor is turned off when playing non-audio data due to the target mode, preventing the user from viewing the non-audio data content. This ensures that the user can normally view the screen-casting content during the screen-casting process and improves the user experience. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0013] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0014] Figure 1This is a schematic diagram illustrating an operational scenario between a display device, a control device, and a mobile device according to one or more embodiments of the present disclosure. Figure 2 This is a hardware configuration block diagram of a control device according to one or more embodiments of the present disclosure; Figure 3 This is a hardware configuration block diagram of a display device according to one or more embodiments of the present disclosure; Figure 4 This is a schematic diagram of software configuration in a display device according to one or more embodiments of the present disclosure; Figure 5 This is a schematic diagram showing the icon control interface of an application in a display device according to one or more embodiments of the present disclosure; Figure 6 This is a system framework diagram for screen casting processing according to one or more embodiments of this disclosure; Figure 7 A schematic flowchart illustrating a screen mirroring method provided in an embodiment of this disclosure; Figure 8 A schematic diagram of the display interface when the display device is currently in target mode, as provided in an embodiment of this disclosure; Figure 9 This is a schematic diagram of the display interface when the display switch of a display is in the on state and the display is displaying non-audio data, according to an embodiment of the present disclosure. Figure 10 A schematic flowchart illustrating another screen casting method provided in this embodiment of the present disclosure; Figure 11 This is a schematic diagram of an interface when a display shows a prompt message, provided in an embodiment of the present disclosure. Figure 12 A schematic diagram of the display interface when a display device enters an input page, provided as an embodiment of this disclosure; Figure 13 A schematic diagram illustrating the screen projection process of a display device provided in an embodiment of this disclosure; Figure 14 This is a schematic diagram of the overall process of a screen projection method provided in an embodiment of the present disclosure. Detailed Implementation

[0015] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0016] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0017] It should be noted that the brief descriptions of terms in this disclosure are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0018] It should be noted that in this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. For example, a product or apparatus that comprises a list of components is not necessarily limited to all components expressly listed, but may include other components not expressly listed or inherent to such product or apparatus.

[0019] In recent years, with the rapid development of the internet and the popularization of artificial intelligence (AI) technology, display devices, such as smart TVs, have gradually become smart terminals in people's homes and the first choice for living room entertainment. Especially during family gatherings, the large screen advantage of smart TVs becomes prominent, making them a mainstream form of entertainment.

[0020] Meanwhile, as user demands increase, the requirements for large screens also rise. For example, users want to share live game footage on a large screen; or they want to play their favorite videos on a large screen. To meet these needs, screen mirroring can effectively solve this problem. Current display devices can simultaneously support multiple screen mirroring protocols, such as Miracast and AirPlay.

[0021] When casting a data stream to a display device, if the user only wants to hear the audio, they can enter target mode on the display device. In target mode, the display device only plays audio data, and the monitor's display is off. However, if the user casts another data stream, such as a video, the display device will remain in target mode, and the monitor's display will remain off, preventing the user from viewing the video content and resulting in a poor user experience.

[0022] To address the aforementioned issues, this disclosure proposes a screen mirroring processing method. The controller of the display device first obtains the type of the currently received screen mirroring data stream and the current playback state of the display device. Then, it determines that the type of the currently received screen mirroring data stream is non-audio data and that the current playback state of the display device is in target mode. It then controls the display switch of the monitor to be turned on and displays the non-audio data. When the playback state is in target mode, the display switch of the monitor is turned off. In this technical solution, the display device can automatically control the display switch to be turned on and display the non-audio data when it determines that the type of the currently received screen mirroring data stream is non-audio data and that the current playback state of the display device is in target mode. This avoids the situation where the display switch is off when non-audio data is being played due to the target mode, preventing the user from viewing the non-audio data content. This ensures that the user can normally view the screen mirroring content during the screen mirroring process, improving the user experience.

[0023] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device, a control device, and a mobile device according to one or more embodiments of this disclosure. Figure 1 The system includes a control device 100, a display device 200, a mobile device 300, and a server 400. The mobile device 300 can transmit its playback data (such as audio, video, or images) to the display device 200 for playback and display. Meanwhile, the control device 100 (such as a remote control) using the display device 200 can perform relevant control functions on the display device 200.

[0024] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device 200 may include infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods, etc., through which the display device 200 can be controlled. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, and control panel input.

[0025] In some embodiments, mobile terminals, tablet computers, computers, laptops and other smart devices may also be used to control the display device 200.

[0026] In some embodiments, the display device 200 may receive instructions not through the aforementioned smart device or control device, but through touch or gestures.

[0027] In some embodiments, the display device 200 can also be controlled in ways other than control devices and smart devices. For example, it can be controlled by receiving user voice commands directly through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving user voice commands through a voice control device set outside the display device 200.

[0028] In some embodiments, the mobile terminal can install software applications with the display device 200 to connect and communicate via network communication protocols, achieving one-to-one control operations and data communication. Audio and video content displayed on the mobile terminal can also be transmitted to the display device 200 for synchronized display. The display device 200 can also communicate with the server 400 via various communication methods, allowing the display device 200 to connect via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 can be a cluster or multiple clusters, and may include one or more types of servers. The server 400 can provide various content and interactive features to the display device 200. The display device 200 can be a liquid crystal display, an OLED display, or a projection display device, etc. In addition to providing broadcast television reception functions, the display device 200 can also be equipped with intelligent network television functions that provide computer support.

[0029] Figure 2 This is a hardware configuration block diagram of a control device according to one or more embodiments of the present disclosure. Figure 2 As shown, the control device includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device can receive user input commands and convert them into commands that the display device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the display device 200. The communication interface 130 is used for external communication and includes at least one of a Wi-Fi chip, a Bluetooth module, NFC, or a replacement module. The user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, buttons, or a replacement module.

[0030] Figure 3 This is a hardware configuration block diagram of a display device according to one or more embodiments of the present disclosure. Figure 3As shown, the display device 200 includes at least one of a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface (i.e., a user input interface) 280. The controller 250 includes a central processing unit, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first to nth interface for input / output. The display 260 may be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and may also be a projection device and a projection screen. The tuner / demodulator 210 receives broadcast television signals via wired or wireless means, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals. The communicator 220 is a component used to communicate with the server 400 according to various communication protocol types. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. Display device 200 can establish the transmission and reception of control signals and data signals with control device 100 or server 400 via communicator 220. Detector 230 is used to collect signals from the external environment or signals interacting with the outside world. Controller 250 and tuner 210 can be located in different separate devices; that is, tuner 210 can also be located in an external device of the main device containing controller 250, such as an external set-top box. User interface 280 can be used to receive control signals from control devices (such as infrared remote controls).

[0031] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200. The user can input user commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input user commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0032] In some embodiments, a "user interface" is the medium through which an application or operating system interacts and exchanges information with a user, enabling the conversion between the internal form of information and a form acceptable to the user. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be interface elements such as icons, windows, and controls displayed on the screen of an electronic device. Controls can include at least one of the visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0033] Figure 4 This is a schematic diagram of software configuration in a display device according to one or more embodiments of the present disclosure. Figure 4 As shown, the system is divided into three layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), and the Operating System layer.

[0034] In some embodiments, at least one application runs in the application layer. These applications can be Windows programs, system settings programs, or clock programs that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the applications in the application layer include, but are not limited to, the examples above. In this embodiment, the application layer may include TV Input (the input module of a television, also known as a channel list module) and an AirPlay Android Application Package (AirPlay APK). The AirPlay APK is mainly used to obtain the attributes of the entire device (display device), such as obtaining the volume of the entire device, etc., which is not limited here.

[0035] In some embodiments, the framework layer mainly includes: a switching channel (switching Input, switching TIS), an AirPlay service, and audio / video middleware. The AirPlay service can interact with the AirPlay process and the AirPlay APK, control the lifecycle of the AirPlay APK, and interact with the switching channel and audio / video middleware through interface calls; the AirPlay service interacts with the AirPlay APK via broadcast; the switching channel interacts with the TV Input through the Java Native Interface (JNI).

[0036] In some embodiments, the operating system layer mainly includes the implementation of the AirPlay process and functional modules such as image, video, and audio. The operating system here can be Linux. The AirPlay process is primarily a resident native process, including the AirPlay protocol stack, capable of calling specific AirPlay playback functional modules (image, video, or audio, etc.) to implement corresponding functions. The AirPlay process and the AirPlay service interact through network communication. This network communication can be based on network communication protocols, such as socket protocols or other protocols; this embodiment does not limit this.

[0037] In some embodiments, when a user operates a mobile device to cast their screen, the AirPlay protocol stack can receive the casting information or casting request sent by the mobile device and send the casting information or casting request to the AirPlay service. After receiving the casting information or casting request (including casting parameters), the AirPlay service can determine the type of the data stream to be cast (such as video, audio, or mirroring), and can also determine whether this is the first casting. If it is the first casting, it calls the interface in the Android TV Input Framework (TIF) to make TV Input switch channels, switching the current channel of the display device to the channel corresponding to AirPlay. If it is not the first casting, the AirPlay service will obtain the type of the currently received casting data stream and call the audio and video middleware interface to obtain the current playback status of the display device. If the type of the currently received casting data stream is non-audio data, and it is determined that the current playback status of the display device is in target mode, then AirPlay will call the audio and video middleware interface to turn off the switch corresponding to the Audio Only function, so that the display switch of the monitor is turned on. At this time, the monitor screen will light up and the monitor can display non-audio data.

[0038] It should be noted that, Figure 4 This explanation uses the Android platform as an example to illustrate the software configuration of a display device, and is not intended to limit its scope.

[0039] Figure 5 This is a schematic diagram showing the icon control interface of an application in a display device according to one or more embodiments of the present disclosure. Figure 5As shown, the application layer contains at least one application whose corresponding icon control can be displayed on the screen, such as: live TV application icon control, video-on-demand application icon control, media center application icon control, application center icon control, game application icon control, etc. Live TV applications can provide live television from different signal sources. Video-on-demand applications can provide video from different storage sources. Unlike live TV applications, video-on-demand provides video display from certain storage sources. Media center applications can provide applications for playing various multimedia content. The application center can provide storage for various applications.

[0040] In some embodiments, the display device described above is a terminal device with display functionality, such as a television, mobile phone, computer, or educational device. In this display device: The output interface (monitor 260, and / or audio output interface 270) is configured to output user interaction information; Communicator 220, used for communicating with the server; Controller 250 is configured as follows: Obtain the type of the currently received screen-casting data stream and the current playback status of the display device; The system determines that the currently received screen-casting data stream is non-audio data, and determines that the current playback state of the display device is in target mode. It then controls the display switch of the monitor to be turned on and controls the monitor to display the non-audio data. When the playback state is in target mode, the display switch of the monitor is turned off.

[0041] In some embodiments, the controller 250 is further configured to: After determining that the type of the currently received screen projection data stream is non-audio data and that the current playback state of the display device is in target mode, the system controls the display to display a prompt message, which is used to prompt the user whether to turn on the display switch; Upon receiving an operation to turn on the display switch, the system controls the display switch to be in the on state and controls the display to display the non-audio data.

[0042] In some embodiments, the controller 250 is specifically configured to: In response to the current screen projection request, determine the channel information of the display device; When the channel of the display device is determined to be the channel corresponding to AirPlay based on the channel information, and the current screen casting request is a non-mirror screen casting request, the type of the currently received screen casting data stream and the current playback status of the display device are obtained.

[0043] In some embodiments, the controller 250 is specifically configured to: Call the audio / video middleware interface to get the status of the switch corresponding to the Audio Only function; When the state is "on", it is determined that the current playback state of the display device is in the target mode.

[0044] In some embodiments, the controller 250 is specifically configured to: The audio / video middleware interface is invoked to turn off the switch corresponding to the Audio Only function, thereby controlling the display switch of the monitor to be in the on state.

[0045] In some embodiments, the controller 250 is further configured to: The current screen mirroring request is determined to be the first screen mirroring request. A screen mirroring connection is established between the display device and the sender of the current screen mirroring request, and the channel of the display device is switched to the channel corresponding to the air play. Based on the screen mirroring connection and the corresponding channel for air playback, the system receives the data stream sent by the sender and plays the data stream.

[0046] In some embodiments, the screen mirroring protocol between the display device and the sender of the current screen mirroring request is the AirPlay protocol.

[0047] In summary, this disclosure, by executing the aforementioned screen casting process on a display device, allows the display device controller to first obtain the type of the currently received screen casting data stream and the current playback state of the display device. Then, it determines that the type of the currently received screen casting data stream is non-audio data and that the current playback state of the display device is in target mode. It then controls the display switch of the monitor to be on and controls the monitor to display non-audio data. When the playback state is in target mode, the display switch of the monitor is off. In this technical solution, the display device can automatically control the display switch to be on and control the monitor to display non-audio data when it determines that the type of the currently received screen casting data stream is non-audio data and that the current playback state of the display device is in target mode. This avoids the situation where the display switch of the monitor is off when non-audio data is being played due to the target mode, preventing the user from viewing the non-audio data content. This ensures that the user can normally view the screen casting content during the casting process, improving the user experience.

[0048] Figure 6 This is a system framework diagram for screen mirroring processing according to one or more embodiments of this disclosure. Figure 6As shown, the system may include an acquisition module 401, a determination module 402, and a control module 403. The system first acquires the type of the currently received projection data stream and the current playback state of the display device through the acquisition module 401. Then, the determination module 402 determines that the type of the currently received projection data stream is non-audio data and that the current playback state of the display device is in target mode. Finally, the control module 403 controls the display switch of the monitor to be on and controls the monitor to display non-audio data. When the playback state is in target mode, the display switch of the monitor is off. In this technical solution, the display device can automatically control the display switch to be on and control the monitor to display non-audio data when it is determined that the type of the currently received projection data stream is non-audio data and that the current playback state of the display device is in target mode. This eliminates the need for manual user operation and avoids the situation where the display switch is off when non-audio data is being played due to the target mode, preventing the user from viewing the non-audio data content. This ensures that the user can normally view the projection content during the projection process, improving the user experience.

[0049] To illustrate the screen mirroring solution in this disclosure in more detail, the following will use an exemplary approach in conjunction with... Figure 7 To explain, it is understandable that Figure 7 The steps involved may include more or fewer steps in actual implementation, and the order of these steps may also be different, as long as the screen projection processing method provided in the embodiments of this disclosure can be achieved. The embodiments of this disclosure do not limit the steps.

[0050] Figure 7 This is a flowchart illustrating a screen mirroring method provided in an embodiment of this disclosure. This embodiment is applicable to situations where the display switch is off during screen mirroring, preventing the viewer from viewing the mirrored content. Figure 7 As shown, the screen mirroring method specifically includes the following steps: S510 obtains the type of the currently received screen mirroring data stream and the current playback status of the display device.

[0051] The currently received screen-casting data stream can be understood as the screen-casting data stream currently received by the display device and sent by the mobile device. This means that a user can operate a mobile device to cast the screen, and during the casting process, the user can cast multiple times, for example, first audio data, then video data, then image data, etc. Correspondingly, the mobile device will send multiple screen-casting data streams to the display device. In this embodiment, the mobile device sending the screen-casting data streams to the display device during the casting process is not specifically limited; it can be the same mobile device or different mobile devices. Mobile devices can include client devices that support the AirPlay protocol, such as mobile phones, iPads, or MacBooks. The type of screen-casting data stream can include audio data and non-audio data. Audio data refers to sound-related data, such as music and recordings; non-audio data can include video streams, live streams, or broadcast streams containing visual content. The current playback state of the display device can include: the display device is currently playing only audio, the display device is currently playing only visual content, or the display device is currently playing both audio and visual content simultaneously.

[0052] To address the issue of users being unable to view screen-projected content due to the monitor being switched off during the projection process, the display device's controller acquires the type of the currently received projection data stream and the current playback status of the display device. This allows it to determine whether the currently received projection data stream is non-audio data and whether the display device is currently in target mode, thus facilitating the execution of subsequent steps.

[0053] S520 determines that the type of the currently received screen-casting data stream is non-audio data, and determines that the current playback state of the display device is in target mode, controls the display switch of the display device's monitor to be turned on, and controls the monitor to display non-audio data.

[0054] When playback is in target mode, the monitor's display switch is off; that is, the monitor is black, and the user cannot view the content. The monitor's display switch can be understood as a switch that controls whether the monitor is in a black or bright state.

[0055] Based on the type of the currently received projection data stream and the current playback status of the display device, the controller determines that the received projection data stream is non-audio data and that the display device is currently in target mode. This indicates that the display device needs to project and play non-audio data, but the monitor's display switch is off, preventing normal playback of non-audio data content. Therefore, the controller turns on the monitor and displays the non-audio data (screen content), thus achieving normal playback of the non-audio data.

[0056] In this embodiment, the controller of the display device first obtains the type of the currently received projection data stream and the current playback state of the display device. Then, it determines that the type of the currently received projection data stream is non-audio data and that the current playback state of the display device is in target mode. It then controls the display switch of the monitor to be turned on and controls the monitor to display the non-audio data. When the playback state is in target mode, the display switch of the monitor is turned off. In this technical solution, the display device can automatically control the display switch to be turned on and control the monitor to display non-audio data when it determines that the type of the currently received projection data stream is non-audio data and that the current playback state of the display device is in target mode. This eliminates the need for manual user operation and avoids the situation where the display switch of the monitor is turned off when non-audio data is being played due to the target mode, preventing the user from viewing the non-audio data content. This ensures that the user can normally view the projection content during the projection process, improving the user experience.

[0057] In some embodiments, optionally, determining that the current playback state of the display device is in target mode may specifically include: Call the audio / video middleware interface to get the status of the switch corresponding to the Audio Only function; When the state is "on", it is determined that the current playback state of the display device is in the target mode.

[0058] The Audio Only function provides the display device with the ability to play only sound and turn off the display screen. Turning off the display screen means controlling the monitor's display switch to be in the off state. The switch corresponding to the Audio Only function can be understood as a module used to control whether this function is turned on or off. The state of the switch corresponding to the Audio Only function can be controlled by the user using a remote control or a mobile device connected to the display device. For example, by clicking the Menu button on the remote control (or any other button, not limited in this embodiment), the switch corresponding to the Audio Only function can be accessed, thereby controlling its state. The audio / video middleware interface can be understood as an interface that records and controls the state of the switch corresponding to the Audio Only function.

[0059] Specifically, the display device controller calls the audio / video middleware interface to obtain the status of the switch corresponding to the Audio Only function. When the switch corresponding to the Audio Only function is in the on state, it can be determined that the current playback state of the display device is in the target mode.

[0060] In this embodiment, the method described above determines that the current playback state of the display device is in target mode, which is simple, fast, and efficient.

[0061] In some embodiments, optionally, controlling the display switch of the monitor to be in the on state may specifically include: The audio / video middleware interface is invoked to turn off the switch corresponding to the Audio Only function, thereby controlling the display switch of the monitor to be in the on state.

[0062] Specifically, the display device controller calls the audio / video middleware interface to turn off the switch corresponding to the Audio Only function. That is, the display device only plays sound and the function of turning off the display device screen is turned off. After turning off the switch corresponding to the Audio Only function, the display switch of the monitor can be controlled to be turned on, and the monitor can display the picture normally.

[0063] In this embodiment, the above method can automatically control the display switch of the monitor to be in the on state, which is simple and efficient, requires no manual operation by the user, and avoids the situation where the monitor cannot display the picture normally when playing non-audio data because the monitor is in a black screen state.

[0064] For example, Figure 8 This is a schematic diagram of the display interface when the display device is currently in target mode, as provided in an embodiment of this disclosure. Figure 8As shown, when the display device is currently in target mode (i.e., the switch corresponding to the Audio Only function is not turned on), the display switch of the monitor is in the off state, and at this time, the monitor interface is a black screen.

[0065] For example, Figure 9 This is a schematic diagram of the display interface when the display switch is on and the display is displaying non-audio data, as provided in an embodiment of this disclosure. Figure 9 As shown, when the monitor's display switch is on and the monitor is displaying non-audio data, the monitor will normally display the screen content corresponding to the non-video data.

[0066] It should be noted that: Figure 9 The screen content shown is for illustrative purposes only. The screen content can be determined based on the playback progress corresponding to non-video data, and this embodiment does not limit it.

[0067] Figure 10 This is a flowchart illustrating another screen casting method provided in this embodiment. This embodiment is an optimization based on the above embodiment. Optionally, this embodiment can be applied to describe the process after determining that the currently received screen casting data stream is non-audio data and that the display device's current playback state is in the target mode. Figure 10 As shown, the screen mirroring method specifically includes the following steps: S610, obtains the type of the currently received screen mirroring data stream and the current playback status of the display device.

[0068] S620 determines that the type of the currently received screen-casting data stream is non-audio data, and determines that the current playback state of the display device is in target mode, and controls the display to show a prompt message.

[0069] The prompt message is used to ask the user whether to turn on the monitor's display.

[0070] The display device controller determines that the type of the currently received screen-casting data stream is non-audio data, and determines that the current playback state of the display device is in target mode. It then controls the display to show a prompt message, which prompts the user whether to turn on the display.

[0071] S630 receives an operation to turn on the display switch, controls the display switch to be in the on state, and controls the display to display video data.

[0072] When the controller of the display device receives a user-triggered operation to turn on the display switch, it controls the display switch to be in the on state and controls the display to display video data.

[0073] In this embodiment, the controller of the display device first obtains the type of the currently received screen-casting data stream and the current playback state of the display device. Then, it determines that the type of the currently received screen-casting data stream is non-audio data and that the current playback state of the display device is in target mode. It then controls the display to display a prompt message. Finally, upon receiving an operation to turn on the display switch, it controls the display switch to be in the on state and controls the display to display video data. By displaying a prompt message on the display, the user is informed that the display switch was turned off during the previous screen casting, which enhances the human-computer interaction experience. At the same time, after receiving the operation to turn on the display switch, it controls the display switch to be in the on state and controls the display to display video data. This avoids the situation where the display switch is in the off state (i.e., black screen state) when non-audio data is being played due to the target mode, preventing the user from viewing the non-audio data content. This ensures that the user can normally view the screen-casting content during the screen casting process, improving the user experience.

[0074] For example, Figure 11 This is a schematic diagram of an interface for displaying prompt information on a monitor, provided as an embodiment of this disclosure. Figure 11 As shown, after determining that the currently received projection data stream is non-audio data and that the display device's current playback state is in target mode, the display device's controller can control the monitor to display a prompt message. This prompt message can include a countdown prompt, specifically, the display device's controller can start a 5-minute countdown, and a pop-up message will appear 15 seconds before the countdown ends. Figure 11 The prompt page shown includes a prompt asking whether to turn the monitor on or off, with options of "Yes" and "No". "Yes" turns the monitor on, while "No" turns it off. The 15s indicator in the upper right corner is a timer; its value decreases over time until it reaches 0.

[0075] pass Figure 11 The displayed prompt page asks the user whether to turn on the monitor to avoid the screen remaining black. If no user action is received, the display device controller will automatically close the prompt page and turn on the monitor after 15 seconds.

[0076] It is understood that 5 minutes and 15 seconds are custom values, and other values ​​are also possible. This embodiment does not limit them.

[0077] In some embodiments, optionally, obtaining the type of the currently received projection data stream and the current playback status of the display device may specifically include: In response to the current screen projection request, determine the channel information of the display device; When the channel of the display device is determined to be the channel corresponding to AirPlay based on the channel information, and the current screen casting request is a non-mirror screen casting request, the type of the currently received screen casting data stream and the current playback status of the display device are obtained.

[0078] The current screen mirroring request can be understood as: the screen mirroring request currently received by the display device, sent by the mobile device. The screen mirroring request can carry relevant screen mirroring parameters, such as mirroring, screen mirroring video, or screen mirroring audio.

[0079] Specifically, after receiving a current screen mirroring request from a mobile device, the display device controller responds to the request by determining the display device's channel information. This communication information can include the identifier of the channel currently corresponding to the display device. When the display device controller determines, based on the channel information, that the display device's channel is the channel corresponding to AirPlay (indicating that the current screen mirroring request is not the first one), and that the current screen mirroring request is not a mirroring request, it obtains the type of the currently received screen mirroring data stream and the current playback status of the display device.

[0080] It's important to note that if the current casting request is a first-time casting request or a mirror casting request, the display device will execute channel switching logic, meaning it will switch the display device's channel to the channel corresponding to AirPlay. After the display device executes the channel switching logic, the monitor will not go black. However, if the current casting request is neither a first-time casting request nor a mirror casting request, the display device will not execute the channel switching logic. In this case, if the display device's current playback state is in target mode (which can be set during previous casting), the monitor will remain black (i.e., the monitor's display switch is off).

[0081] In this embodiment, under the above conditions, obtaining the type of the currently received screen projection data stream and the current playback status of the display device is beneficial for subsequently determining whether to execute the step of controlling the display switch of the monitor to be in the on state, and preventing the situation where non-audio data video content cannot be played due to the display switch of the monitor being in the off state.

[0082] It should be noted that if the current screen casting request is the first screen casting request, the display device will execute the channel switching logic. There is no restriction on the type of screen casting data stream corresponding to the first screen casting request. That is, regardless of whether the screen casting data stream is audio data or non-audio data, the display device will execute the channel switching logic.

[0083] For example, Figure 12This is a schematic diagram of the display interface when a display device enters an input page, as provided in an embodiment of this disclosure. Figure 12 As shown, users can operate the input button on the remote control or operate other mobile devices connected to the display device to enter the input page. This page can include multiple options, such as: Home, TV, AirPlay, HDMI (High Definition Multimedia Interface) 1, and HDMI 2. When switching from one option to another on this page, the display device will execute the channel switching logic.

[0084] In some embodiments, when the AirPlay entry is in the settings and the AirPlay option is selected, the display device will also execute the logic of switching channels.

[0085] In some embodiments, optionally, the method may further include: The current screen mirroring request is determined to be the first screen mirroring request. A screen mirroring connection is established between the display device and the sender of the current screen mirroring request, and the channel of the display device is switched to the channel corresponding to the air play. Based on the screen mirroring connection and the corresponding channel for air playback, the system receives the data stream sent by the sender and plays the data stream.

[0086] The sender of the current screen mirroring request can be any mobile device; this embodiment does not limit this.

[0087] Specifically, after receiving a screen mirroring request from a mobile device, the display device's controller responds by determining the display device's channel information. If the channel information indicates that the display device's channel is not the channel corresponding to AirPlay, it determines that the current screen mirroring request is the first one. At this point, the display device's controller establishes a screen mirroring connection between the display device and the sender of the current request, and switches the display device's channel to the channel corresponding to AirPlay. Simultaneously, because the current screen mirroring request is the first one, the display device executes the channel switching logic. After executing this logic, the display will not experience a black screen. Therefore, based on this screen mirroring connection and the channel corresponding to AirPlay, the display device's controller can normally play the data stream after receiving it from the sender.

[0088] In this embodiment, the above process ensures that the corresponding data stream plays normally during the first screen projection.

[0089] In some embodiments, the screen mirroring protocol between the display device and the sender of the current screen mirroring request may be the AirPlay protocol.

[0090] AirPlay is a playback technology in the iOS operating system that enables the transfer of files (including videos, photos, and mirrors) from iOS or later versions to devices that support AirPlay.

[0091] Specifically, the AirPlay protocol contains channel switching logic. Therefore, in this embodiment, the screen mirroring protocol between the display device and the sender of the current screen mirroring request is the AirPlay protocol.

[0092] For example, Figure 13 This is a schematic diagram illustrating a screen projection process for a display device provided in an embodiment of this disclosure. Figure 13 As shown, the screen mirroring process of a display device may include the following steps: 1. When a user casts their screen via a mobile device, the screen casting driver module of the display device can receive the screen casting parameters sent by the mobile device, such as mirroring, screen casting video or screen casting audio, and will determine whether the current screen casting is the first time. 2. If the current screen mirroring is the first time, since the display device is not currently on an AirPlay channel, the screen mirroring driver sends a channel switching message to the channel switching module. This causes the channel switching module to switch the display device's channel to the corresponding AirPlay channel, and then calls the AirPlay function module (such as audio, video, or mirroring) to implement the corresponding function. If the current screen mirroring is a mirrored screen mirroring, regardless of whether it is the first time, the screen mirroring driver will send a channel switching message to the channel switching module and implement mirrored screen mirroring through the mirroring function module, so that the display device can synchronously play the screen content of the mobile device. 3. If the current screen casting is not the first time and is not a mirrored screen casting, the screen casting driver defaults to the channel corresponding to AirPlay for the display device. Then, when the type of the currently received screen casting data stream is non-audio data, the screen casting driver calls the audio / video middleware interface to obtain the status of the switch corresponding to the Audio Only function. If the status of the switch corresponding to the Audio Only function is on, the driver controls the status of the switch corresponding to the Audio Only function to be off, so as to control the display switch to be on and control the display to display non-audio data; if the status of the switch corresponding to the Audio Only function is off, the driver controls the display to display non-audio data.

[0093] For example, Figure 14 This is a schematic diagram illustrating the overall flow of a screen mirroring method provided in an embodiment of this disclosure. Figure 14As shown, the overall process may include the following steps: 1. When a user selects any music playback software on their mobile device to cast audio (i.e., the user casts audio data), the TV will display the audio information from the mobile device and start playing the audio. 2. On the Audio page of the display device, the user can press the menu button on the remote control (or use other methods) and click the "Audio Only" option in the menu. This turns on the switch corresponding to the Audio Only function. At this time, the display device will only play sound and turn off the display screen. 3. When a user selects non-audio data, such as video data, for screen casting on a mobile device, because the display device has previously played audio, the logic for switching channels will not be executed when playing video again. It is assumed that the display device's channel has not changed. Since the switch corresponding to the Audio Only function is in the on state, the display screen will not light up and will be in a black screen state. 4. The display device determines whether the type of the currently received screen-projected data stream is non-audio data, and whether the switch corresponding to the Audio Only function is in the on state; 5. If the display device determines that the type of the currently received screen projection data stream is non-audio data, and the switch corresponding to the Audio Only function is in the on state, then turn off the switch corresponding to the Audio Only function to control the display switch of the monitor to be in the on state, and control the display device to play the current non-audio data normally. 6. If the display device determines that the type of the currently received screen-casting data stream is audio data, or the state of the switch corresponding to the Audio Only function is not in the on state (i.e., in the off state), control the display device to play the current non-audio data normally.

[0094] In some embodiments, optionally, the change in the screen mirroring state can be determined in the following way, at which point the current state of Airplay can be identified as Audio / Video / Screen.

[0095] The code is shown below: public void notifyUIPage (String ui) { / / Determine the current AirPlay status if (“screen”.equals(ui) || “audio”.equals(ui) || “video”.equals(ui)) { quitAudioOnlyWhenAudioToVideo(ui); / / setAirplayState(ui); / / Record the current state of the UI. } }

[0096] In the code file above, the parameter `ui` represents the current AirPlay state, which can be Screen (equivalent to the AirPlay home screen), Video, or Audio. The `notifyUIPage` method is called whenever a user casts their screen. For example, if the user casts video data, then `ui = video`; if the user casts audio data, then `ui = audio`.

[0097] The implementation logic of quitAudioOnlyWhenAudioToVideo(ui) is as follows: First, retrieve the previous AirPlay status. If it was Audio and the current AirPlay status is Video, then check if the switch corresponding to the Audio Only function is on. If the Audio Only function is turned off, the monitor screen will light up.

[0098] The code is shown below: public void quitAudioOnlyWhenAudioToVideo (String currentUi) { String lastUi = getAirplayState(); / / Get the last UI state int audioOnlyState = getAudioOnlyState(); / / Get the current audioOnly state of the system. / / Determine if the previous state was Audio, the current state is Video, and audioOnlyState is on. if (“audio”.equals(lastUi) && “video”.equals(currentUi) &&audioOnlyState == 1) { quitAudioOnly(); / / Turn off Audio Only and turn on the screen. } }

[0099] In summary, this disclosure, by executing the aforementioned screen casting process on a display device, allows the display device controller to first obtain the type of the currently received screen casting data stream and the current playback state of the display device. Then, it determines that the type of the currently received screen casting data stream is non-audio data and that the current playback state of the display device is in target mode. It then controls the display switch of the monitor to be on and controls the monitor to display non-audio data. When the playback state is in target mode, the display switch of the monitor is off. In this technical solution, the display device can automatically control the display switch to be on and control the monitor to display non-audio data when it determines that the type of the currently received screen casting data stream is non-audio data and that the current playback state of the display device is in target mode. This avoids the situation where the display switch of the monitor is off when non-audio data is being played due to the target mode, preventing the user from viewing the non-audio data content. This ensures that the user can normally view the screen casting content during the casting process, improving the user experience.

[0100] This disclosure provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the screen projection processing method described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0101] The computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0102] This disclosure provides a computer program product, including: when the computer program product is run on a computer, causing the computer to implement the above-described screen projection processing method.

[0103] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the discussion in some embodiments above is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the embodiments and various different variations of the embodiments suitable for specific application considerations.

Claims

1. A display device, characterized in that, include: monitor; The controller is configured to: acquire the type of the currently received projection data stream and the current playback status of the display device; Obtain the type of the currently received screen-casting data stream and the current playback status of the display device; When the type of the currently received screen-casting data stream is non-audio data, and the current playback state of the display device is in target mode, the display switch of the display is turned on, and the display is controlled to display the non-audio data. When the playback state is in the target mode, the display switch of the monitor is in the off state.

2. The display device according to claim 1, characterized in that, The controller is also configured to: After determining that the type of the currently received screen projection data stream is non-audio data and that the current playback state of the display device is in target mode, the system controls the display to display a prompt message, which is used to prompt the user whether to turn on the display switch; Upon receiving an operation to turn on the display switch, the system controls the display switch to be in the on state and controls the display to display the non-audio data.

3. The display device according to claim 1, characterized in that, The controller is specifically configured as follows: In response to the current screen projection request, determine the channel information of the display device; When the channel of the display device is determined to be the channel corresponding to AirPlay based on the channel information, and the current screen casting request is a non-mirror screen casting request, the type of the currently received screen casting data stream and the current playback status of the display device are obtained.

4. The display device according to any one of claims 1-3, characterized in that, The controller is specifically configured as follows: Call the audio / video middleware interface to get the status of the switch corresponding to the Audio Only function; When the switch corresponding to the Audio Only function is in the on state, it is determined that the current playback state of the display device is in the target mode.

5. The display device according to claim 4, characterized in that, The controller is specifically configured as follows: The audio / video middleware interface is invoked to turn off the switch corresponding to the Audio Only function, thereby controlling the display switch of the monitor to be in the on state.

6. The display device according to claim 3, characterized in that, The controller is also configured to: The current screen mirroring request is determined to be the first screen mirroring request. A screen mirroring connection is established between the display device and the sender of the current screen mirroring request, and the channel of the display device is switched to the channel corresponding to the air play. Based on the screen mirroring connection and the corresponding channel for air playback, the system receives the data stream sent by the sender and plays the data stream.

7. A display device, Includes controllers, displays, and audio / video middleware; The controller receives the screen projection data stream sent by the mobile device. The display is driven by the controller and displays the screen content in the screen projection data stream when the display switch of the display is in the on state. The audio and video middleware is used to manage the state of the sound-only function switch. When the sound-only function switch is in the on state, the display device enters the target mode. In the target mode, the display switch of the display is in the off state. The controller responds to the current screen projection request and determines the channel information of the display device; When it is determined from the channel information that the channel of the display device is the channel corresponding to AirPlay and the current screen casting request is a non-mirror screen casting request, the type of the currently received screen casting data stream and the status of the sound-only function switch are obtained. When it is determined that the type of the currently received screen-casting data stream is non-audio data and the sound-only function switch is in the on state, the controller turns off the sound-only function switch through the audio-visual middleware interface, so that the display switch of the display is switched to the on state, and controls the display to display the non-audio data.

8. A screen projection processing method, characterized in that, include: Get the type of the currently received screen-casting data stream and the current playback status of the display device; When the type of the currently received screen-casting data stream is non-audio data, and the current playback state of the display device is in target mode, the display switch of the display is turned on, and the display is controlled to display the non-audio data. When the playback state is in the target mode, the display switch of the monitor is in the off state.

9. The method according to claim 8, characterized in that, After determining that the type of the currently received projection data stream is non-audio data, and after determining that the current playback state of the display device is in target mode, the method further includes: The display is controlled to show a prompt message, which prompts the user whether to turn on the display. Upon receiving an operation to turn on the display switch, the system controls the display switch to be in the on state and controls the display to display the non-audio data.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in claim 8.

Images