Display device and multipath screen projection sound broadcast control method

Abstract

The invention provides a display device and a multipath projection screen sound broadcast control method. According to the method, a terminal device establishing a projection screen connection with the display device and a connection sequence of the projection screen connection established between the terminal device and the display device can be recorded based on a projection screen list. When the display device establishes the projection screen connection with at least two terminal devices, the sound mode of multi-path projection screen is adjusted through the projection screen list, the sound mode of one terminal device in the projection screen list is set to be a non-silent mode, and the other terminal devices are set to be silent modes, so that the display device only plays the sound of one path of projection screen. According to the method, the sound can be dynamically set according to different screen projection scenes, the sound can be dynamically controlled according to the independent selection of a user, and the problem of sound conflicts during multi-path screen projection is solved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a multi-channel screen projection and sound control method. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, communication terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users. Display devices can establish communication connections with terminal devices such as mobile phones, and based on specific screen projection protocols, obtain the screen projection data of terminal devices, so as to display the screen projection images of terminal devices through display devices.

[0003] Some display devices can establish communication connections with multiple terminal devices at the same time and receive screen projection data from multiple terminal devices, forming a multi-channel screen projection scenario. However, during the audio transmission process of multi-channel screen projection, if the audio playback of multi-channel screen projection is not controlled, the audio of multiple screen projections will be played at the same time, resulting in sound conflicts and reducing the user experience. Summary of the invention

[0004] The present application provides some embodiments of a display device and a multi-channel screen projection sound control method to solve the problem of sound conflict during multi-channel screen projection.

[0005] In a first aspect, the present application provides a display device, including a display, a communicator, an audio output interface, and a controller. The display is configured to display a multi-channel projection interface, the multi-channel projection interface includes at least two projection windows, and the projection window is used to display the projection screen of the terminal device. The communicator is configured to establish a projection connection with at least two terminal devices. The audio output interface is configured to play audio data. The controller is configured to perform the following program steps:

[0006] Control the audio output interface to play audio data of a terminal device whose audio parameter in the screen projection list is a first parameter value, wherein the audio parameter is used to indicate whether the audio data is allowed to be played or not, and the first parameter value is used to indicate that the audio data is allowed to be played, and the screen projection list is used to record the terminal devices that establish a screen projection connection with the display device, and the connection order in which the terminal devices establish a screen projection connection with the display device;

[0007] In response to an exit event of disconnecting the screen projection connection relationship between the display device and the first terminal device, removing the first terminal device from the screen projection list, and recording the exit time of the exit event, the first terminal device being a terminal device in the screen projection list whose current audio parameter is a first parameter value;

[0008] If a connection event for establishing a screen projection connection relationship between the display device and the second terminal device is detected within a preset time threshold after the exit time, the second terminal device is added to the screen projection list, and the audio parameter of the second terminal device is set to the first parameter value;

[0009] If a connection event for establishing a screen projection connection relationship between the display device and the second terminal device is not detected within a preset time threshold after the exit time, a third terminal device is determined in the screen projection list according to the connection order, and the audio parameter of the third terminal device is set to the first parameter value.

[0010] In a second aspect, the present application further provides a multi-channel screen projection sound playback control method, which is applied to the above-mentioned display device, and the method includes:

[0011] Control the audio output interface to play audio data of a terminal device whose audio parameter in the screen projection list is a first parameter value, wherein the audio parameter is used to indicate whether the audio data is allowed to be played or not, and the first parameter value is used to indicate that the audio data is allowed to be played, and the screen projection list is used to record the terminal devices that establish a screen projection connection with the display device, and the connection order in which the terminal devices establish a screen projection connection with the display device;

[0012] In response to an exit event of disconnecting the screen projection connection relationship between the display device and the first terminal device, removing the first terminal device from the screen projection list, and recording the exit time of the exit event, the first terminal device being a terminal device in the screen projection list whose current audio parameter is a first parameter value;

[0013] If a connection event for establishing a screen projection connection relationship between the display device and the second terminal device is detected within a preset time threshold after the exit time, the second terminal device is added to the screen projection list, and the audio parameter of the second terminal device is set to the first parameter value;

[0014] If a connection event for establishing a screen projection connection relationship between the display device and the second terminal device is not detected within a preset time threshold after the exit time, a third terminal device is determined in the screen projection list according to the connection order, and the audio parameter of the third terminal device is set to the first parameter value.

[0015] It can be seen from the above technical solutions that the present application provides a display device and a multi-channel screen projection sound playback control method, which can record the terminal devices that establish a screen projection connection with the display device based on the screen projection list, as well as the connection order of the terminal devices and the display device to establish the screen projection connection. When the display device establishes a screen projection connection with at least two terminal devices, the sound mode of the multi-channel screen projection is adjusted through the screen projection list, and the sound mode of one terminal device in the screen projection list is set to a non-silent mode, and the other terminal devices are set to a silent mode, so that the display device only plays the sound of one channel of screen projection. The method can dynamically set the sound according to different screen projection scenarios, and can also dynamically control the sound according to the user's autonomous choice, so as to solve the problem of sound conflicts during multi-channel screen projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0018] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0019] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0020] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0021] Figure 5 A schematic diagram of mirror screen projection provided in some embodiments of the present application;

[0022] Figure 6 A schematic diagram of push screen projection provided in some embodiments of the present application;

[0023] Figure 7 A schematic diagram of a multi-screen projection interface provided in some embodiments of the present application;

[0024] Figure 8 A schematic diagram of a functional mode of a display device provided in some embodiments of the present application;

[0025] Fig. 9 A schematic diagram of a guide page provided for some embodiments of the present application;

[0026] Fig.10 A schematic diagram of the interface display when casting a screen in some embodiments of the present application;

[0027] Fig.11 A schematic diagram of the effect of dynamically adjusting the display area provided in some embodiments of the present application;

[0028] Fig.12 A schematic diagram of a playback control button control provided in some embodiments of the present application;

[0029] Fig.13 A schematic diagram of the effect of dynamically adjusting the number of play control button controls provided in some embodiments of the present application;

[0030] Fig.14 A schematic diagram of the flow of voice playback control provided in some embodiments of the present application;

[0031] Fig.15 A schematic diagram of the display effect of the sound view provided in some embodiments of the present application;

[0032] Fig.16 A schematic diagram of the effect of dynamically adjusting the sound view provided in some embodiments of the present application;

[0033] Fig.17 A schematic diagram of the display effect of disconnecting one-way screen projection provided in some embodiments of the present application;

[0034] Fig.18 A flow chart of sound switching when screen projection is disconnected provided in some embodiments of the present application;

[0035] Fig.19 A schematic diagram of the effect of locking the sound mode of the screen projection window provided in some embodiments of the present application;

[0036] Fig. 20 A schematic diagram of switching full screen effects provided in some embodiments of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0038] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0039] It should be understood that the terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0040] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0041] The display device provided in the embodiments of the present application may have various implementation forms, for example, it may be a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figure 1 and Figure 2 This is a specific implementation of the display device of the present application.

[0042] Figure 1 FIG. 1 is a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control apparatus 100 .

[0043] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wired. The user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, etc.

[0044] In some embodiments, a smart device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop computer, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the smart device.

[0045] In some embodiments, the display device 200 may not use the above-mentioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0046] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0047] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be one cluster or multiple clusters, and may include one or more types of servers.

[0048] like Figure 2 As shown, the display device 200 includes at least one of a tuner and demodulator 210 , a communicator 220 , a detector 230 , a device interface 240 , a controller 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a communication device interface 280 .

[0049] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output.

[0050] The display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from the controller, and display video content, image content, and menu control interface components and user control UI interface.

[0051] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0052] The communicator 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communicators 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communicator 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communicator 220 including a Bluetooth function.

[0053] The communicator 220 can connect the display device 200 to the external device or the server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 to the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 to the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0054] In some embodiments, the display device 200 may also support establishing a communication connection relationship with multiple external devices at the same time. Multiple external devices may be connected through the same. For example, the external device is a terminal device 500 such as a mobile phone or a tablet computer. The first terminal device 510 and the second terminal device 520 may establish a communication connection with the display device 200 by accessing the wireless local area network where the display device 200 is located. Multiple external devices may also be connected to the display device 200 through different types of connection methods. For example, the first terminal device 510 is connected to the display device 200 via a wireless local area network; the second terminal device 520 is connected to the display device 200 via Bluetooth.

[0055] In order to realize the communication connection between the display device 200 and the terminal device 500, the display device 200 and the terminal device 500 are respectively provided with communicators supporting the same communication type. For example, when the display device 200 is provided with a communicator 220 including a WiFi module, the terminal device 500 should also be provided with a communicator including a WiFi module. In addition, when the same type of communicators are provided, a specific communication transmission protocol is also required between the display device 200 and the terminal device 500 for data transmission. For example, WiFi protocol, Bluetooth connection protocol, ZigBee protocol, NFC protocol, etc.

[0056] The audio output interface 270 is used to output the audio signal to other devices to play the audio. In some embodiments, the display device 200 has a built-in speaker, and the audio output interface 270 can input the audio signal to the speaker to play the audio through the speaker. In other embodiments, the display device 200 can be connected to an external audio device such as an amplifier, a speaker, or a loudspeaker through the audio output interface 270, and then the audio signal can be input to the external audio device to play the audio through the external audio device.

[0057] The audio output interface 270 may include, but is not limited to, any one or more of the following interfaces: an RCA port, a butterfly clip port, an SPDIF port, and a headphone port, etc. It may also be a composite output interface formed by the above multiple interfaces.

[0058] The communication device interface 280 may be used to receive a control signal from the control device 100 (eg, an infrared remote controller, etc.).

[0059] The detector 230 is used to collect signals from the external environment or the external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0060] The device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0061] The tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0062] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0063] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0064] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM (Random Access Memory, RAM), a ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0065] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

[0066] "User interface" is the medium interface for interaction and information exchange between applications or operating systems and users. It realizes the conversion between the internal form of information and the form acceptable to users. The commonly used form of user interface is the Graphical User Interface (GUI), which refers to the user interface related to computer operation displayed in a graphical way. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.

[0067] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 includes a controller 110 , a communication interface 130 , a user input / output interface, a memory 190 , and a power supply 180 .

[0068] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0069] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0070] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0071] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0072] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0073] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0074] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0075] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0076] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0077] like Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0078] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0079] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0080] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (NotificationManager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0081] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0082] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0083] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0084] After establishing a communication connection relationship with the terminal device 500, the display device 200 may establish a screen projection connection channel, and transmit the projection data based on the screen projection connection channel, so that the display device 200 can display the projection screen of the terminal device 500. In some embodiments, in order to establish a screen projection connection relationship, the user may send a screen projection connection request through the terminal device 500, and send the screen projection connection request to the display device 200 through the WiFi network. The display device 200 then completes the transmission protocol configuration according to the screen projection connection request, thereby establishing a transmission channel for the projection data with the terminal device 500.

[0085] The display device 200 can also achieve a screen projection connection with the terminal device 500 through different connection methods. For example, when the display device 200 and the terminal device 500 are connected to the same wireless local area network, a screen projection connection can be established based on a WiFi network. For another example, when both the display device 200 and the terminal device 500 are provided with NFC (Near Field Communication) components, a screen projection connection relationship can be established through the NFC component. Obviously, other wired or wireless connection methods can also be used between the display device 200 and the terminal device 500 to establish a screen projection connection relationship, such as RF radio frequency connection, infrared connection, cellular network, and other connection methods that can be associated with those skilled in the art based on the connection methods provided in the above embodiments.

[0086] After the screen projection connection is established, the display device 200 can receive the screen projection related data of the terminal device 500 through the screen projection data channel, so as to play the screen projection related content in the display device 200. Under different screen projection data transmission protocols, the screen projection data obtained by the display device 200 is in different forms. For example, the display device 200 and the terminal device 500 can establish a screen projection connection through the Digital Living Network Alliance (DLNA) protocol, Miracast protocol, AirPlay push protocol, WiDi screen projection, NFC mirror screen projection, LeTV screen projection and other protocols.

[0087] Among them, the DLNA protocol is a protocol for sharing media resources between devices. The protocol uses network technology (wired or wireless) to interconnect various devices and exchange data through standard protocols, so that media resources can be shared between devices. The DLNA protocol is based on the UPnP protocol, which enables devices to discover and control each other and achieve seamless transmission and sharing of media resources. The DLNA protocol supports a variety of devices, including TVs, speakers, mobile phones, computers, etc., allowing users to seamlessly switch and share media content between different devices.

[0088] Miracast protocol is a wireless display standard based on Wi-Fi direct connection and developed by the Wi-Fi Alliance. Miracast screen projection protocol is applicable to Android devices, that is, Android devices can share video images through Miracast screen projection. Smart devices with Android 5.0 or above have built-in Miracast protocol, so you can directly wirelessly project the screen through the WiFi P2P function between the projection device and the WiFi network card of the receiving device without installing any software. In addition, Miracast screen projection also supports synchronous transmission of audio and video.

[0089] Airplay protocol is a wireless playback technology launched by Apple, which is applicable to the wireless screen projection protocol of iOS and Mac systems. Apple devices have built-in Airplay screen projection function. When used with a screen projection device that supports AirPlay server service, the effect is clear and stable. The protocol comes with a screen projection code function. However, it is necessary to ensure that the Apple device and the screen projection device are used in the same LAN (same network segment), and cannot be used across network segments or VLANs.

[0090] WiDi screen projection is a wireless playback technology led by Intel, and is also based on WiFi Direct technology. Windows 8 / 10 / 11 laptops natively support WiDi screen projection, which can be directly projected without installing any software, and supports USB reverse control function.

[0091] In some embodiments, after the display device 200 establishes a screen projection connection with the terminal device 500, the screen projection data can be transmitted through the mirror mode and the push mode. Among them, the mirror mode means that the display device 200 synchronously displays the screen content displayed by the terminal device 500. For example, Figure 5 As shown, the display device 200 displays the main interface of the terminal device 500, and when the user performs an interactive operation on the terminal device 500, resulting in a change in the content of the main interface, the content displayed by the display device 200 also changes. To this end, the terminal device 500 can generate a screen projection data stream in the form of a video based on the displayed content, and send the screen projection data stream to the display device 200 for display.

[0092] The push mode means that the display device 200 accesses the network link according to the network link pushed by the terminal device 500 to obtain the corresponding projection screen. Figure 6 As shown, when the terminal device 500 plays the video media, a screen projection control is provided in the upper right corner of the play interface. After the user clicks the screen projection control and selects the display device 200 as the device for displaying the screen projection image, the terminal device 500 can send the URL address of the currently played video media to the display device 200. After receiving the URL address, the display device 200 accesses the URL address to obtain the video media data and plays the video media.

[0093] Among the screen projection protocols supported by the display device 200 and the terminal device 500, some protocols are based on the mirror mode, and some protocols are based on the push mode. For example, under the DLNA protocol, the display device 200 can obtain the link address of a resource in the terminal device 500 through the screen projection data channel, and obtain the media content data by accessing the link address, thereby playing the media data. At the same time, the display device 200 can also obtain control instructions sent by the terminal device 500 through the screen projection data channel when playing media data, such as fast forward, pause, stop screen projection, etc. For example, after establishing a connection through the communication protocol, the terminal device 500 such as a mobile phone and the display device 200 will negotiate the device functions and network conditions to select a suitable audio and video transmission format. After establishing the online protocol (Session) of the audio and video streaming, a series of real-time streaming protocol (RTSP) control commands are used to control the playback and termination of the audio and video streaming.

[0094] Under other screen projection connection protocols such as Miracast, Airplay, NFC mirroring, and LeTV, the display device 200 can directly obtain the video data stream from the terminal device 500. The video data stream can be the video data stream corresponding to the final display screen of the display unit of the terminal device 500, that is, the video data stream includes the media content screen, UI interface, and other video screen content that can be displayed in the display unit of the terminal device 500. The video data stream can also be only the video data that can be obtained in the terminal device 500, such as the media video data being played. At this time, the terminal device 500 can be used as a signal source for the display device 200, and the display device 200 can continuously obtain video data through the screen projection data channel, and display it in real time on the display 260, so as to achieve the effect of synchronous display with the terminal device 500.

[0095] It should be noted that some screen projection protocols can not only obtain the video data stream in the terminal device 500, but also obtain the audio data stream in the terminal device 500, so that the video data and audio data can be sent to the display device 200 synchronously, so that the display device 200 can play the video and audio synchronously.

[0096] The terminal device 500 can obtain different projection data according to different projection protocols. For example, for system applications or direct projection at the system level, the terminal device 500 can use Miracast projection, Airplay projection or WiDi projection protocol to establish a projection data channel. Since the above-mentioned Miracast projection protocol and Airplay projection protocol require system-level application permissions, the Miracast projection protocol and Airplay projection protocol are only applicable to system applications, or the display device 200 and the terminal device 500 of the same manufacturer directly establish a projection connection through the built-in rules of the operating system.

[0097] The display device 200 can establish a screen projection connection with the terminal device 500 based on any of the screen projection methods provided in the above embodiments. In some embodiments, the display device 200 can also establish a screen projection connection relationship with multiple communication terminals through one or more of the above screen projection methods. For example, the display device 200 can simultaneously establish a screen projection connection with the first terminal device 510 and the second terminal device 520, and divide the display area in the display screen to display the projection screen shared by the first terminal device 510 and the second terminal device 520 respectively.

[0098] In order to display the projection screen shared by multiple terminal devices 500 on the display device 200, in some embodiments, when the display device 200 establishes a projection connection with multiple terminal devices 500, the display device 200 can display a multi-channel projection interface, which can include multiple projection windows. The projection window is used to display the projection screen of the corresponding terminal device 500 under a projection connection channel. For example, Figure 7 As shown, the multi-channel screen projection interface may include two screen projection windows, namely a first screen projection window and a second screen projection window. The first screen projection window is used to display the projection screen of the first terminal device 510; the second screen projection window is used to display the projection screen of the second terminal device 520.

[0099] It should be noted that the projection window in the multi-channel projection interface may not be strongly bound to the terminal device 500, that is, in some embodiments, the display content of the projection window may be determined according to the order in which the terminal device 500 establishes a projection connection with the display device 200. For example, when the first terminal device 510 establishes a projection connection with the display device 200 before the second terminal device 520, the first projection window on the left side of the multi-channel projection interface displays the projection screen of the first terminal device 510. When the first terminal device 510 establishes a projection connection with the display device 200 later than the second terminal device 520, the first projection window on the left side of the multi-channel projection interface displays the projection screen of the second terminal device 520.

[0100] In order to obtain a better picture display effect and improve the coordination of the multi-channel projection interface, in some embodiments, the display device 200 can also set the layout of the projection window. Figure 8 As shown, the display device 200 can have multiple functional modules built in, including: a sound management module, a data management module, a message processing module, a service management module, an algorithm processing module, a view management module, a mode management module, an application interaction processing module, a position management module, a replacement management module, a broadcast control management module, a time management module, a focus management module, and a screen projection management module, etc.

[0101] Among them, the data management module mainly manages all message data involved in the current template, and synchronously updates the data information. The message processing module is mainly responsible for the transmission of related messages between different modules, such as transmitting data changes to the View layer for refresh display, or synchronizing View changes to the data management module for change processing of broadcast control data, etc. The service management module is responsible for the opening and closing of the services of the entire application and some customized processing, such as starting specific services in specific scenarios. The algorithm processing module is responsible for all layout position calculations of the entire application. The View management module is mainly responsible for all UI displays of the entire template application, and receives data to refresh the View display. The sound management module is mainly responsible for the processing of sound in the screen projection, such as the processing of sound when there is no screen projection content to screen projection content, and the processing of sound when there is one screen projection content to two screen projection content.

[0102] The application interaction processing module is responsible for the processing of interaction with external applications. For example, any entry to start the current template application needs to interact with this module, and different customized management is performed according to the difference in the startup entry. The mode management module is responsible for the sound switching rules of the projection window, such as no sound switching when the current sound mode of the projection window is locked. The position management module is responsible for the rules for displaying the projection position in the left and right windows, such as on the left or right. The replacement management module is responsible for the replacement rules when the number of projection paths exceeds the maximum display number, such as how to replace when there is focus or no focus. The broadcast control management module is responsible for the broadcast control processing of different media types in the projection, such as fast forward, fast rewind, play pause, etc. The time management module is responsible for the timing processing of focus display in the projection, whether it is the large frame focus or the broadcast control position focus, or the timing processing of the sound mode switching in the projection. The focus management module is responsible for the focus display and key processing in the entire application. The projection management module is responsible for the projection protocol interaction and the client list management and the status information processing of the window display content.

[0103] It should be noted that the multi-channel screen projection interface can realize screen display based on the multi-channel screen projection viewing application. The multi-channel screen projection viewing application is an application that can display multi-protocol multi-channel mirror projection and push content, and is an upper-level application inside the display device 200. The framework layer of the display device 200 can provide a Java interface for docking with hardware, etc. The multi-channel screen projection viewing application realizes docking with the lower layer and obtains information through the Java interface of the Framework layer. Other National, Runtime, HAL, and Linux kernel layers are all standard layers of the Android architecture, and each layer interacts according to the business process.

[0104] The display device 200 first starts the application through the service management module and performs some initial preparations, such as starting the monitoring mirror or pushing the corresponding protocol service; the View management module initializes the View displayed by the application layer; the data management module creates some data for the data processing module; the algorithm processing module performs algorithmic layout calculations on the projection content, and the position management module is responsible for positioning the left and right positions of the projection content.

[0105] The service management module can start the screen projection application and register the Session during the process of establishing the screen projection connection; Usession Connection Listener, Usession Prepare Listener, Usession DisplayListener and Usession Event Listener. Among them, Session represents a conversation between the client and the server, which can save the client's information. Usession Connection Listener is used to monitor the connection and disconnection events of the Session. Usession Prepare Listener interacts with the screen projection preparation of the Client. Usession DisplayListener monitors the changes in playback information and status and the status changes of the Client. Usession Event Listener processes information such as Client control.

[0106] After the initialization of View is completed, the display device 200 will display the default guide page when entering the application. The guide page will display some device information, such as the device name of the current display device 200 and the corresponding network information. Then, the guide page will be displayed in equal proportion to the left and right by controlling the standard control TextView and ImageView of the system and the corresponding Layout, such as Fig. 9 shown.

[0107] After displaying the guide page, the display device 200 can monitor and receive the screen projection connection instruction sent by the terminal device 500, and establish a screen projection connection channel according to the screen projection connection instruction, so as to display it in the screen projection window. For example, the screen projection management module is always ready for the access of a new Session. When Client1 is ready to start the connection, the screen projection management module monitors the onConnect connection of the new Session through UsessionConnectionListener, and then creates the playback control SurfaceView / TextureView. After the creation is successful, the position management module is started to set and manage the display position of the current Session. The position management module processes and obtains the position. The first screen projection is displayed on the left by default. The mViewPosition of Client1 is marked as 1, where 1 represents the left side and 2 represents the right side. It can also be represented by other parameters.

[0108] To facilitate the management of View and Session, in some embodiments, a management object MultiCastPlayView can be set, wherein the management object MultiCastPlayView can be associated with information such as Session, SurfaceView / TextureView, and mViewPosition. For example, for the first terminal device 510, the data corresponding to its Client1 can be represented as: Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1.

[0109] When the display device 200 prepares the View as the server, it can notify the terminal device 500 as the mobile terminal to record or send the video stream through UsesessionPrepareListener. When the terminal device 500 is ready to send video information, the display device 200 knows that it will be cast through onFirstFrame() of UsesessionDisplayListener, so it can get the current width and height of Client1's video from the Session, determine the direction of the current video stream through the width and height, and then calculate the current position and size to be displayed through the algorithm.

[0110] That is, the first projection window and the second projection window can also present corresponding shapes according to the size of the displayed projection screen. For example, when the first projection screen and the second projection screen are both 1920×1080 horizontal screens, the display device 200 can present the first projection window and the second projection window with a size of 960×540 in the multi-channel projection interface.

[0111] In some embodiments, in order to distinguish the display range of the projection area, a certain blank area may be left in the first projection window and the second projection window, the projection window and the guide area. For example, the blank area is 12 pixels, and the resolution of the display device 200 is 1920×1080. Fig.10 As shown, when the first terminal device 510 (Client 1) is displayed in horizontal format, the size of the projection screen in the corresponding projection video stream is 1920×1080, and the guide page display size is 954×538. At this time, the video stream ratio and the right guide video stream ratio are close to 16:9 and are both horizontal. It can be calculated through the algorithm that the left and right sides can be displayed in equal proportion, that is, they can be displayed with equal width and height, that is, the width of the video stream and the guide page is 954, and the height is about 538.

[0112] Since the terminal device 500 may be placed in different states according to different usage modes, for example, a terminal device 500 such as a smart phone may be in a portrait state and a landscape state during use. The portrait state refers to a state in which the width of the displayed image is less than the height, and the landscape state refers to a state in which the width of the displayed image is greater than the height. In different placement states, the terminal device 500 may present different screen layouts.

[0113] Based on this, in some embodiments, when the display device 200 displays the projection screens of multiple terminal devices 500 by area division, each display area will present different screen ratios due to the different placement states of the terminal devices 500. For example, the first terminal device 510 is in a horizontal screen state, the first projection screen size is 1920×1080, and the second terminal device 520 is in a vertical screen state, and the second projection screen size is 512×1088. Then the first projection screen window can be 954×538, and the second projection screen window can be 508×1080.

[0114] In order to make full use of the display space of the multi-way projection interface and reduce large black screen areas, in some embodiments, when the sizes of the first projection screen and the second projection screen are inconsistent, the display device 200 can set the width or height of the first projection window and the second projection window to be consistent, and scale the projection windows according to the width or height.

[0115] For example, when the first terminal device 510 is in portrait mode, the size of the first projection screen is 512×1088; the second terminal device 520 is in landscape mode, the size of the second projection screen is 1920×1080, and the display resolution of the display device 200 is 1920×1080, the default size of the first projection window and the second projection window (i.e., the guide page display size) is 954×538. At this time, the video stream ratio is different from the right guide video stream ratio and the direction is different. It can be calculated through the algorithm that the left video stream display height is about 848 and the width is about 400. Excluding the blank area, the remaining width is the guide page width, and the height is consistent with the video stream.

[0116] To this end, the display device 200 can set the parameters of the first projection window, namely Client 1: width1 (width), height1 (height); Guide (guide page): width_guide, height_guide. The height of the projection screen video stream from the upper edge of the screen is set to top, because height1 and height_guide are ultimately the same height, and can be replaced by the same variable height. The video stream ratio ratio and the guide page ratio ratio_guide are both known constants. Through the following combined calculations, width1, top, height, and width_guide can be calculated; that is, width1+width_guide=1920; top*2+height=1080; width1 / height=ratio1; width_guide / height=ratio_guide.

[0117] Based on the above algorithm, the algorithm processing module of the display device 200 can obtain the corresponding width and height position of the content displayed on the left and right sides. That is, the width and height of Client1 can be calculated to be 848 and 400, and the corresponding position coordinates are (0, 116) from the upper left corner, and the guide page coordinates are (412, 116).

[0118] After the above position calculation is completed, the data management module of the display device 200 can sort and store, and then transmit the data, including: Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1, Position: (0, 116), Width: 400, Height: 848; Session: View: layout, MViewPosition: Position: (412, 116), Width: 1508, Height: 848. After receiving the data information, the View management module of the display device 200 sets the display parameters through LayoutParams, including the position from the upper boundary and the left boundary (i.e., coordinates), as well as the corresponding display width and height, and then displays the position through the setLayoutParams method. That is:

[0119] FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width, height, 0); lp1.width=400; lp1.height=848; lp1.leftMargin=0; lp1.topMargin=116; mPlayerView, setLayoutParams(lp1);

[0120] FrameLayout.LayoutParamslp1=newFrameLayout.LayoutParams(width, height, 0); lp1.width=1508; lp1.height=848; lp1.leftMargin=412; lp1.topMargin=116; mPlayerView, setLayoutParams(lp1).

[0121] By setting the display position of the projection SurfaceView / TextureView and the display position of the guide Layout respectively, you can get the following Fig.11 The display effect of the multi-channel screen projection interface is shown.

[0122] In some embodiments, when the display device 200 displays the multi-channel projection interface, it can also display the broadcast control function control in the multi-channel projection interface, and the user can perform interactive operations through the broadcast control function control to input control instructions to control the display device 200 to perform interactive responses. For example, since the projection contents of the display device 200 and the terminal device 500 are various, there are mirrored video streams, pushed URL addresses, and different media types such as pictures and music, the broadcast control function controls of different media types are also different. For example, the mirrored video projection does not support fast forward and rewind operations nor does it support play and pause operations; the pushed URL address can fast forward, rewind, play, pause, etc.; the music can fast forward, rewind, play, pause, etc.; the picture does not support fast forward, rewind, play, pause, etc.

[0123] The display device 200 can dynamically adjust the broadcast control function controls according to the display area of ​​the projection window in the multi-channel projection interface, so that the display position of the broadcast control function controls is adapted to the projection window to facilitate the user to perform interactive operations. For example, the broadcast control function controls belonging to the same projection window can form a broadcast control list. The broadcast control list includes several broadcast control function controls such as full screen, fast forward, fast rewind, play / pause, mute / unmute, and close. The size of each broadcast control function control is 60×60, and the horizontal spacing of the broadcast control function controls is 24. Combined with the left and right margins of the broadcast control list, the minimum display width for displaying six broadcast control function controls is 528, the minimum display width for displaying three broadcast control function controls is 276, the minimum display width for displaying two broadcast control function controls is 192, and the minimum display width for displaying one broadcast control function control is, and the same applies to other numbers.

[0124] In some embodiments, when the screen projection management module listens to the onConnect connection of the new Session through UsessionConnectionListener, the media resource type of the current Client can be obtained from the Session, such as VIDEO_URL\MIRROR_STREAM\MUSIC\PIC, etc. For example, if the currently displayed VIDEO_URL address is pushed, the display device 200 can perform full screen, fast forward, fast rewind, play / pause, mute / unmute, close and other operations. At this time, the broadcast control data can be added to this Session, that is, the type data of each broadcast control function control can be expressed as: DoublePushControlBean[mStatus=××, mType=××, mResId=××]. Based on this, the entire broadcast control list is the List list of the above-mentioned DoublePushControlBean, and the corresponding data can be expressed as: DoublePushControlListBean[List <doublepushcontrolbean>=××, Status=××].

[0125] For example, when there are 6 playback control function controls below the projection window, the corresponding data format is:

[0126] Full screen button control: DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132];

[0127] Rewind button control: DoublePushControlBean[mStatus=true, mType=rewind, mResId=2131231110];

[0128] Play button control: DoublePushControlBean[mStatus=true, mType=playstatus, mResId=2131231149];

[0129] Fast forward button control: DoublePushControlBean[mStatus=true, mType=forward, mResId=2131231111];

[0130] Mute button control: DoublePushControlBean[mStatus=true, mType=mutestatus, mResId=2131231152];

[0131] Close button control: DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121].

[0132] In order to display the above-mentioned broadcast control function control, after the data management module obtains the display size of the window, the View management module performs typesetting and display as follows Fig.12 The effect shown in the figure is that the left window is the URL address media resource, which can be played and controlled. The algorithm processing module calculates that the left window size is 1508×848 and the right window size is 400×848. The following data results are obtained:

[0133] Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1, Position: (0, 116), Width: 1508, Height: 848, mDoublePushControlListBean:…

[0134] Session: SessionClient2, View: SurfaceView / TextureView, MViewPosition: 2, Position: (1520, 116), Width: 400, Height: 848, mDoublePushControlListBean:…

[0135] After calculating the window display position, the View management module will layout and display the left and right windows. During the display process, the playback control View needs to be initialized and displayed according to the playback control list data of the Session. For example, the display device 200 can use the Recycleview control for display, and the playback control menu is displayed by setting the margin and position of the Recycleview. The display parameters are also set through LayoutParams, including the position from the upper boundary and the left boundary, i.e., the coordinates, and the corresponding display width and height, and then the position is displayed through the setLayoutParams method.

[0136] After calculation by the algorithm processing module, the display area width 1508 of Client1 is larger than the minimum width 528 of the play control bar, so it can be fully displayed. Fig.12 The display effect can fully display the six broadcast control function controls in the center, or display them in any other position. Users can jump to the broadcast control function controls and operate them by pressing the down button on the remote control.

[0137] When the media content of the VIDEO_URL type changes or the direction of the video stream changes, the display device 200 can rearrange the display of the projection content, such as Fig.13 The left window shown is a VIDEO_URL type media asset, but the algorithm processing module calculates that the left window size is 459×816 and the right window size is 1449×816. The following data results are obtained:

[0138] Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1, Position: (0, 132), Width: 459, Height: 816, mDoublePushControlListBean:…

[0139] Session: SessionClient2, View:, MViewPosition:, Position: (1520, 132), Width: 1449, Height: 816, mDoublePushControlListBean:…

[0140] After calculation by the algorithm processing module, the display area width 459 of Client1 is less than the minimum width 528 of the playback control bar, but greater than 276, so some playback control function controls can be displayed, such as Fig.13 The display effect can fully display the three playback control function controls in the center, or display them in any other position. You can jump to the playback control function controls by pressing the down button on the remote control to operate them. That is, display the three button controls of full screen, mute, and close. The data format is as follows:

[0141] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132];

[0142] DoublePushControlBean[mStatus=true, mType=mutestatus, mResId=2131231152];

[0143] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121].

[0144] When the window size of the projection window changes, the display device 200 can notify the Recycleview to refresh the View through the notifyDataSetChanged of the Adapter, changing from 6 buttons to 3 buttons.

[0145] like Fig.12 As shown in the playback control list, when the user clicks the third playback control button, Recycleview will monitor the user's onClick operation, and for the index type of the playback control list, that is, the mType type is playstatus, the setPlayerPause() interface of the Session is called to process the playback control. Similarly, fast forward and rewind also call the setPlayerSeekBackward() and setPlayerSeekForward() of the Session to process the playback control.

[0146] The screen projection management module monitors the screen projection session status. When the video changes, the algorithm processing module is notified through the message processing module. The algorithm processing module calculates the screen projection display area and then informs the data management module through the message processing module. After the data management module organizes the data, it notifies the View management module through the message processing module to redraw and display the View.

[0147] Similarly, when the projection management module listens to the onConnect connection of the new Session through UsessionConnectionListener, it can obtain from the Session that the media resource type of the current Client is MIRROR_STREAM, then the large screen can be full screen, mute / unmute, and closed. At this time, the broadcast control data can be added to this Session as follows: DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132];

[0148] DoublePushControlBean[mStatus=true, mType=mutestatus, mResId=2131231152];

[0149] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121].

[0150] According to the data from the data management module, the right window size is 400×848, and the size of the three controls is 276. Since the right window width 400 is larger than the width 276 occupied by the three controls, the three mirrored play control buttons can be fully displayed. Then the Recycleview is initialized and displayed. The display parameters are set through LayoutParams, including the position from the upper and left boundaries, i.e. the coordinates, and the corresponding display width and height. Then the position is displayed through the setLayoutParams method. The corresponding play control list data is modified to Session: SessionClient2, View: SurfaceView / TextureView, MViewPosition: 2, Position: (1520, 116), Width: 400, Height: 848, mDoublePushControlListBean:…

[0151] It can be seen that in the above embodiment, the display device 200 can adjust the number of button controls from six to three for display when the width of the projection window cannot meet the arrangement of six button controls. However, when the width of the projection window cannot meet the three broadcast control button controls, it is necessary to continue to adjust the number of broadcast control buttons according to the projection window. If the right window is 258×864, the window width is less than the minimum display width of 276 when three button controls are displayed, and greater than the minimum display width of 192 when two button controls are displayed. Therefore, the display device 200 can display two broadcast control button controls below the projection window. In addition, since the volume can be mutually exclusive controlled by the other party's buttons, only the full screen and close button controls can be retained, that is:

[0152] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132];

[0153] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121].

[0154] Similarly, if the right window size is 156×864, the window width is less than the minimum display width of 192 required for two button controls, but greater than the minimum display width of 108 required for a button control, so one button can be displayed. Since closing the projection window can be controlled by stopping the projection operation on the terminal device 500, only the full screen button can be retained, that is: DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132].

[0155] By analogy, if the right window size is 88×864, the window width is less than the display width 108 required for a button control, so the play control button cannot be displayed. Therefore, the display device 200 can cancel the display of the play control button control below the right window. It should be noted that the display method of the play control button control provided in the above embodiment is only an example. When the play control button control cannot be fully displayed horizontally, the display device 200 can also display it by vertical arrangement, can be arranged and displayed in a rectangular array or a circular array, or can be displayed by edge sliding, but the basic display principle is universal and is not limited to the above expression method.

[0156] In some embodiments, based on the screen projection data transmission protocol, the display device 200 can not only obtain video data, but also obtain audio data, so that the display device 200 can synchronously play video and audio based on the audio output interface 270.

[0157] In some embodiments, the display device 200 can control the audio playback of multiple screen projections by executing a multiple screen projection sound playback control method. The multiple screen projection sound playback control method can be applied to the display device 200 provided in the above embodiment. In order to meet the implementation of the multiple screen projection sound playback control method, the display device 200 should at least include a display 260, a communicator 220 and a controller 250.

[0158] The display 260 is configured to display a multi-channel screen projection interface, which includes at least two screen projection windows, and the screen projection window is used to display the projection screen of the terminal device. The communicator 220 is configured to establish a screen projection connection with at least two terminal devices. The audio output interface 270 is configured to play audio data, and the controller 250 is configured to execute the program steps corresponding to the multi-channel screen projection sound broadcast control method, which specifically includes the following contents:

[0159] The display device 200 can maintain a global projection list (SessionList) for managing session access and removal operations, that is, the projection list is used to record the terminal devices 500 that establish projection connections with the display device 200, and the connection order in which the terminal devices 500 establish projection connections with the display device 200. The display device 200 can adjust the sound mode of multiple projections through the projection list, set the sound mode of a terminal device 500 in the projection list to a non-mute mode, and set other terminal devices 500 to a mute mode, so that the display device only plays the sound of one projection.

[0160] Among them, the sound management module of the display device 200 can set an audio parameter when setting the sound mode. When the sound mode is non-silent mode, the audio parameter can be a first parameter value, and the first parameter value is used to indicate that audio data is allowed to be played. When the sound mode is silent mode, the audio parameter can be a second parameter value, and the second parameter value is used to indicate that audio data is not allowed to be played. This is used to characterize various logics and display processing during the sound mode switching process.

[0161] Setting the sound mode of a terminal device 500 in the screen projection list to silent mode can be achieved in a variety of ways. After the display device 200 establishes a screen projection connection with the terminal device 500, when the display device 200 obtains the screen projection data containing video data and audio data, the audio data can be filtered out and only the video data is received, thereby muting the sound of the screen projection. The display device 200 can also receive video data and audio data, but does not play the audio data through the audio output interface 270, thereby muting the sound of the screen projection. The display device 200 can also, based on the screen projection protocol, cause the terminal device 500 that establishes a screen projection connection with the display device 200 not to send audio data, thereby muting the sound of the screen projection. For example, in some screen projection protocols, it can be set to transmit only video data, so that the terminal device 500 only sends video data and does not send audio data.

[0162] In some embodiments, the display device 200 can set the sound mode of the first projection connection to a non-silent mode, that is, the terminal device 500 (Client) that takes the priority to occupy the sound holds the sound. Or set the sound mode of the last projection connection to a non-silent mode, that is, the terminal device 500 (Client) that has the latest projection connection holds the sound.

[0163] like Fig.14 As shown, it is a flow chart of the sound playback control provided in the embodiment of the present application. When the display device 200 has no screen projection connection, the display device 200 can set the audio parameter of the first terminal device to the first audio parameter in response to a connection event of establishing a screen projection connection relationship between the display device 200 and the first terminal device. When the display device 200 has a screen projection connection, the display device 200 can set the audio parameter of the first terminal device to the second audio parameter in response to a connection event of establishing a screen projection connection relationship between the display device 200 and the second terminal device, and set the audio parameter of the second terminal device to the first audio parameter, or set the audio parameter of the second terminal device to the second audio parameter. Among them, the first terminal device is a terminal device 500 whose current audio parameter is the first parameter value.

[0164] For example, when UsessionConnectionListener listens to a session connection, SessionList adds Client1, and sets the audio mode of Client1 to non-mute mode through Session's setAudioOutputState, that is, the first screen projection makes sound. At this time, if UsessionConnectionListener listens to another session connection, SessionList adds Client2, because the first screen projection is sound-enabled, and if both channels make sound at the same time, it will cause sound confusion. Therefore, after the second session is connected, the audio mode of Client2 is set to mute mode through Session's setAudioOutputState, that is, the second screen projection does not make sound.

[0165] In order to facilitate users to distinguish between the projection end that makes sound and the projection end that does not make sound, the display device 200 can create a sound view in response to a connection event that establishes a projection connection relationship between the display device 200 and the terminal device 500. The sound view is a control for performing playback control on audio data in the projection interface. The sound view may include a first icon and a second icon, wherein the first icon is used to indicate a non-silent mode and the second icon is used to indicate a silent mode. When the sound mode of the terminal device 500 in the projection list is a non-silent mode, the control display 260 displays the first icon of the sound view, and when the sound mode of the terminal device 500 in the projection list is a silent mode, the control display 260 displays the second icon of the sound view, thereby intuitively presenting the sound mode of the current projection to the user through the icon of the sound view.

[0166] In order to facilitate the switching of the sound view icon, the state identifier of the sound view can be associated with the audio parameter of the terminal device 500, that is, the display device 200 can read the audio parameter of the terminal device 500 in the screen projection list, and if the audio parameter is the first parameter value, the state identifier of the sound view is set to the first identifier value. If the audio parameter is the second parameter value, the state identifier of the sound view is set to the second identifier value.

[0167] When displaying the sound view, the display device 200 can read the state identifier of the sound view, and if the state identifier is a first identifier value, control the display 260 to display the first icon of the sound view. If the state identifier is a second identifier value, control the display 260 to display the second icon of the sound view.

[0168] For example, taking the above embodiment as an example, when UsessionConnectionListener monitors a Session connection, SessionList adds Client1, and sets the audio mode of Client1 to non-mute mode through Session's setAudioOutputState, that is, the first projection makes sound. Fig.15 As shown, the first projection content is displayed through the left projection window, and according to the display area of ​​the left projection window, the full screen, off, mute / non-mute, play / pause, fast forward, and fast rewind playback control views are created and displayed. Among them, since the sound mode corresponding to the projection of the left projection window is non-mute mode, the status identifier of the sound view representing "mute / non-mute" in the playback control view is set to the first identifier value, that is, mStatus = false, indicating that the current mode is non-mute mode, that is, the playback control list data of Client1 is as follows:

[0169] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132],

[0170] DoublePushControlBean[mStatus=true, mType=rewind, mResId=2131231110],

[0171] DoublePushControlBean[mStatus=true, mType=playstatus, mResId=2131231149],

[0172] DoublePushControlBean[mStatus=true, mType=forward, mResId=2131231111],

[0173] DoublePushControlBean[mStatus=false, mType=mutestatus, mResId=2131231152],

[0174] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121]

[0175] At this time, if UsessionConnectionListener listens to another Session connection, SessionList will add Client2, because the first projection is sound-enabled, and if both channels make sound at the same time, it will cause sound confusion. Therefore, after the second Session is connected, the sound mode of Client2 will be set to silent mode through Session's setAudioOutputState, that is, the second projection will not make sound. Fig.15 As shown, the second projection content is displayed through the right projection window, and according to the display area of ​​the right projection window, the full screen, off, mute / non-mute, play / pause, fast forward, and fast rewind playback control views are created and displayed. Among them, since the sound mode corresponding to the projection of the right projection window is the silent mode, the status flag of the sound view representing "mute / non-mute" in the playback control view is set to the second flag value, that is, mStatus = true, indicating that the current mode is silent, that is, the playback control list data of Client2 is as follows:

[0176] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132],

[0177] DoublePushControlBean[mStatus=true, mType=rewind, mResId=2131231110],

[0178] DoublePushControlBean[mStatus=true, mType=playstatus, mResId=2131231149],

[0179] DoublePushControlBean[mStatus=true, mType=forward, mResId=2131231111],

[0180] DoublePushControlBean[mStatus=true, mType=mutestatus, mResId=2131231152],

[0181] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121]

[0182] According to the above-mentioned playback control list data of Client1 and Client2, the icon display status of the view of the mutestatus type in the Recycleview of the left and right projection windows is set respectively, and the display of the playback control view is set by setBackgroundResource, as shown below. Fig.15 The display effect shown, wherein the sound view below the left projection window displays the first icon, and the sound view below the right projection window displays the second icon.

[0183] When the display device 200 displays the sound view, the user can control the sound mode of the corresponding projection window based on the sound view. That is, the display device 200 can monitor the user's operation through the onClick event of Recycleview, and mute / unmute the selected Client according to the user's choice, and perform the opposite operation on the unselected Client.

[0184] For example, Fig.15 In the sound view shown in , the user can use the direction keys on the remote control compatible with the display device 200 to control the focus cursor to move to the "mute / unmute" playback control function button below the left projection window, and press the "confirm" key on the remote control to enter the mute command. At this time, Recycleview will monitor the user's onClick operation, set the sound mode of Client1 from non-mute mode to mute mode through Session's setAudioOutputState, and set the sound mode of Client2 from mute mode to non-mute mode, and at the same time set the display of the playback control view through setBackgroundResource, showing as follows Fig.16 The display effect shown, wherein the sound view below the left projection window displays the second icon, and the sound view below the right projection window displays the first icon.

[0185] In some embodiments, when a screen projection whose sound mode is not in silent mode in the projection list is disconnected from the projection connection, the sound can be dynamically switched according to the projection list. That is, the display device 200 can remove the first terminal device from the projection list in response to an exit event that disconnects the projection connection relationship between the display device 200 and the first terminal device, and determine the third terminal device in the projection list according to the connection order, and set the audio parameter of the third terminal device to the first parameter value. Among them, the first terminal device is the terminal device 500 in the projection list whose current audio parameter is the first parameter value.

[0186] For example, Fig.15 Take the two-way screen projection as an example. The first screen projection Client1 has sound. At this time, Client1 is disconnected and Client1 is removed from SessionList. Only Client2 is left in SessionList. At this time, Client2 can be unmuted. First, the sound of the second screen projection is unmuted through the setAudioOutputState method of Session in the screen projection management module. The message management module notifies the data management module of this message. The data management module updates the playlist data of Client2. The updated data is as follows:

[0187] DoublePushControlBean[mStatus=false, mType=mutestatus, mResId=2131231152]

[0188] After the data update is completed, the message processing module notifies the View management module of the message, and the View management module performs dynamic refresh display processing on the Recycleview, presenting the following Fig.17 The display effect shown.

[0189] The above implementation can realize dynamic switching of sound, but in some scenarios, the user does not want to switch the sound, for example, Fig.15 Taking the two-way screen projection as an example, the first screen projection Client 1 is in non-mute mode, and the second screen projection Client 2 is in mute mode. During the screen projection process, if the media type of the first screen projection Client 1 is push type, it may switch the episodes of the film and television series. During the episode switching process, according to the differences between different mobile terminals, there will be a situation of switching sessions, which is equivalent to a new Client access. There are also cases where some mobile terminals perform mirror projection and then switch to push type. In this case, there will also be a new Session access, which is equivalent to a new Client access.

[0190] In the above scenario, according to the previous processing logic, after the first projection Client 1 is disconnected, the display device 200 will unmute the second projection Client 2 to switch the sound. When UsesionConnectionListener monitors a new Session connection, based on the principle of giving priority to the projection that holds the sound, the display device 200 may mute the newly connected third projection Client 3. In other words, according to the above processing logic, the sound mode may be switched from non-muted mode to silent mode after the projection switches the episode number or changes the media type. But for the user, the user's goal is to continue playing according to the previous sound mode after the projection switches the episode number or changes the media type, that is, the third projection Client 3 continues to use the sound mode of the first projection Client 1.

[0191] Therefore, the display device 200 may set a time threshold, and the sound may not be switched within the time threshold of the exit event, and the sound may be switched after the time threshold of the exit event. Fig.18 As shown, the display device 200 can remove the first terminal device from the screen projection list in response to an exit event that disconnects the screen projection connection relationship between the display device 200 and the first terminal device, and record the exit time of the exit event.

[0192] If a connection event for establishing a screen projection connection relationship between the display device 200 and the second terminal device is detected within a preset time threshold after the exit time, the second terminal device is added to the screen projection list, and the audio parameter of the second terminal device is set to the first parameter value. After the audio parameter of the second terminal device is set to the first parameter value, if a connection event for establishing a screen projection connection relationship between the display device 200 and the fourth terminal device is detected, the processing can continue according to the pre-set sound processing logic, that is, the display device 200 can respond to the connection event for establishing a screen projection connection relationship between the display device 200 and the fourth terminal device, set the audio parameter of the second terminal device to the second audio parameter, and set the audio parameter of the fourth terminal device to the first audio parameter. Alternatively, set the audio parameter of the fourth terminal device to the second audio parameter.

[0193] If a connection event for establishing a screen projection connection relationship between the display device 200 and the second terminal device is not detected within a preset time threshold after the exit time, the third terminal device is determined in the screen projection list according to the connection order, and the audio parameter of the third terminal device is set to the first parameter value.

[0194] Among them, when determining the third terminal device in the projection list according to the connection order, multiple methods can be included. The display device 200 can select the terminal device 500 that establishes a projection connection with the display device 200 earliest in the projection list according to the connection order as the third terminal device. Alternatively, the terminal device that establishes a projection connection with the display device 200 latest in the projection list is selected as the third terminal device according to the connection order.

[0195] For the broadcast control list data, if the media type of the currently exited screen projection is the same as the media type of the newly connected screen projection within the time threshold, the broadcast control list data can be directly reused. That is, the display device 200 can record the first media type and the broadcast control list of the screen projection data of the first terminal device in response to the exit event that disconnects the screen projection connection relationship between the display device 200 and the first terminal device, and the broadcast control list includes the broadcast control data of the broadcast control view associated with the first media type. If a connection event is detected to establish a screen projection connection relationship between the display device 200 and the second terminal device within the preset time threshold after the exit time, the second media type of the screen projection data of the second terminal device is obtained. If the first media type is the same as the second media type, the broadcast control list is set for the second terminal device.

[0196] For example, Fig.15 Take the two-way screen projection as an example. When Client1 is disconnected through UsessionConnectionListener, SessionList will remove Client1. After removal, the timer CountTimeUtils is started and the duration is set to 2s. The sound switching method is locked within 2s, and the sound switching operation of the left and right screen projection windows will not be performed, that is, the setAudioOutputState method of Session is invalid. If a new Client3 is connected within 2S, SessionList will add Client3 and perform View layout operations according to Client3's VideoChanged. At the same time, Client3 continues to use the original sound mode of Client1.

[0197] If the media asset type of Client3 is the same as that of Client1, the play control list data is reused. The play control list data corresponding to Client3 is as follows:

[0198] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132],

[0199] DoublePushControlBean[mStatus=true, mType=rewind, mResId=2131231110],

[0200] DoublePushControlBean[mStatus=true, mType=playstatus, mResId=2131231149],

[0201] DoublePushControlBean[mStatus=true, mType=forward, mResId=2131231111],

[0202] DoublePushControlBean[mStatus=false, mType=mutestatus, mResId=2131231152],

[0203] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121]

[0204] If UsessionConnectionListener has not detected any new Client3 access after the 2S timer ends, the sound of Client2 in SessionList will be reset, and the sound mode of Client2 will be set to non-silent mode through the setAudioOutputState method of Session. The data will be synchronized to the data management module for updating, and will be synchronized to the View management module through the message processing module to refresh the View display.

[0205] In order to improve the display effect, the display device 200 can also record the display position parameters of the first projection window in response to the exit event of disconnecting the projection connection relationship between the display device 200 and the first terminal device, wherein the first projection window is a projection window that displays the projection screen of the first terminal device. If a connection event for establishing a projection connection relationship between the display device 200 and the second terminal device is detected within a preset time threshold after the exit time, the display 260 is controlled to display the projection screen of the second terminal device in the first projection window according to the display position parameters.

[0206] For example, Fig.17 As shown in the figure, the media resource screen of the second projection Client2 is displayed through the right projection window. When the UsessionConnectionListener detects that Client2 is disconnected, SessionList will remove Client2 and record the display position parameters of the right projection window. After removal, the timer CountTimeUtils is started and the duration is set to 2s. If a new Client3 is connected within 2S, SessionList will add Client3 and set the display position of Client3 according to the recorded display position parameters, so that the media resource screen of Client3 is displayed through the right projection window.

[0207] In some embodiments, the display device 200 also supports locking the sound mode of the projection window. After the sound mode of a projection window is locked, the projection window can always maintain the sound mode before the current lock without switching the sound. In this regard, the projection management module of the display device 200 performs monitoring management of the Session and sets the sound mode through the sound management module. When the user operates the View, the user action is monitored by the View management module, and then the View is refreshed. At the same time, the data management module is notified through the message processing module to update the data. When the projection management module monitors the exit event of the projection connection relationship, the time management module is notified through the message processing module to supervise the time to determine whether to switch the sound. At the same time, the above operations are all dynamic sound switching when the mode management module notifies the current projection window that the sound mode is not locked. If the sound mode of the current projection window is locked, the locked projection window will not switch the sound, and the projection window on the other side will always maintain the default sound mode for display, and the sound mode will not be switched.

[0208] Therefore, the display device 200 can respond to the lock instruction of the target projection window, record the current target audio parameters of the target projection window, and set a lock value for the target projection window. The lock value indicates that the window state of the target projection window is in a locked state, which is used to indicate that the audio parameters of the terminal device that displays the projection screen in the target projection window are set to the target audio parameters.

[0209] When the display device 200 receives a display instruction to display the projection screen of the target terminal device in the target projection window, it can respond to the display instruction and read the lock value of the target projection window to detect the window state of the target projection window. If the window state is locked, the target audio parameters of the target projection window are obtained, and the audio parameters of the target terminal device are set to the target audio parameters. The target audio parameters are the audio parameters of the terminal device whose projection screen is displayed in the target projection window when the window state is set to the locked state.

[0210] For example, taking the above Fig.16 Take the two-way projection as an example. The sound mode of the first projection Client1 is silent mode, and the second projection Client2 is non-silent mode. At this time, the user long presses Fig.16 The "sound" playback control function control under the left projection window shown in the figure is used to input the lock command of the left projection window. The display device 200 can respond to the lock command to lock the current mute mode of the left projection window and set the mBlockStatus in the DoublePushControlBean of the current mutestatus type to true. That is, after the View management module monitors the user's long press action, it refreshes the View and presents the following Fig.19 The display effect shown in the figure shows the third icon in the sound view below the left projection window. Then the data management module is notified of this operation through the message processing module. The data management module sets mBlockStatus to true. After the setting is successful, the sound mode of the locked left projection window will be memorized, that is, the global memory mIsPlayControlVoiceBlocked is true and mIsPlayControlVoiceBlockedLeft is true. The play control list data of Client1 is as follows:

[0211] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132],

[0212] DoublePushControlBean[mStatus=true, mType=rewind, mResId=2131231110],

[0213] DoublePushControlBean[mStatus=true, mType=playstatus, mResId=2131231149],

[0214] DoublePushControlBean[mStatus=true, mType=forward, mResId=2131231111],

[0215] DoublePushControlBean[mStatus=true, mType=mutestatus, mBlockStatus=true, mResId=2131231152],

[0216] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121]

[0217] After the above operation, the sound mode of the left projection window will not change regardless of whether there are new clients or disconnected clients. If Client2 is disconnected, remove Client2 from SessionList. At this time, although only Client1 in silent mode remains in SessionList, the sound will not be reset because mIsPlayControlVoiceBlocked is true.

[0218] And when the media image of the newly connected Client4 is displayed in the left projection window, because mIsPlayControlVoiceBlocked and mIsPlayControlVoiceBlockedLeft are true, the sound mode of Client4 is still silent mode. The specific data is as follows:

[0219] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132],

[0220] DoublePushControlBean[mStatus=true, mType=rewind, mResId=2131231110],

[0221] DoublePushControlBean[mStatus=true, mType=playstatus, mResId=2131231149],

[0222] DoublePushControlBean[mStatus=true, mType=forward, mResId=2131231111],

[0223] DoublePushControlBean[mStatus=true, mType=mutestatus, mBlockStatus=true, mResId=2131231152],

[0224] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121]

[0225] In some embodiments, when the screen projection is switched from a small window to a full-screen window, the sound also needs to be processed logically accordingly. The display device 200 can respond to a full-screen instruction for full-screen display of the target screen projection window, delete other terminal devices other than the target terminal device in the screen projection list, and disconnect the screen projection connection with other terminal devices. Among them, the target terminal device is a terminal device whose screen projection screen is displayed in the target screen projection window. Then read the audio parameters of the target terminal device. If the audio parameter is the second audio parameter, set the audio parameter of the target terminal device to the first audio parameter, and display the target screen projection window in full screen. If the audio parameter is the first audio parameter, display the target screen projection window in full screen.

[0226] For example, taking the above Fig.15 Taking the two channels shown in the figure as an example, the sound mode of the first channel, Client 1, is non-mute mode, and the second channel, Client 2, is mute mode. Fig. 20 As shown, the user clicks Fig.15 The "full screen" playback control function control below the right projection window is shown. First, remove Client1 from the SessionList, and then remove the View corresponding to Client1 from the ViewList of the display window. Then reset the display window and set the display area and position of Client2 according to the window size. At the same time, use the setAudioOutputState method of Session to unmute Client2, and the following is displayed: Fig. 20 The display effect shown.

[0227] It should be noted that the specific solutions for sound processing are not only the above-mentioned examples, but multiple projection windows and multiple projection protocols can also refer to the above rules. In addition, sound processing can not only perform muting / unmuting processing, but also set the volume, such as entering the secondary menu through the sound view or adding other display buttons to adjust the volume.

[0228] Based on the multi-channel screen projection sound broadcast control method provided in the above embodiments, some embodiments of the present application also provide a display device 200, the display device 200 includes a display 260, a communicator 220 and a controller 250, the display 260 is configured to display a multi-channel screen projection interface, the multi-channel screen projection interface includes at least two screen projection windows, the screen projection windows are used to display the projection screen of the terminal device 500; the communicator 220 is configured to establish a communication connection with at least two terminal devices 500; the audio output interface 270 is configured to play audio data. The controller 250 is configured to perform the following program steps:

[0229] Control the audio output interface 270 to play the audio data of the terminal device whose audio parameter in the screen projection list is the first parameter value, wherein the audio parameter is used to indicate whether the audio data is allowed to be played or not, the first parameter value is used to indicate that the audio data is allowed to be played, and the screen projection list is used to record the terminal device 500 that establishes a screen projection connection with the display device 200, and the connection order of the terminal device 500 and the display device 200 to establish a screen projection connection.

[0230] In response to an exit event of disconnecting the projection connection relationship between the display device 200 and the first terminal device, the first terminal device is removed from the projection list, and the exit time of the exit event is recorded. The first terminal device is a terminal device 500 in the projection list whose current audio parameter is the first parameter value.

[0231] If a connection event for establishing a screen projection connection relationship between the display device 200 and the second terminal device is detected within a preset time threshold after the exit time, the second terminal device is added to the screen projection list, and the audio parameter of the second terminal device is set to the first parameter value.

[0232] If a connection event for establishing a screen projection connection relationship between the display device 200 and the second terminal device is not detected within a preset time threshold after the exit time, a third terminal device is determined in the screen projection list according to the connection order, and the audio parameter of the third terminal device is set to the first parameter value.

[0233] It can be known from the above technical scheme that the display device provided by the above embodiment can record the terminal devices that establish a screen projection connection with the display device based on the screen projection list, as well as the connection order of the terminal devices and the display device to establish the screen projection connection. When the display device establishes a screen projection connection with at least two terminal devices, the sound mode of the multi-channel screen projection is adjusted through the screen projection list, and the sound mode of one terminal device in the screen projection list is set to a non-silent mode, and the other terminal devices are set to a silent mode, so that the display device only plays the sound of one channel of screen projection. The method can dynamically set the sound according to different screen projection scenarios, and can also dynamically control the sound according to the user's autonomous choice, so as to solve the problem of sound conflict during multi-channel screen projection.

[0234] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0235] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0236] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0237] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.< / doublepushcontrolbean>

Claims

1. A display device, characterized in that: include: A display, configured to display a multi-channel screen projection interface, wherein the multi-channel screen projection interface includes at least two screen projection windows, and the screen projection window is used to display the screen projection image of the terminal device; A communicator, configured to establish a screen projection connection with at least two terminal devices; An audio output interface configured to play audio data; The controller is configured as: Control the audio output interface to play audio data of a terminal device whose audio parameter in the screen projection list is a first parameter value, wherein the audio parameter is used to indicate whether the audio data is allowed to be played or not, and the first parameter value is used to indicate that the audio data is allowed to be played, and the screen projection list is used to record the terminal devices that establish a screen projection connection with the display device, and the connection order in which the terminal devices establish a screen projection connection with the display device; In response to an exit event of disconnecting the screen projection connection relationship between the display device and the first terminal device, removing the first terminal device from the screen projection list, and recording the exit time of the exit event, the first terminal device being a terminal device in the screen projection list whose current audio parameter is a first parameter value; If a connection event for establishing a screen projection connection relationship between the display device and the second terminal device is detected within a preset time threshold after the exit time, the second terminal device is added to the screen projection list, and the audio parameter of the second terminal device is set to the first parameter value; If a connection event for establishing a screen projection connection relationship between the display device and the second terminal device is not detected within a preset time threshold after the exit time, a third terminal device is determined in the screen projection list according to the connection order, and the audio parameter of the third terminal device is set to the first parameter value.

2. The display device according to claim 1, characterized in that The controller determines the third terminal device in the screen projection list according to the connection order, and is further configured to: Selecting, in the screen projection list, a terminal device that establishes a screen projection connection with the display device earliest according to the connection order as a third terminal device; Alternatively, a terminal device that establishes a screen projection connection with the display device the latest is selected from the screen projection list according to the connection order as the third terminal device.

3. The display device according to claim 1, characterized in that The controller is also configured to: In response to a connection event of establishing a screen projection connection relationship between the display device and the fourth terminal device, setting an audio parameter of the second terminal device to a second audio parameter, and setting an audio parameter of the fourth terminal device to a first audio parameter, wherein the second parameter value is used to indicate that audio data is not allowed to be played; Alternatively, the audio parameter of the fourth terminal device is set to the second audio parameter.

4. The display device according to claim 1, characterized in that The controller is also configured to: In response to a connection event of establishing a screen projection connection relationship between the display device and the terminal device, creating a sound view, wherein the sound view is a control in the screen projection interface for performing playback control on audio data; Reading a status identifier of the sound view, where the status identifier is associated with an audio parameter of the terminal device; If the state identifier is a first identifier value, controlling the display to display a first icon of the sound view, where the first icon is used to indicate a non-silent mode; If the state identifier is a second identifier value, the display is controlled to display a second icon of the sound view, where the second icon is used to indicate a silent mode.

5. The display device according to claim 4, characterized in that The controller is also configured to: Reading audio parameters of the terminal device; If the audio parameter is a first parameter value, setting the state flag of the sound view to the first flag value; If the audio parameter is a second parameter value, the state flag of the sound view is set to a second flag value.

6. The display device according to claim 1, characterized in that The controller is also configured to: In response to a full-screen instruction for full-screen display of a target projection window, deleting other terminal devices other than the target terminal device in the projection list, and disconnecting the projection connection with the other terminal devices, wherein the target terminal device is a terminal device whose projection screen is displayed in the target projection window; Reading audio parameters of the target terminal device; If the audio parameter is the second audio parameter, setting the audio parameter of the target terminal device to the first audio parameter, and displaying the target projection window in full screen; If the audio parameter is the first audio parameter, the target projection window is displayed in full screen.

7. The display device according to claim 1, characterized in that The controller is also configured to: In response to a display instruction to display a projection screen of a target terminal device in a target projection window, detecting a window state of the target projection window; If the window state is a locked state, obtain the target audio parameters of the target projection window, where the target audio parameters are audio parameters of the terminal device that displays the projection screen on the target projection window when the window state is set to a locked state; The audio parameters of the target terminal device are set as target audio parameters.

8. The display device according to claim 1, characterized in that The controller is also configured to: In response to an exit event of disconnecting the screen projection connection relationship between the display device and the first terminal device, recording a first media asset type and a play control list of the screen projection data of the first terminal device, wherein the play control list includes play control data of a play control view associated with the first media asset type, and the play control view is a control in the screen projection interface for performing play control on the screen projection data; When a connection event for establishing a screen projection connection relationship between the display device and the second terminal device is detected within a preset time threshold after the exit time, a second media asset type of the screen projection data of the second terminal device is obtained; If the first media asset type is the same as the second media asset type, the play control list is set for the second terminal device.

9. The display device according to claim 1, characterized in that: The controller is also configured to: In response to an exit event of disconnecting the screen projection connection relationship between the display device and the first terminal device, recording a display position parameter of a first screen projection window, where the first screen projection window is a screen projection window that displays a screen projection image of the first terminal device; If a connection event establishing a screen projection connection relationship between the display device and the second terminal device is detected within a preset time threshold after the exit time, the display is controlled to display the projection screen of the second terminal device in the first screen projection window according to the display position parameters.

10. A multi-channel screen projection sound broadcast control method, characterized in that: The display device according to any one of claims 1 to 9, the display device comprising a display, a communicator, an audio output interface and a controller, the display being configured to display a multi-channel screen projection interface, the multi-channel screen projection interface comprising at least two screen projection windows, the screen projection windows being used to display the screen projection image of the terminal device; The communicator is configured to establish a screen projection connection with at least two terminal devices; The audio output interface is configured to play audio data; the method comprises: Control the audio output interface to play audio data of a terminal device whose audio parameter in the screen projection list is a first parameter value, wherein the audio parameter is used to indicate whether the audio data is allowed to be played or not, and the first parameter value is used to indicate that the audio data is allowed to be played, and the screen projection list is used to record the terminal devices that establish a screen projection connection with the display device, and the connection order in which the terminal devices establish a screen projection connection with the display device; In response to an exit event of disconnecting the screen projection connection relationship between the display device and the first terminal device, removing the first terminal device from the screen projection list, and recording the exit time of the exit event, the first terminal device being a terminal device in the screen projection list whose current audio parameter is a first parameter value; If a connection event for establishing a screen projection connection relationship between the display device and the second terminal device is detected within a preset time threshold after the exit time, the second terminal device is added to the screen projection list, and the audio parameter of the second terminal device is set to the first parameter value; If a connection event for establishing a screen projection connection relationship between the display device and the second terminal device is not detected within a preset time threshold after the exit time, a third terminal device is determined in the screen projection list according to the connection order, and the audio parameter of the third terminal device is set to the first parameter value.

Images