Display device and multi-path projection sound broadcasting method

Abstract

The application provides a display device and a multi-path projection sound broadcasting method. The method can record a terminal device establishing a projection connection with the display device based on a projection list, and the connection order of the terminal device and the display device establishing the projection connection. When the display device establishes the projection connection with at least two terminal devices, the sound mode of multi-path projection is adjusted through the projection list, the sound mode of one terminal device in the projection list is set as a non-mute mode, and the sound modes of other terminal devices are set as mute modes, so that the display device plays only one path of projection sound. The method can dynamically set the sound according to different projection scenes, and can dynamically control the sound according to the autonomous selection of the user, so as to solve the problem of sound conflict in multi-path projection.

Description

[0001] This application is a divisional application of a domestic application (application number: 202311533185.8, application date: 2023-11-16, patent name: a display device and a multi-channel projection audio playback control method). Technical Field

[0002] This application relates to the field of display device technology, and in particular to a display device and a method for controlling audio playback on multi-channel projection. Background Technology

[0003] Display devices refer to terminal devices capable of outputting specific display images, such as smart TVs, communication terminals, smart advertising screens, and projectors. Taking smart TVs as an example, smart TVs are television products based on Internet application technology, possessing open operating systems and chips, and having open application platforms. They enable two-way human-computer interaction and integrate multiple functions such as audio-visual entertainment and data to meet diverse and personalized user needs. Display devices can establish communication connections with terminal devices such as mobile phones and, based on specific screen mirroring protocols, obtain screen mirroring data from the terminal devices to display the projected images.

[0004] Some display devices can establish communication connections with multiple terminal devices simultaneously and receive screen projection data from multiple terminal devices, forming a multi-channel projection scenario. However, during the audio transmission process of multi-channel projection, if the audio playback of multiple projections is not controlled, the audio from multiple projections will play simultaneously, causing sound conflicts and degrading the user experience. Summary of the Invention

[0005] This application provides several embodiments to offer a display device and a multi-channel projection audio playback control method to solve the problem of audio conflicts during multi-channel projection.

[0006] In a first aspect, this application provides a display device, including a monitor, a communicator, an audio output interface, and a controller. The monitor is configured to display a multi-channel projection interface, which includes at least two projection windows for displaying the projection screen of a terminal device. The communicator is configured to establish projection connections with at least two terminal devices. The audio output interface is configured to play audio data. The controller is configured to execute the following program steps: The audio output interface is controlled to play audio data from terminal devices in the screen casting list whose audio parameters are set to the first parameter value. The audio parameters are used to indicate whether audio data playback is allowed or not. The first parameter value is used to indicate that audio data playback is allowed. The screen casting list is used to record terminal devices that have established screen casting connections with the display device, as well as the connection order in which the terminal devices and the display device establish screen casting connections. In response to an exit event that disconnects the screen projection connection between the display device and the first terminal device, the first terminal device is removed from the screen projection list, and the exit time of the exit event is recorded. The first terminal device is the terminal device in the screen projection list whose current audio parameter is a first parameter value. If a connection event is detected within a preset time threshold after the exit time to establish a screen projection connection between the display device and the second terminal device, the second terminal device is added to the screen projection list, and the audio parameters of the second terminal device are set to the first parameter value. If no connection event is detected within a preset time threshold after the exit time to establish a screen projection connection between the display device and the second terminal device, the third terminal device is determined in the screen projection list according to the connection order, and the audio parameters of the third terminal device are set to the first parameter value.

[0007] Secondly, this application also provides a multi-channel projection audio playback control method, applied to the aforementioned display device, the method comprising: The audio output interface is controlled to play audio data from terminal devices in the screen casting list whose audio parameters are set to the first parameter value. The audio parameters are used to indicate whether audio data playback is allowed or not. The first parameter value is used to indicate that audio data playback is allowed. The screen casting list is used to record terminal devices that have established screen casting connections with the display device, as well as the connection order in which the terminal devices and the display device establish screen casting connections. In response to an exit event that disconnects the screen projection connection between the display device and the first terminal device, the first terminal device is removed from the screen projection list, and the exit time of the exit event is recorded. The first terminal device is the terminal device in the screen projection list whose current audio parameter is a first parameter value. If a connection event is detected within a preset time threshold after the exit time to establish a screen projection connection between the display device and the second terminal device, the second terminal device is added to the screen projection list, and the audio parameters of the second terminal device are set to the first parameter value. If no connection event is detected within a preset time threshold after the exit time to establish a screen projection connection between the display device and the second terminal device, the third terminal device is determined in the screen projection list according to the connection order, and the audio parameters of the third terminal device are set to the first parameter value.

[0008] As can be seen from the above technical solutions, this application provides a display device and a method for controlling audio playback in multi-channel projection. The method records the terminal devices that establish projection connections with the display device based on a projection list, as well as the connection order between the terminal devices and the display device. When the display device establishes projection connections with at least two terminal devices, the audio modes of the multi-channel projection are adjusted through the projection list. The audio mode of one terminal device in the projection list is set to non-mute mode, while the other terminal devices are set to mute mode, thus allowing the display device to play only the audio from one projection channel. This method allows for dynamic audio settings based on different projection scenarios and dynamic audio control based on user selection, resolving the problem of audio conflicts during multi-channel projection. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application; Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application; Figure 3 This is a schematic diagram of the hardware configuration of the control device provided in some embodiments of this application; Figure 4 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application; Figure 5 These are schematic diagrams illustrating screen mirroring provided for some embodiments of this application; Figure 6 These are schematic diagrams illustrating push-to-screen projection provided in some embodiments of this application; Figure 7 This is a schematic diagram of a multi-screen projection interface provided in some embodiments of this application; Figure 8 This is a schematic diagram of the functional modes of a display device provided in some embodiments of this application; Figure 9 A schematic diagram of a guide page provided for some embodiments of this application; Figure 10 This application provides schematic diagrams illustrating the interface display during one-channel screen mirroring in some embodiments. Figure 11 This is a schematic diagram illustrating the effect of dynamically adjusting the display area provided in some embodiments of this application; Figure 12 This is a schematic diagram of a play control button control provided in some embodiments of this application; Figure 13 This is a schematic diagram illustrating the effect of dynamically adjusting the number of play control buttons provided in some embodiments of this application; Figure 14 A schematic diagram illustrating the sound playback control process provided for some embodiments of this application; Figure 15 A schematic diagram illustrating the display effect of the sound view provided in some embodiments of this application; Figure 16 This application provides schematic diagrams illustrating the effects of dynamically adjusting the sound view in some embodiments. Figure 17 A schematic diagram illustrating the display effect when one channel of screen mirroring is disconnected, provided for some embodiments of this application; Figure 18 This application provides a flowchart of the sound switching process when screen mirroring is disconnected, as shown in some embodiments. Figure 19 This is a schematic diagram illustrating the effect of locking the sound mode of the screen projection window provided in some embodiments of this application; Figure 20 These are schematic diagrams illustrating the full-screen switching effect provided in some embodiments of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0012] Based on the exemplary embodiments shown in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can constitute a complete technical solution on its own.

[0013] It should be understood that the terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate, for example, to allow implementation in orders other than those given in the embodiments illustrated or described in this application.

[0014] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0015] The display device provided in this application can have various implementation forms, such as a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figure 1 and Figure 2 This is one specific embodiment of the display device of this application.

[0016] Figure 1 This is a schematic diagram illustrating the operational scenario between the display device and the control unit according to the embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0017] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0018] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, an application running on the smart device may be used to control the display device 200.

[0019] In some embodiments, the display device 200 may receive instructions not through the aforementioned smart device or control device, but through touch or gestures.

[0020] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0021] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0022] like Figure 2 As shown, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a communication device interface 280.

[0023] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first interface to an nth interface for input / output.

[0024] The display 260 includes a display screen assembly for presenting images, a driving assembly for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0025] The display 260 can be an LCD display, an OLED display, or a projection display, and can also be a projection device and a projection screen.

[0026] The communicator 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communicators 220 depending on the supported communication methods. For example, if the display device 200 supports wireless network communication, it may have a communicator 220 with WiFi functionality. If the display device 200 supports Bluetooth connectivity, it needs to have a communicator 220 with Bluetooth functionality.

[0027] The communicator 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0028] In some embodiments, the display device 200 can also support establishing communication connections with multiple external devices simultaneously. These multiple external devices can be the same, for example, terminal devices 500 such as mobile phones or tablets. The first terminal device 510 and the second terminal device 520 can establish communication connections with the display device 200 by accessing the wireless local area network (WLAN) where the display device 200 is located. The multiple external devices can also connect to the display device 200 using different types of connection methods. For example, the first terminal device 510 connects to the display device 200 via a WLAN; the second terminal device 520 connects to the display device 200 via Bluetooth.

[0029] To establish a communication connection between the display device 200 and the terminal device 500, both devices are equipped with communicators supporting the same communication type. For example, if the display device 200 has a communicator 220 containing a WiFi module, the terminal device 500 should also have a communicator containing a WiFi module. Furthermore, in addition to having the same type of communicator, the display device 200 and the terminal device 500 also require a specific communication transmission protocol for data transfer. Examples include WiFi, Bluetooth, ZigBee, and NFC protocols.

[0030] The audio output interface 270 is used to output audio signals to other devices for audio playback. In some embodiments, the display device 200 has a built-in speaker, and the audio output interface 270 can input audio signals to the speaker for audio playback. In other embodiments, the display device 200 can connect to external audio devices such as amplifiers, speakers, or loudspeakers via the audio output interface 270, thereby inputting audio signals to the external audio devices for audio playback.

[0031] The audio output interface 270 may include, but is not limited to, one or more of the following: RCA port, clip-on port, SPDIF port, and headphone port. It may also be a composite output interface formed by multiple of the above interfaces.

[0032] The communication device interface 280 can be used to receive control signals from the control device 100 (such as an infrared remote control).

[0033] Detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0034] Device interface 240 may include, but is not limited to, one or more of the following: High Definition Multimedia Interface (HDMI), analog or high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0035] The tuner / demodulator 210 receives broadcast television signals via wired or wireless means, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals.

[0036] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0037] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the monitor 260, the controller 250 can execute operations related to the object selected by the user command.

[0038] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM (Random Access Memory), a ROM (Read-Only Memory), a first to an nth interface for input / output, a communication bus, etc.

[0039] Users can input commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, users can input commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0040] A "user interface" is the medium through which an application or operating system interacts and exchanges information with the user. It converts information from its internal form to a form that the user can accept. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0041] Figure 3 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of the central control device. (Example) Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory 190, and a power supply 180.

[0042] The control device 100 is configured to control the display device 200, and to receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0043] In some embodiments, the control device 100 may be an intelligent device. For example, the control device 100 may be equipped with various applications for controlling the display device 200 according to user needs.

[0044] In some embodiments, such as Figure 1 As shown, the mobile terminal 300 or other smart electronic devices can perform similar functions to the control device 100 after installing the application of the control display device 200.

[0045] The controller 110 includes a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation of the control device 100, as well as the communication and cooperation between internal components and the external and internal data processing functions.

[0046] Under the control of the controller 110, the communication interface 130 enables communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of other near-field communication modules such as WiFi chip 131, Bluetooth module 132, and NFC module 133.

[0047] User input / output interface 140, wherein the input interface includes at least one of other input interfaces such as microphone 141, touchpad 142, sensor 143, and button 144.

[0048] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, which can encode user input commands via WiFi, Bluetooth, or NFC protocols and send them to the display device 200.

[0049] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can also store various control signal instructions input by the user.

[0050] The power supply 180 is used to provide operating power support for the various components of the control device 100 under the control of the controller.

[0051] like Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Layer"), and the kernel layer.

[0052] In some embodiments, at least one application runs in the application layer. These applications may be Windows programs, system settings programs, or clock programs that come with the operating system; they may also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0053] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0054] like Figure 4As shown, the application framework layer in this embodiment includes managers, content providers, etc., wherein the managers include at least one of the following modules: ActivityManager, which interacts with all activities running in the system; LocationManager, which provides access to system location services for system services or applications; PackageManager, which retrieves various information related to application packages currently installed on the device; NotificationManager, which controls the display and clearing of notification messages; and WindowManager, which manages icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0055] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling display window changes (e.g., shrinking the display window, shaking the display, distorting the display, etc.).

[0056] In some embodiments, the system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to perform.

[0057] In some embodiments, the kernel layer is a layer between hardware and software. For example... Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0058] After establishing a communication connection with the terminal device 500, the aforementioned display device 200 can establish a projection connection channel and transmit projection data based on the projection connection channel, so that the display device 200 can display the projection screen of the terminal device 500. In some embodiments, in order to establish a projection connection, the user can send a projection connection request through the terminal device 500 and send the projection connection request to the display device 200 via a WiFi network. The display device 200 then completes the transmission protocol configuration according to the projection connection request, thereby establishing a transmission channel for projection data with the terminal device 500.

[0059] Display device 200 can also establish a projection connection with terminal device 500 through different connection methods. For example, when display device 200 and terminal device 500 are connected to the same wireless local area network, a projection connection can be established based on the WiFi network. As another example, when both display device 200 and terminal device 500 are equipped with NFC (Near Field Communication) components, a projection connection can be established through the NFC components. Obviously, display device 200 and terminal device 500 can also use other wired or wireless connection methods to establish a projection connection, such as RF radio frequency connection, infrared connection, cellular network, and other connection methods that those skilled in the art can conceive of based on the connection methods provided in the above embodiments.

[0060] After establishing a screen mirroring connection, the display device 200 can receive screen mirroring-related data from the terminal device 500 through the screen mirroring data channel, thereby playing the screen mirroring-related content on the display device 200. Under different screen mirroring data transmission protocols, the form of screen mirroring data acquired by the display device 200 varies. For example, the display device 200 and the terminal device 500 can establish a screen mirroring connection through protocols such as the Digital Living Network Alliance (DLNA) protocol, Miracast protocol, AirPlay push protocol, WiDi screen mirroring, NFC mirroring, and LeBao screen mirroring.

[0061] DLNA is a protocol used to enable media resource sharing between devices. This protocol utilizes network technology (wired or wireless) to interconnect various devices, exchanging data through standard protocols to allow media resources to be shared between them. Based on the UPnP protocol, DLNA enables devices to discover and control each other, achieving seamless transmission and sharing of media resources. DLNA supports various devices, including televisions, stereos, mobile phones, and computers, allowing users to seamlessly switch and share media content across different devices.

[0062] The Miracast protocol is a wireless display standard developed by the Wi-Fi Alliance, based on Wi-Fi Direct. The Miracast screen mirroring protocol is compatible with Android devices, meaning Android devices can share video feeds via Miracast. Android 5.0 and later smart devices have the Miracast protocol built-in, allowing direct wireless screen mirroring between the mirroring device and the receiving device's Wi-Fi network card without installing any software. Furthermore, Miracast screen mirroring supports simultaneous audio and video transmission.

[0063] AirPlay is a wireless playback technology developed by Apple, suitable for wireless screen mirroring on iOS and Mac systems. Apple devices have built-in AirPlay mirroring functionality. When used with a screen mirroring device that supports AirPlay server service, the effect is clear and stable, and the protocol includes a built-in screen mirroring code. However, the Apple device and the screen mirroring device must be on the same local area network (same network segment) and cannot be used across network segments or VLANs.

[0064] WiDi screen mirroring is a wireless playback technology spearheaded by Intel, based on WiFi Direct technology. Windows 8 / 10 / 11 laptops natively support WiDi screen mirroring, allowing direct screen mirroring without installing any software, and also support USB reverse control functionality.

[0065] In some embodiments, after the display device 200 establishes a screen projection connection with the terminal device 500, screen projection data can be transmitted through mirroring mode and push mode. Mirroring mode refers to the display device 200 synchronously displaying the screen content displayed by the terminal device 500. For example, ... Figure 5 As shown, display device 200 displays the main interface of terminal device 500, and when the user performs interactive operations on terminal device 500, causing changes in the content of the main interface, the content displayed on display device 200 also changes accordingly. Therefore, terminal device 500 can generate a video projection data stream based on the displayed content and send the projection data stream to display device 200 for display.

[0066] Push mode refers to the display device 200 accessing a network link pushed by the terminal device 500 to obtain the corresponding projected screen. For example, ... Figure 6 As shown, when the terminal device 500 is playing video media, a screen mirroring control is provided in the upper right corner of the playback interface. After the user clicks the screen mirroring control and selects the display device 200 as the device to display the screen mirrored image, the terminal device 500 can send the URL address of the currently playing video media to the display device 200. After receiving the URL address, the display device 200 accesses the URL address to obtain the video media data and plays the video media.

[0067] Among the screen mirroring protocols supported by display device 200 and terminal device 500, some protocols are based on mirroring mode, while others are based on push mode. For example, under the DLNA protocol, display device 200 can obtain the link address of a resource in terminal device 500 through the screen mirroring data channel, and then obtain media asset content data by accessing that link address, thereby playing the media asset data. Simultaneously, display device 200 can also obtain control commands sent by terminal device 500 through the screen mirroring data channel while playing media asset data, such as fast forward, pause, and stop screen mirroring. For example, after establishing a connection through the communication protocol, terminal device 500 (such as a mobile phone) and display device 200 will negotiate device functions and network conditions to select a suitable audio-visual transmission format. After establishing the audio-visual streaming connection protocol (Session), a series of Real-Time Streaming Protocol (RTSP) control commands are then used to control the playback and termination of the audio-visual streaming.

[0068] Under other screen mirroring protocols such as Miracast, AirPlay, NFC mirroring, and LeBao screen mirroring, display device 200 can directly obtain video data streams from terminal device 500. This video data stream can be the video data stream corresponding to the final display screen of the terminal device 500's display unit; that is, the video data stream includes media content, UI interface, and other video content that can be displayed on the display unit of the terminal device 500. Alternatively, the video data stream can simply be the video data that can be obtained from the terminal device 500, such as currently playing media video data. In this case, terminal device 500 can act as a signal source for display device 200. Display device 200 can continuously obtain video data through the screen mirroring data channel and display it in real time on monitor 260, achieving a synchronized display effect with terminal device 500.

[0069] It should be noted that some screen mirroring protocols can not only obtain the video data stream from the terminal device 500, but also the audio data stream from the terminal device 500. This allows the video and audio data to be sent synchronously to the display device 200, enabling the display device 200 to play the video and audio simultaneously.

[0070] Terminal device 500 can obtain different screen mirroring data based on different screen mirroring protocols. For example, for system applications or screen mirroring directly at the system level, terminal device 500 can establish a screen mirroring data channel using Miracast, AirPlay, or WiDi screen mirroring protocols. Since the Miracast and AirPlay screen mirroring protocols require system-level application permissions, they are only applicable to system applications, or for display devices 200 and terminal device 500 from the same manufacturer to establish a screen mirroring connection directly through built-in rules in the operating system.

[0071] The display device 200 can establish a projection connection with the terminal device 500 based on any of the projection methods provided in the above embodiments. In some embodiments, the display device 200 can also establish projection connections with multiple communication terminals through one or more of the above projection methods. For example, the display device 200 can simultaneously establish projection connections with the first terminal device 510 and the second terminal device 520, and divide the display screen into display areas for displaying the projection screen shared by the first terminal device 510 and the second terminal device 520 respectively.

[0072] To enable the display device 200 to display a shared projection screen from multiple terminal devices 500, in some embodiments, when the display device 200 establishes a projection connection with multiple terminal devices 500, the display device 200 can display multiple projection interfaces, which may include multiple projection windows. Each projection window displays the projection screen from a corresponding terminal device 500 within a single projection connection channel. For example, ... Figure 7 As shown, the multi-channel projection interface can include two projection windows: a first projection window and a second projection window. The first projection window displays the projection screen of the first terminal device 510; the second projection window displays the projection screen of the second terminal device 520.

[0073] It should be noted that the casting window in the multi-channel casting interface does not necessarily have a strong binding relationship with the terminal device 500. That is, in some embodiments, the content displayed in the casting window can be determined according to the order in which the terminal device 500 and the display device 200 establish casting connections. For example, when the first terminal device 510 establishes a casting connection with the display device 200 before the second terminal device 520, the first casting window on the left side of the multi-channel casting interface displays the casting screen of the first terminal device 510. Conversely, when the first terminal device 510 establishes a casting connection with the display device 200 after the second terminal device 520, the first casting window on the left side of the multi-channel casting interface displays the casting screen of the second terminal device 520.

[0074] To achieve better image display and improve the coordination of multi-screen projection interfaces, in some embodiments, the display device 200 can also configure the layout of the projection windows. For this purpose, such as... Figure 8 As shown, the display device 200 can have multiple built-in functional modules, including: sound management module, data management module, message processing module, service management module, algorithm processing module, view management module, mode management module, application interaction processing module, location management module, substitution management module, broadcast control management module, time management module, focus management module, and screen projection management module, etc.

[0075] The data management module primarily manages all message data involved in the current template and synchronizes and updates the data information. The message processing module mainly handles the transmission of relevant messages between different modules, such as transmitting data changes to the View layer for refresh display, or synchronizing View changes to the data management module for broadcast control data processing. The service management module is responsible for starting and stopping services throughout the application and handling customized processes, such as starting specific services in specific scenarios. The algorithm processing module is responsible for all layout and position calculations throughout the application. The View management module is primarily responsible for all UI displays of the template application, receiving data and refreshing the View display. The sound management module is primarily responsible for handling sound during screen mirroring, such as sound processing when switching from no screen mirroring content to screen mirroring content, and sound processing when switching from one screen mirroring content to two screen mirroring content.

[0076] The application interaction processing module is responsible for handling interactions with external applications. For example, any entry point to launch the current template application needs to interact with this module, and different customized management is performed based on the differences in the launch entry point. The mode management module is responsible for the sound switching rules of the casting window, such as not switching sound when the current sound mode of the casting window is locked. The position management module is responsible for the rules of the casting position displayed on the left or right side of the window, such as whether it is on the left or right side. The substitution management module is responsible for the substitution rules when the number of casting channels exceeds the maximum display limit, such as how to substitute when there is focus or no focus. The playback control management module is responsible for the playback control processing of different media asset types in the casting, such as fast forward, rewind, play / pause, etc. The time management module is responsible for the focus display timing processing in the casting, whether it is the large frame focus, the playback control position focus, or the timing processing of sound mode switching in the casting. The focus management module is responsible for the focus display and button processing in the entire application. The casting management module is responsible for casting protocol interaction and client list management and window display content status information processing.

[0077] It should be noted that the multi-channel projection interface can be based on a multi-channel projection viewing application to display content. This application, capable of displaying multi-protocol, multi-channel mirroring and push content, is a higher-level application within the display device 200. The framework layer of the display device 200 provides Java interfaces for hardware interfaces. The multi-channel projection viewing application uses these Java interfaces to interface with lower layers and obtain information. Other layers such as Nativie, Runtime, HAL, and the Linux kernel are standard Android architecture layers, and these layers interact and process according to business processes.

[0078] The display device 200 first starts the application through the service management module and performs some initialization preparations, such as starting the listening image or pushing the corresponding protocol service; the view management module initializes the view displayed in the application layer; the data management module creates some data for the data processing module; the algorithm processing module performs algorithm layout calculations on the projected content; and the position management module is responsible for locating the left and right positions of the projected content.

[0079] The service management module can launch the screen mirroring application and register a Session during the screen mirroring connection establishment process; it can also listen for Session Connection Listener, Session Prepare Listener, Session DisplayListener, and Session Event Listener. A Session represents a single conversation between the client and the server and can store information from the client. The Session Connection Listener listens for Session connection and disconnection events. The Session Prepare Listener interacts with the client during screen mirroring preparation. The Session DisplayListener listens for playback information, state changes, and client state changes. The Session Event Listener handles client control information.

[0080] After the View is initialized, display device 200 will show a default onboarding page when entering the application. This onboarding page displays device information, such as the device name and corresponding network information of the current display device 200. Then, by controlling the standard system controls TextView and ImageView and their corresponding Layouts, the onboarding page is displayed in a symmetrical left-right proportion. Figure 9 As shown.

[0081] After the guide page is displayed, the display device 200 can listen for and receive the screen mirroring connection command sent by the terminal device 500, and establish a screen mirroring connection channel according to the screen mirroring connection command, thereby displaying it in the screen mirroring window. For example, the screen mirroring management module is always ready for the access of a new session. When Client1 is ready to start the connection, the screen mirroring management module listens for the onConnect connection of a new session through UseSessionConnectionListener, and then creates the playback control SurfaceView / TextureView. After successful creation, the position management module is started to set and manage the display position of the current session. The position is obtained through the position management module. The first screen mirroring is displayed on the left by default, and Client1's mViewPosition is marked as 1, where 1 represents the left side and 2 represents the right side. Other parameters can also be used to represent this.

[0082] To facilitate the management of View and Session, in some embodiments, a management object MultiCastPlayView can be set up. The management object MultiCastPlayView can be associated with information such as Session, SurfaceView / TextureView, and mViewPosition. For example, for the first terminal device 510, the data corresponding to its Client1 can be represented as: Session:SessionClient1, View:SurfaceView / TextureView, MViewPosition:1.

[0083] Once the display device 200, acting as the server, has prepared the View, it can notify the terminal device 500, acting as the mobile device, to record or send the video stream via the UseSessionPrepareListener. When the terminal device 500 is ready to send video information, the display device 200 learns through the onFirstFrame() method of the UseSessionDisplayListener that it needs to cast the video. Therefore, it can obtain the current video width and height of Client1 from the Session, determine the current video stream direction based on the width and height, and then calculate the position and size to be displayed using an algorithm.

[0084] That is, the first and second projection windows can also present a shape that adapts to the size of the projected image. For example, when both the first and second projection images are landscape images of 1920×1080, the display device 200 can present a first and second projection window with a size of 960×540 in the multi-projection interface.

[0085] In some embodiments, to distinguish the display range of the projection area, a certain blank area can be left between the first projection window and the second projection window, the projection window and the guide area. For example, the blank area is 12 pixels, and the resolution of the display device 200 is 1920×1080. Figure 10 As shown, when the first terminal device 510 (Client1) is displayed in landscape mode, the size of the screen-projected image in the corresponding video stream is 1920×1080. Then the size of the guide page is 954×538. At this time, the ratio of the video stream and the ratio of the guide video stream on the right are close to 16:9 and both are landscape. Therefore, it can be calculated by the algorithm that the left and right sides can be displayed in equal proportions, that is, they can be displayed with equal width and height. That is, the width of the video stream and the guide page is 954 and the height is about 538.

[0086] Because the terminal device 500 can be placed in different states depending on its usage, such as smartphones, it may be in portrait or landscape mode. Portrait mode refers to a display where the width of the screen is less than its height, while landscape mode refers to a display where the width is greater than its height. In these different placement states, the terminal device 500 can present different screen layouts.

[0087] Therefore, in some embodiments, when the display device 200 displays the projection screens of multiple terminal devices 500 through a region division method, each display area will present a different screen ratio depending on the placement state of the terminal devices 500. For example, if the first terminal device 510 is in landscape mode and the first projection screen size is 1920×1080, and the second terminal device 520 is in portrait mode and the second projection screen size is 512×1088, then the first projection window can be 954×538, and the second projection window can be 508×1080.

[0088] In order to make full use of the display space of the multi-screen projection interface and reduce large black screen areas, in some embodiments, the display device 200 can set the width or height of the first projection window and the second projection window to be the same when the sizes of the first projection screen and the second projection screen are different, and scale the projection window according to the width or height.

[0089] For example, when the first terminal device 510 is in portrait mode and the first projection screen size is 512×1088, and the second terminal device 520 is in landscape mode and the second projection screen size is 1920×1080, and the display resolution of the display device 200 is 1920×1080, the default size of the first and second projection windows (i.e., the display size of the guide page) is 954×538. At this time, the ratio of the video stream is different from that of the right guide video stream and the direction is different. Therefore, it can be calculated by the algorithm that the display height of the left video stream is about 848 and the width is about 400. Excluding the empty area, the remaining width is the width of the guide page, and the height is consistent with the video stream.

[0090] Therefore, the display device 200 can set the parameters of the first projection window, namely Client 1: width1 (width), height1 (height); Guide: width_guide, height_guide. The height of the video stream from the top edge of the screen is set to top, because height1 and height_guide will eventually be the same height, so the same variable height can be used instead. The video stream ratio ratio1 and the guide page ratio ratio_guide are both known constants. Then, width1, top, height, and width_guide can be calculated by the following combination of calculations: width1 + width_guide = 1920; top * 2 + height = 1080; width1 / height = ratio1; width_guide / height = ratio_guide.

[0091] Based on the above algorithm, the algorithm processing module of the display device 200 can obtain information such as the width, height and position of the content displayed on the left and right sides. That is, the width and height of Client1 can be calculated to be 848 and 400, respectively, with the corresponding position coordinates being (0, 116) from the top left corner, and the coordinates of the guide page being (412, 116).

[0092] After the above position calculation is completed, the data management module of the display device 200 can sort and store the data, and then transmit the data, including: Session:SessionClient1, View:SurfaceView / TextureView, MViewPosition:1, Position:(0, 116), Width:400, Height:848; Session:View:layout, MViewPosition: Position:(412, 116), Width:1508, Height:848. After receiving the data information, the View management module of the display device 200 sets the display parameters through LayoutParams, including the position (i.e., coordinates) from the top and left boundaries, as well as the corresponding display width and height, and then displays the position through the setLayoutParams method. That is: FrameLayout.LayoutParams lp1 = new FrameLayout.LayoutParams(width, height, 0); lp1.width=400; lp1.height=848; lp1.leftMargin=0; lp1.topMargin=116; mPlayerView.setLayoutParams(lp1); FrameLayout.LayoutParamslp1=newFrameLayout.LayoutParams(width,height,0);lp1.width=1508;lp1.height=848;lp1.leftMargin=412;lp1.topMargin=116;mPlayerView.setLayoutParams(lp1).

[0093] By setting the display position of the projected SurfaceView / TextureView and the display position of the guided layout respectively, you can obtain the following: Figure 11 The display effect of the multi-channel projection interface is shown.

[0094] In some embodiments, when displaying a multi-channel projection interface, the display device 200 can also display playback control function controls within the multi-channel projection interface. Users can use the playback control function to perform interactive operations, input control commands, and control the display device 200 to execute interactive responses. For example, since the projection content of the display device 200 and the terminal device 500 is diverse, including mirrored video streams, pushed URL addresses, and different media asset types such as images and music, the playback control function controls are also different for different media asset types. For example, mirrored video projection does not support fast forward / rewind or play / pause operations; pushed URL addresses can perform fast forward, rewind, play, and pause operations; music can perform fast forward, rewind, play, and pause operations; and images do not support fast forward, rewind, play, and pause operations.

[0095] The display device 200 can dynamically adjust the playback control function controls according to the display area of ​​the projection window in the multi-channel projection interface, so that the display position of the playback control function controls is adapted to the projection window, facilitating user interaction. For example, playback control function controls belonging to the same projection window can form a playback control list. The playback control list includes several playback control function controls such as full screen, fast forward, rewind, play / pause, mute / unmute, and close. The size of each playback control function control is 60×60, and the horizontal spacing between the playback control function controls is 24. Combining the left and right margins of the playback control list, the minimum display width for displaying six playback control function controls is 528, the minimum display width for displaying three playback control function controls is 276, the minimum display width for displaying two playback control function controls is 192, and the minimum display width for displaying one playback control function control is [not specified], and so on for other numbers.

[0096] In some embodiments, when the screen mirroring management module detects a new Session's onConnect connection through UseSessionConnectionListener, it can obtain the current Client's media asset type from the Session, such as VIDEO_URL, MIRROR_STREAM, MUSIC, PIC, etc. For example, if the currently displayed address is the pushed VIDEO_URL, then the display device 200 can perform operations such as full-screen, fast forward, rewind, play / pause, mute / unmute, and close. At this time, playback control data can be added to this Session, i.e., the type data of each playback control function can be represented as: DoublePushControlBean[mStatus=××,mType=××,mResId=××]. Based on this, the entire playback control list is the aforementioned List of DoublePushControlBeans, and the corresponding data can be represented as: DoublePushControlListBean[List <doublepushcontrolbean>=××,Status=××].

[0097] For example, when the projection window includes 6 playback control controls at the bottom, the corresponding data format is: Fullscreen button control: DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132]; Rewind button control: DoublePushControlBean[mStatus=true, mType=rewind, mResId=2131231110]; Play button control: DoublePushControlBean[mStatus=true, mType=playstatus, mResId=2131231149]; Fast forward button control: DoublePushControlBean[mStatus=true, mType=forward, mResId=2131231111]; Mute button control: DoublePushControlBean[mStatus=true, mType=mutestatus, mResId=2131231152]; Close button control: DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121].

[0098] To display the aforementioned broadcast control functions, after the data management module obtains the window's display size, the view management module arranges the layout as follows: Figure 12 The effect shown is that the left window contains URL media assets, which can be controlled for playback. The algorithm processing module calculates the size of the left window to be 1508×848 and the right window size to be 400×848. The following data results are obtained: Session:SessionClient1,View:SurfaceView / TextureView,MViewPosition:1,Position:(0,116),Width:1508,Height:848,mDoublePushControlListBean:… Session:SessionClient2,View:SurfaceView / TextureView,MViewPosition:2,Position:(1520,116),Width:400,Height:848,mDoublePushControlListBean:… After calculating the window display position, the View management module will arrange the left and right windows for display. During the display process, the playback control View needs to be initialized and displayed based on the playback control list data in the Session. For example, the display device 200 can use a RecyclerView control for display. The playback control menu is displayed by setting the margins and position of the RecyclerView. Similarly, the display parameters are set through LayoutParams, including the position (coordinates) from the top and left boundaries, as well as the corresponding display width and height. Then, the position is displayed using the setLayoutParams method.

[0099] After calculation by the algorithm processing module, the display area width of Client1 (1508) is greater than the minimum width of the play control bar (528), so it can be fully displayed, as shown below. Figure 12 The display can center all six broadcast control function controls or display them in any other position. Users can use the down button on the remote control to jump to the broadcast control function controls for operation.

[0100] When the content of media assets of type VIDEO_URL changes or the direction of the video stream changes, the display device 200 can rearrange the display of the projected content, such as... Figure 13 The left window shown contains media assets of type VIDEO_URL, but the algorithm processing module calculates that the left window size is 459×816, and the right window size is 1449×816. The following data results are obtained: Session:SessionClient1,View:SurfaceView / TextureView,MViewPosition:1,Position:(0,132),Width:459,Height:816,mDoublePushControlListBean:… Session:SessionClient2,View:,MViewPosition:,Position:(1520,132),Width:1449,Height:816,mDoublePushControlListBean:… After calculation by the algorithm processing module, the display area width of Client1, 459, is less than the minimum width of the play control bar, 528, but greater than 276. Therefore, it can display some play control function controls, such as... Figure 13 The display can center all three playback control controls or display them anywhere else. Users can use the down button on the remote to navigate to and operate the playback control controls. Specifically, it displays three buttons: full screen, mute, and close. The data format is as follows: DoublePushControlBean[mStatus=true,mType=fullscreen,mResId=2131231132]; DoublePushControlBean[mStatus=true,mType=mutestatus,mResId=2131231152]; DoublePushControlBean[mStatus=true,mType=close,mResId=2131231121].

[0101] When the size of the projection window changes, the display device 200 can notify the Recycleview to refresh the view through the Adapter's notifyDataSetChanged method, changing the number of buttons from 6 to 3.

[0102] like Figure 12 As shown in the playback control list, when the user clicks the third playback control button, the RecycleView will listen to the user's onClick operation. Based on the index type of the playback control list, i.e., mType of playstatus, it will call the Session's setPlayerPause() interface to handle the playback control. Similarly, fast forward and rewind are handled by calling Session's setPlayerSeekBackward() and setPlayerSeekForward(), etc.

[0103] The screen casting management module monitors the screen casting session status. When the video changes, it notifies the algorithm processing module through the message processing module. The algorithm processing module calculates the screen casting display area and then informs the data management module through the message processing module. After the data management module organizes the data, it notifies the view management module through the message processing module to redraw and display the view.

[0104] Similarly, when the screen mirroring management module listens for a new Session's onConnect connection through UseSessionConnectionListener, it can obtain the current Client's media asset type as MIRROR_STREAM from the Session. Then, on the large screen, it can perform full-screen, mute / unmute, and close operations. At this time, it can add playback control data to this Session, as follows: DoublePushControlBean[mStatus=true,mType=fullscreen,mResId=2131231132]; DoublePushControlBean[mStatus=true,mType=mutestatus,mResId=2131231152]; DoublePushControlBean[mStatus=true,mType=close,mResId=2131231121].

[0105] According to data from the data management module, the right-side window size is 400×848, and the three controls are each 276 pixels wide. Since the right-side window width of 400 pixels is greater than the width of the three controls (276 pixels), the three mirrored play control buttons can be fully displayed. Then, the RecycleView is initialized and its display is processed. Display parameters are set using LayoutParams, including the coordinates of the position from the top and left edges, as well as the corresponding width and height. The positions are then displayed using the setLayoutParams method. The corresponding play control list data is modified as follows: Session:SessionClient2, View:SurfaceView / TextureView, MViewPosition:2, Position:(1520, 116), Width:400, Height:848, mDoublePushControlListBean:… As can be seen, in the above embodiments, the display device 200 can adjust the number of button controls from six to three when the width of the projection window cannot accommodate the arrangement of six button controls. However, when the width of the projection window also cannot accommodate three playback control buttons, the number of playback control buttons needs to be adjusted further according to the projection window. If the right window is 258×864, the window width is less than the minimum display width of 276 for three button controls and greater than the minimum display width of 192 for two button controls. Therefore, the display device 200 can display two playback control buttons below the projection window. Furthermore, since the volume can be mutually exclusive controlled through the other party's buttons, only the full-screen and close buttons need to be retained, i.e.: DoublePushControlBean[mStatus=true,mType=fullscreen,mResId=2131231132]; DoublePushControlBean[mStatus=true,mType=close,mResId=2131231121].

[0106] Similarly, if the right window size is 156×864, the window width is less than the minimum display width of 192 required for two button controls, but greater than the minimum display width of 108 required for one button control, so one button can be displayed. Since closing the projection window can be controlled by stopping the projection operation on the terminal device 500, only the full-screen button can be retained, i.e.: DoublePushControlBean[mStatus=true,mType=fullscreen,mResId=2131231132].

[0107] Similarly, if the right-side window size is 88×864, the window width is less than the required display width of 108 for a button control, so the play control button cannot be displayed. Therefore, the display device 200 can cancel the display of the play control button control at the bottom of the right-side window. It should be noted that the above embodiment and the provided display method of the play control button control are only examples. When the play control button control cannot be fully displayed horizontally, the display device 200 can also display it vertically, in a rectangular array or a circular array, or by sliding along the edge. However, the basic display principle is universal and is not limited to the above display methods.

[0108] In some embodiments, based on the screen projection data transmission protocol, the display device 200 can acquire not only video data but also audio data, thereby enabling the display device 200 to synchronously play video and audio based on the audio output interface 270.

[0109] In some embodiments, the display device 200 can control the audio playback of multiple projected screens by executing a multi-channel projection audio playback control method. This multi-channel projection audio playback control method can be applied to the display device 200 provided in the above embodiments. To facilitate the implementation of the multi-channel projection audio playback control method, the display device 200 should include at least a display 260, a communicator 220, and a controller 250.

[0110] The display 260 is configured to display a multi-channel projection interface, which includes at least two projection windows for displaying the projection screen of the terminal devices. The communicator 220 is configured to establish projection connections with at least two terminal devices. The audio output interface 270 is configured to play audio data, and the controller 250 is configured to execute the program steps corresponding to the multi-channel projection audio playback control method, specifically including the following: Display device 200 can maintain a global session list to manage session access and removal. This session list records the terminal devices 500 that establish session connections with display device 200, and the connection order between the terminal devices 500 and display device 200. Display device 200 can adjust the audio modes of multiple session projections through this session list, setting the audio mode of one terminal device 500 in the session list to non-mute mode and setting the other terminal devices 500 to mute mode, thus allowing the display device to play only the audio from one session projection.

[0111] When setting the sound mode, the sound management module of the display device 200 can set an audio parameter. When the sound mode is non-mute, the audio parameter can be a first parameter value, which indicates that audio data is allowed to be played. When the sound mode is mute, the audio parameter can be a second parameter value, which indicates that audio data is not allowed to be played. This is used to characterize the various logic and display processing during the sound mode switching process.

[0112] Setting the sound mode of a terminal device 500 in the screen mirroring list to mute can be achieved in several ways. After the display device 200 establishes a screen mirroring connection with the terminal device 500, when the display device 200 receives screen mirroring data containing video and audio data, it can filter out the audio data and only receive the video data, thereby muting the sound of that screen mirroring channel. Alternatively, the display device 200 can receive both video and audio data but not play the audio data through the audio output interface 270, thus muting the sound of that screen mirroring channel. The display device 200 can also, based on the screen mirroring protocol, prevent the terminal device 500, which has established a screen mirroring connection with the display device 200, from sending audio data, thereby muting the sound of that screen mirroring channel. For example, some screen mirroring protocols can be set to transmit only video data, so that the terminal device 500 only sends video data and not audio data.

[0113] In some embodiments, the display device 200 can set the audio mode of the first screen mirroring connection to be established to a non-mute mode, meaning that the terminal device 500 (Client) that first preempts the audio holds the audio. Alternatively, the audio mode of the last screen mirroring connection to be established can be set to a non-mute mode, meaning that the terminal device 500 (Client) that most recently established the screen mirroring connection holds the audio.

[0114] like Figure 14 The diagram shows a flow chart of sound playback control provided in an embodiment of this application. When the display device 200 has no projection connection, it can respond to a connection event establishing a projection connection between itself and the first terminal device by setting the audio parameters of the first terminal device to a first audio parameter. When the display device 200 has a projection connection, it can respond to a connection event establishing a projection connection between itself and the second terminal device by setting the audio parameters of the first terminal device to a second audio parameter, and setting the audio parameters of the second terminal device to the first audio parameter, or vice versa. The first terminal device is terminal device 500 whose current audio parameter is the first parameter value.

[0115] For example, when UseSessionConnectionListener detects a Session connection, SessionList adds Client1 and sets Client1's audio mode to non-mute mode using Session's setAudioOutputState, meaning the first screen mirroring will emit sound. If UseSessionConnectionListener then detects another Session connection, SessionList adds Client2. Because the first screen mirroring has sound enabled, simultaneous sound from both channels would cause audio interference. Therefore, after the second Session connection, SessionList's setAudioOutputState will set Client2's audio mode to mute mode, meaning the second screen mirroring will not emit sound.

[0116] To help users distinguish between projection devices that emit sound and those that do not, the display device 200 can create a sound view in response to a connection event that establishes a projection connection between the display device 200 and the terminal device 500. The sound view is a control in the projection interface that performs playback control on audio data. The sound view may include a first icon and a second icon, where the first icon indicates non-mute mode and the second icon indicates mute mode. When the sound mode of the terminal device 500 in the projection list is non-mute, the display 260 is controlled to display the first icon of the sound view; when the sound mode of the terminal device 500 in the projection list is mute, the display 260 is controlled to display the second icon of the sound view, thus intuitively presenting the current projection sound mode to the user through the icons of the sound view.

[0117] To facilitate switching the sound view icon, a relationship can be established between the sound view's status identifier and the audio parameters of the terminal device 500. Specifically, the display device 200 can read the audio parameters of the terminal device 500 from the projection list. If the audio parameter is a first parameter value, the sound view's status identifier is set to the first identifier value. If the audio parameter is a second parameter value, the sound view's status identifier is set to the second identifier value.

[0118] When the sound view is displayed, the display device 200 can read the status identifier of the sound view. If the status identifier is a first identifier value, the display device 200 controls the monitor 260 to display the first icon of the sound view. If the status identifier is a second identifier value, the display device 260 controls the monitor 260 to display the second icon of the sound view.

[0119] For example, taking the above embodiment as an example, when UseSessionConnectionListener detects a Session connection, SessionList adds Client1 and sets Client1's audio mode to non-mute mode using Session's setAudioOutputState, meaning the first screen projection outputs sound. Figure 15 As shown, the first stream of screen-cast content is displayed through the left-side screen-casting window. Based on the display area of ​​the left-side screen-casting window, playback control views are created and displayed for full-screen, off, mute / non-mute, play / pause, fast forward, and rewind. Since the sound mode corresponding to the left-side screen-casting window is non-mute, the status flag representing "mute / non-mute" in the playback control view is set to the first flag value, i.e., mStatus=false, indicating that the current mode is non-mute. The playback control list data for Client1 is as follows: DoublePushControlBean [mStatus=true,mType=fullscreen,mResId=2131231132], DoublePushControlBean[mStatus=true,mType=rewind,mResId=2131231110], DoublePushControlBean[mStatus=true,mType=playstatus,mResId=2131231149], DoublePushControlBean[mStatus=true,mType=forward,mResId=2131231111], DoublePushControlBean[mStatus=false,mType=mutestatus,mResId=2131231152], DoublePushControlBean[mStatus=true,mType=close,mResId=2131231121] At this point, if the UseSessionConnectionListener detects another Session connection, it adds Client2 to the SessionList. Because the first screen mirroring session has sound enabled, simultaneous sound from both sessions would cause audio interference. Therefore, after the second Session connection is established, the Session's setAudioOutputState method will set Client2's sound mode to silent mode, meaning the second screen mirroring session will not emit sound. Figure 15 As shown, the second screen-cast content is displayed through the right-side screen-casting window. Based on the display area of ​​the right-side screen-casting window, playback control views are created and displayed for full-screen, off, mute / unmute, play / pause, fast forward, and rewind. Since the sound mode corresponding to the right-side screen-casting window is mute, the status flag representing "mute / unmute" in the playback control view is set to the second flag value, mStatus=true, indicating that the current mode is mute. The Client2 playback control list data is as follows: DoublePushControlBean[mStatus=true,mType=fullscreen,mResId=2131231132], DoublePushControlBean[mStatus=true,mType=rewind,mResId=2131231110], DoublePushControlBean[mStatus=true,mType=playstatus,mResId=2131231149], DoublePushControlBean[mStatus=true,mType=forward,mResId=2131231111], DoublePushControlBean[mStatus=true,mType=mutestatus,mResId=2131231152], DoublePushControlBean[mStatus=true,mType=close,mResId=2131231121] Based on the play control list data from Client1 and Client2, the icon display status of the mutestatus type view in the RecycleView of the left and right casting windows is set respectively. The display of the play control view is set using setBackgroundResource, presenting the following... Figure 15 The display effect shown is as follows: the sound view below the left projection window displays the first icon, and the sound view below the right projection window displays the second icon.

[0120] Once the display device 200 displays the sound view, the user can control the corresponding sound mode of the projection window based on the sound view. That is, the display device 200 can listen for user actions through the RecyclerView's onClick event and, based on the user's selection, mute / unmute the selected client, while performing the reverse operation on unselected clients.

[0121] For example, such as Figure 15 As shown in the audio view, the user can use the directional keys on a remote control adapted to the display device 200 to move the focus cursor to the "Mute / Unmute" playback control function button below the left projection window and press the "OK" button on the remote control to input the mute command. At this time, the RecycleView will listen to the user's onClick operation, and use Session's setAudioOutputState to set the audio mode of Client1 from unmute mode to mute mode, and the audio mode of Client2 from mute mode to unmute mode. Simultaneously, it will use setBackgroundResource to set the display of the playback control view, presenting it as shown in the image. Figure 16 The display effect shown is that the sound view below the left projection window displays the second icon, and the sound view below the right projection window displays the first icon.

[0122] In some embodiments, when a projection connection in the projection list where the sound mode is not silent is disconnected, the sound can be dynamically switched according to the projection list. That is, in response to an exit event that disconnects the projection connection between the display device 200 and the first terminal device, the display device 200 can remove the first terminal device from the projection list, determine a third terminal device in the projection list according to the connection order, and set the audio parameters of the third terminal device to a first parameter value. Here, the first terminal device is the terminal device 500 in the projection list whose audio parameters are currently set to the first parameter value.

[0123] For example, with Figure 15 Taking the two-way screen mirroring as an example, in the first channel, Client1, has sound. If Client1 disconnects, removing Client1 from the SessionList leaves only Client2. Now, Client2 can be unmuteed. First, the `setAudioOutputState` method of the Session in the screen mirroring management module is used to unmute the sound of the second channel. This message is then sent to the data management module via the message management module. The data management module then updates the playlist data for Client2 as follows: DoublePushControlBean[mStatus=false,mType=mutestatus,mResId=2131231152] Once the data update is complete, the message processing module notifies the View management module, which then dynamically refreshes the RecyclerView to display it as shown below. Figure 17 The display effect shown.

[0124] The above implementation can achieve dynamic switching of sound, but in some scenarios, users do not want to switch the sound, for example, with Figure 15 Taking the two-way screen mirroring example, the first mirroring client 1 is in non-mute mode, and the second mirroring client 2 is in mute mode. During the mirroring process, if the media asset type of the first mirroring client 1 is push-type, it may switch between episodes of the movie or TV series. During the episode switching process, depending on the different mobile devices, there may be a switch of sessions, which is equivalent to a new client access. There are also some cases where some mobile devices perform mirroring and then switch to push-type, which will also result in a new session access, which is also equivalent to a new client access.

[0125] In the above scenario, according to the previous processing logic, after the first projection client 1 disconnects, the display device 200 will demute the second projection client 2 to switch the sound. When the UseConnectionListener detects a new session connection, based on the principle that the projection client that first preempts the sound holds the sound, the display device 200 may mute the newly connected third projection client 3. In other words, according to the above processing logic, the sound mode may switch from non-mute mode to mute mode after a projection client switches episodes or changes media type. However, from the user's perspective, the goal is to continue playback according to the previous sound mode after a projection client switches episodes or changes media type, i.e., the third projection client 3 uses the sound mode of the first projection client 1.

[0126] Therefore, the display device 200 can be set with a time threshold, during which the sound will not be switched, and only after the time threshold of the exit event has elapsed. For example... Figure 18 As shown, the display device 200 can, in response to an exit event that disconnects the screen mirroring connection between the display device 200 and the first terminal device, remove the first terminal device from the screen mirroring list and record the exit time of the exit event.

[0127] If, within a preset time threshold after the exit time, a connection event establishing a projection connection between display device 200 and the second terminal device is detected, the second terminal device is added to the projection list, and the audio parameters of the second terminal device are set to the first parameter value. After setting the audio parameters of the second terminal device to the first parameter value, if a connection event establishing a projection connection between display device 200 and the fourth terminal device is detected, the pre-set sound processing logic can continue. That is, display device 200 can respond to the connection event establishing a projection connection between display device 200 and the fourth terminal device by setting the audio parameters of the second terminal device to the second audio parameter and the audio parameters of the fourth terminal device to the first audio parameter. Alternatively, the audio parameters of the fourth terminal device can be set to the second audio parameter.

[0128] If no connection event is detected within the preset time threshold after the exit time to establish a screen projection connection between the display device 200 and the second terminal device, the third terminal device is determined in the screen projection list according to the connection order, and the audio parameters of the third terminal device are set to the first parameter value.

[0129] The method of determining the third terminal device in the projection list according to the connection order can include several approaches. Display device 200 can select the terminal device 500 that established the projection connection with display device 200 earliest from the projection list, as the third terminal device, according to the connection order. Alternatively, it can select the terminal device that established the projection connection with display device 200 latest from the projection list, as the third terminal device.

[0130] For the broadcast control list data, if the media asset type of the currently exited projection channel is the same as that of a newly connected projection channel within a time threshold, the broadcast control list data can be directly reused. That is, in response to an exit event that disconnects the projection connection between the display device 200 and the first terminal device, the display device 200 can record the first media asset type and broadcast control list of the projection data from the first terminal device. The broadcast control list includes the broadcast control data of the broadcast control view associated with the first media asset type. If, within a preset time threshold after the exit time, a connection event is detected establishing a projection connection between the display device 200 and the second terminal device, the second media asset type of the projection data from the second terminal device is obtained. If the first media asset type and the second media asset type are the same, the broadcast control list is set for the second terminal device.

[0131] For example, with Figure 15 Taking the two-way screen mirroring example, if Client1 disconnects via UseConnectionListener, SessionList will remove Client1. After removal, a timer CountTimeUtils will be started with a duration of 2 seconds. During these 2 seconds, the audio switching method will be locked, preventing audio switching between the left and right mirroring windows; that is, the Session's setAudioOutputState method will be disabled. If a new Client3 joins within 2 seconds, SessionList will add Client3 and perform View layout operations based on Client3's VideoChanged. Meanwhile, Client3 will continue to use the original audio mode of Client1.

[0132] If the media asset type of Client3 is the same as that of Client1, then the broadcast control list data is reused. The broadcast control list data for Client3 is as follows: DoublePushControlBean[mStatus=true,mType=fullscreen,mResId=2131231132], DoublePushControlBean[mStatus=true,mType=rewind,mResId=2131231110], DoublePushControlBean[mStatus=true,mType=playstatus,mResId=2131231149], DoublePushControlBean[mStatus=true,mType=forward,mResId=2131231111], DoublePushControlBean[mStatus=false,mType=mutestatus,mResId=2131231152], DoublePushControlBean[mStatus=true,mType=close,mResId=2131231121] If the UseSessionConnectionListener has not detected a new Client3 connection after the 2-second countdown, then the sound of Client2 in the SessionList will be reset. The sound mode of Client2 will be set to non-mute mode using the setAudioOutputState method of the Session. The data will be synchronized to the data management module for update, and then synchronized to the View management module through the message processing module for View refresh display.

[0133] To improve the display effect, the display device 200 can also record the display position parameters of the first projection window in response to an exit event that disconnects the projection connection between the display device 200 and the first terminal device, wherein the first projection window is the projection window that displays the projection screen of the first terminal device. If a connection event that establishes a projection connection between the display device 200 and the second terminal device is detected within a preset time threshold after the exit time, the display 260 is controlled to display the projection screen of the second terminal device in the first projection window according to the display position parameters.

[0134] For example, such as Figure 17 As shown, the media assets of the second projection client, Client2, are displayed through the right projection window. When Client2 disconnects via UseConnectionListener, SessionList removes Client2 and records its display position in the right projection window. After removal, a timer, CountTimeUtils, is started with a duration of 2 seconds. If a new client, Client3, connects within 2 seconds, SessionList adds Client3 and sets its display position according to the recorded display position parameters, ensuring that Client3's media assets are displayed through the right projection window.

[0135] In some embodiments, the display device 200 also supports sound mode locking for the projection window. Once the sound mode of a projection window is locked, the window can maintain its previous sound mode without switching. To this end, the projection management module of the display device 200 manages session listening and sets the sound mode through the sound management module. When a user interacts with the View, the View management module listens for user actions and refreshes the View. Simultaneously, the message processing module notifies the data management module to update the data. When the projection management module detects an exit event from the projection connection, the message processing module notifies the time management module to monitor the time and determine whether to switch the sound. All of the above operations are performed dynamically only if the mode management module notifies that the sound mode of the current projection window is not locked. If the sound mode of the current projection window is locked, the locked projection window will not switch its sound, while the other projection window will maintain its default sound mode without switching.

[0136] Therefore, the display device 200 can respond to the lock command of the target projection window, record the current target audio parameters of the target projection window, and set a lock value for the target projection window. The lock value indicates that the window state of the target projection window is locked, and is used to instruct the audio parameters of the terminal device displaying the projection screen in the target projection window to be set to the target audio parameters.

[0137] When the display device 200 receives a display instruction to display the screen projection image of the target terminal device in the target projection window, it can respond to the display instruction by reading the lock value of the target projection window to detect the window state. If the window state is locked, it obtains the target audio parameters of the target projection window and sets the audio parameters of the target terminal device to the target audio parameters. The target audio parameters are the audio parameters of the terminal device whose screen projection image is displayed in the target projection window, recorded when the window state is set to locked.

[0138] For example, with the above Figure 16 Taking the two-way screen mirroring example, the first mirroring client 1 is in silent mode, while the second mirroring client 2 is in non-silent mode. At this time, the user long-presses... Figure 16 The "Sound" control at the bottom of the left-side projection window, as shown, allows input of a lock command for the left-side projection window. The display device 200 can then respond to this lock command, locking the current mute mode of the left-side projection window and setting `mBlockStatus` in the `DoublePushControlBean` of the current `mutestatus` type to `true`. This means that after the View management module detects the user's long-press action, it refreshes the View, displaying the result as shown below. Figure 19 The display shows the third icon in the audio view below the left-side projection window. This operation is then notified to the data management module via the message processing module. The data management module sets `mBlockStatus` to `true`. Upon successful setting, the audio mode of the locked left-side projection window is remembered, meaning `mIsPlayControlVoiceBlocked` is globally set to `true` and `mIsPlayControlVoiceBlockedLeft` is also globally set to `true`. Client1's playback control list data is as follows: DoublePushControlBean[mStatus=true,mType=fullscreen,mResId=2131231132], DoublePushControlBean[mStatus=true,mType=rewind,mResId=2131231110], DoublePushControlBean[mStatus=true,mType=playstatus,mResId=2131231149], DoublePushControlBean[mStatus=true,mType=forward,mResId=2131231111], DoublePushControlBean[mStatus=true,mType=mutestatus,mBlockStatus=true,mResId=2131231152], DoublePushControlBean[mStatus=true,mType=close,mResId=2131231121] After the above operations, regardless of whether a new Client is added or a disconnected Client is added, the sound mode in the left-hand projection window will not change. If Client2 disconnects, removing Client2 from the SessionList will not reset the sound because mIsPlayControlVoiceBlocked is true, even though the SessionList now only contains the silent Client1.

[0139] Furthermore, when the media assets of the newly connected Client4 are displayed in the left-hand projection window, because mIsPlayControlVoiceBlocked and mIsPlayControlVoiceBlockedLeft are true, the sound mode of Client4 remains silent. The specific data is as follows: DoublePushControlBean [mStatus=true,mType=fullscreen,mResId=2131231132], DoublePushControlBean[mStatus=true,mType=rewind,mResId=2131231110], DoublePushControlBean[mStatus=true,mType=playstatus,mResId=2131231149], DoublePushControlBean[mStatus=true,mType=forward,mResId=2131231111], DoublePushControlBean[mStatus=true,mType=mutestatus,mBlockStatus=true,mResId=2131231152], DoublePushControlBean[mStatus=true,mType=close,mResId=2131231121] In some embodiments, when the projection switches from a small window to a full-screen window, the audio also needs to undergo corresponding logical processing. The display device 200 can respond to a full-screen command for displaying the target projection window in full-screen mode by deleting other terminal devices from the projection list besides the target terminal device, and disconnecting the projection connection from other terminal devices. Here, the target terminal device is the terminal device whose projection screen is displayed in the target projection window. Then, the audio parameters of the target terminal device are read. If the audio parameters are the second audio parameters, the audio parameters of the target terminal device are set to the first audio parameters, and the target projection window is displayed in full-screen mode. If the audio parameters are the first audio parameters, the target projection window is displayed in full-screen mode.

[0140] For example, with the above Figure 15 Taking the two channels shown as examples, the first channel, Client1, has its audio mode in non-mute mode, while the second channel, Client2, is in mute mode. At this time, as... Figure 20 As shown, the user clicks Figure 15 The "Full Screen" playback control is shown below the right-hand projection window. First, remove Client1 from the SessionList, then remove the View corresponding to Client1 from the ViewList of the display window. Next, reset the display window, setting the display area and position of Client2 according to the window size. Simultaneously, use the Session's setAudioOutputState method to unmute Client2, displaying it as shown below. Figure 20 The display effect shown.

[0141] It should be noted that the specific solutions for sound processing are not limited to the examples above. The same rules apply to multiple screen mirroring windows and multiple screen mirroring protocols. In addition, sound processing can not only mute / unmute, but also adjust the volume, such as by accessing a secondary menu through the sound view or adding other display buttons to adjust the volume.

[0142] Based on the multi-channel projection audio playback control method provided in the above embodiments, some embodiments of this application also provide a display device 200, which includes a display 260, a communicator 220, and a controller 250. The display 260 is configured to display a multi-channel projection interface, which includes at least two projection windows for displaying the projection screen of the terminal device 500. The communicator 220 is configured to establish a communication connection with at least two terminal devices 500. The audio output interface 270 is configured to play audio data. The controller 250 is configured to execute the following program steps: The audio output interface 270 is controlled to play audio data from terminal devices whose audio parameters are the first parameter values ​​in the screen casting list. The audio parameters are used to indicate whether audio data playback is allowed or not, the first parameter value is used to indicate that audio data playback is allowed, and the screen casting list is used to record the terminal devices 500 that have established screen casting connections with the display device 200, as well as the connection order in which the terminal devices 500 and the display device 200 establish screen casting connections.

[0143] In response to an exit event that disconnects the screen mirroring connection between the display device 200 and the first terminal device, the first terminal device is removed from the screen mirroring list, and the exit time of the exit event is recorded. The first terminal device is terminal device 500 in the screen mirroring list whose current audio parameter is the first parameter value.

[0144] If a connection event is detected within a preset time threshold after the exit time, establishing a screen mirroring connection between the display device 200 and the second terminal device, the second terminal device is added to the screen mirroring list, and the audio parameters of the second terminal device are set to the first parameter value.

[0145] If no connection event is detected within the preset time threshold after the exit time to establish a screen projection connection between the display device 200 and the second terminal device, the third terminal device is determined in the screen projection list according to the connection order, and the audio parameters of the third terminal device are set to the first parameter value.

[0146] As can be seen from the above technical solutions, the display device provided in the above embodiments can record the terminal devices that establish a screen mirroring connection with the display device based on the screen mirroring list, as well as the connection order of the terminal devices establishing the screen mirroring connection with the display device. When the display device establishes a screen mirroring connection with at least two terminal devices, the sound mode of multiple screen mirrorings is adjusted through the screen mirroring list. The sound mode of one terminal device in the screen mirroring list is set to non-mute mode, while the other terminal devices are set to mute mode, so that the display device only plays the sound of one screen mirroring. The method can dynamically set the sound according to different screen mirroring scenarios, and can also dynamically control the sound according to the user's choice, solving the problem of sound conflicts when multiple screen mirrorings occur.

[0147] The same or similar parts among the various embodiments in this specification can be referred to mutually, and will not be repeated here.

[0148] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or certain parts of the embodiments of the present invention.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0150] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.< / doublepushcontrolbean>

Claims

1. A display device, characterized in that, include: The display is configured to display a multi-channel projection interface, the multi-channel projection interface including at least two projection windows, the projection windows being used to display the projection screen of the terminal device; The communicator is configured to establish screen mirroring connections with at least two terminal devices; The audio output interface is configured to play audio data; The controller is configured as follows: Maintain a screen mirroring list, which records terminal devices that have established screen mirroring connections with the display device. At any given time, only one terminal device in the screen mirroring list has an audio parameter of the first parameter value. The first parameter value indicates that audio data is allowed to be played, and the second parameter value indicates that audio data is not allowed to be played. In response to a full-screen command for displaying the target projection window in full screen, the target terminal device corresponding to the target projection window is determined; Remove other terminal devices from the screen mirroring list besides the target terminal devices, and disconnect the screen mirroring connection from the other terminal devices; Read the audio parameters of the target terminal device; If the audio parameter is the second parameter value, the audio parameter of the target terminal device is set to the first parameter value, and the target projection window is displayed in full screen; If the audio parameter is the first parameter value, the target projection window will be displayed in full screen.

2. The display device according to claim 1, characterized in that, The controller executes a full-screen command in response to a target projection window being displayed in full-screen mode, and is also configured to: The full-screen command is generated in response to the user clicking the full-screen control corresponding to the target projection window, inputting a voice command, or operating the corresponding button on the remote control.

3. The display device according to claim 1, characterized in that, The controller is also configured to: If the target projection window is in a sound mode locked state before full-screen switching, the audio parameters of the target terminal device will remain at the target audio parameters when locked after full-screen display, and the operation of automatically switching to the first parameter value will not be performed.

4. The display device according to claim 1, characterized in that, The controller is also configured to: In response to the exit command to exit full-screen display, restore the display of the multi-screen projection interface; Re-establish screen mirroring connections with other previously disconnected terminal devices and add those other terminal devices back to the screen mirroring list; The audio parameters of the target terminal device are restored to their original values ​​before the full-screen switch, and the audio parameters of other terminal devices are set according to preset sound rules.

5. The display device according to claim 4, characterized in that, The controller is configured to set the audio parameters of other terminal devices according to preset sound rules, and is also configured to: If the audio parameter of the target terminal device is the first parameter value before the full-screen switch, keep the audio parameter of the target terminal device at the first parameter value, and set the audio parameter of other terminal devices to the second parameter value; If the audio parameter of the target terminal device is the second parameter value before the full-screen switch, set the audio parameter of the terminal device that first established a connection in the screen casting list to the first parameter value, and set the audio parameters of other terminal devices to the second parameter value.

6. The display device according to claim 4, characterized in that, The controller is also configured to: Before disconnecting the screen mirroring connection with other terminal devices, record the audio parameter status of the other terminal devices; When restoring the multi-channel screen mirroring interface, the audio parameters of other terminal devices are set first based on the recorded audio parameter status and in combination with preset sound rules.

7. The display device according to claim 1, characterized in that, The controller is also configured to: During the full-screen display of the target projection window, in response to a connection event that establishes a projection connection between the display device and the new terminal device, the full-screen display is exited and the multi-projection interface is restored. Add the new terminal device to the screen mirroring list and set the audio parameters of the new terminal device according to the preset sound switching rules.

8. The display device according to claim 7, characterized in that, The controller executes the setting of the audio parameters of the new terminal device according to preset sound switching rules, and is also configured to: Set the audio parameters of the terminal device whose current audio parameters are the first parameter value to the second parameter value, and set the audio parameters of the new terminal device to the first parameter value; Alternatively, the audio parameters of the new terminal device can be set to a second parameter value.

9. The display device according to claim 1, characterized in that, The controller is also configured to: When the audio parameters of the target terminal device are set to the first parameter value, the output volume of the audio output interface is kept consistent with the system volume before the full-screen switch.

10. A method for audio processing during full-screen projection switching, characterized in that, Applied to the display device according to any one of claims 1-9, the display device comprising a display, a communicator, an audio output interface, and a controller, the method comprises: Maintain a screen mirroring list, which records terminal devices that have established screen mirroring connections with the display device. At any given time, only one terminal device in the screen mirroring list has an audio parameter of the first parameter value. The first parameter value indicates that audio data is allowed to be played, and the second parameter value indicates that audio data is not allowed to be played. In response to a full-screen command for displaying the target projection window in full screen, the target terminal device corresponding to the target projection window is determined; Remove other terminal devices from the screen mirroring list besides the target terminal devices, and disconnect the screen mirroring connection from the other terminal devices; Read the audio parameters of the target terminal device; If the audio parameter is the second parameter value, the audio parameter of the target terminal device is set to the first parameter value, and the target projection window is displayed in full screen; If the audio parameter is the first parameter value, the target projection window will be displayed in full screen.

Images