A display device and a method for displaying multiple projection windows.

Abstract

This application provides a display device and a method for displaying multiple projection windows. The method responds to a projection connection command sent by a first terminal device by creating a first projection window and releasing a first guide window. It then acquires a first projection session sent by the first terminal device and detects the screen resolution and the reference resolution of the guide page. A first display area and a second display area are calculated based on the screen resolution, the first resolution, and the reference resolution. The first display area and the second display area have equal heights. The display then controls the monitor to display the first projection window in the first display area and the second guide window in the second display area. This method, while displaying multiple windows at equal heights, calculates the display area of ​​each window based on the screen resolution, the projection image, and the resolution of the guide page, maximizing the display of the projection image in the multi-projection interface and improving the user experience.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a method for displaying multiple projection windows. Background Technology

[0002] Display devices are intelligent devices capable of presenting a user interface and supporting user interaction. Taking smart TVs as an example, smart TVs are television products based on Internet application technology, possessing an open operating system and chip, an open application platform, and enabling two-way human-computer interaction. They integrate multiple functions such as audio-visual entertainment and data to meet diverse and personalized user needs. Display devices can establish communication connections with terminal devices such as mobile phones and, based on specific screen mirroring protocols, obtain screen mirroring data from the terminal devices to display the mirrored screen content.

[0003] Some display devices can establish communication connections with multiple terminal devices simultaneously and receive screen projection data from multiple terminal devices, forming a multi-screen projection scenario. However, in a multi-screen projection scenario, the display device can only display the projection window of a single terminal device. If multiple projection windows of terminal devices are created simultaneously, the projection windows cannot communicate directly with each other, resulting in the projection windows obscuring each other on the display device. Summary of the Invention

[0004] This application provides a display device and a method for displaying multiple projection windows to solve the problem of projection windows of the display device obscuring each other.

[0005] In a first aspect, some embodiments of this application provide a display device, including a display, a communicator, and a controller. The display is configured to display a multi-channel projection interface, which includes a first guide window and a second guide window. The first guide window and the second guide window are used to display a guide page; the first guide window and the second guide window share the same root node as the multi-channel projection interface.

[0006] A communicator, configured to establish a communication connection with a terminal device;

[0007] The controller is configured as follows:

[0008] In response to the screen mirroring connection command sent by the first terminal device, a first screen mirroring window is created and the first guide window is released, while the second guide window is retained in the display of the multi-channel screen mirroring interface; the first screen mirroring window is used to display the first screen mirroring screen of the first terminal device, and the first screen mirroring window is the same as the root node of the multi-channel screen mirroring interface;

[0009] Obtain the first screen projection session sent by the first terminal device, wherein the first screen projection session includes the first resolution of the first screen projection screen;

[0010] Detect the screen resolution and compare it with the reference resolution of the onboarding page;

[0011] The window display area is calculated based on the screen resolution, the first resolution, and the reference resolution. The window display area includes a first display area and a second display area, and the heights of the first display area and the second display area are equal.

[0012] The display is controlled to display the first projection window in the first display area, and the display is controlled to display the second guide window in the second display area.

[0013] Secondly, some embodiments of this application also provide a method for displaying a multi-channel projection window, using the display device provided in the first aspect. The display device includes a display, a communicator, and a controller. The display is configured to display a multi-channel projection interface, which includes a first guide window and a second guide window. The first guide window and the second guide window are used to display a guide page. The first guide window and the second guide window are identical to the root node of the multi-channel projection interface. The communicator is configured to establish a communication connection with a terminal device. The method includes:

[0014] In response to the screen mirroring connection command sent by the first terminal device, a first screen mirroring window is created and the first guide window is released, while the second guide window is retained in the display of the multi-channel screen mirroring interface; the first screen mirroring window is used to display the first screen mirroring screen of the first terminal device, and the first screen mirroring window is the same as the root node of the multi-channel screen mirroring interface;

[0015] Obtain the first screen projection session sent by the first terminal device, wherein the first screen projection session includes the first resolution of the first screen projection screen;

[0016] Detect the screen resolution and compare it with the reference resolution of the onboarding page;

[0017] The window display area is calculated based on the screen resolution, the first resolution, and the reference resolution. The window display area includes a first display area and a second display area, and the heights of the first display area and the second display area are equal.

[0018] The display is controlled to display the first projection window in the first display area, and the display is controlled to display the second guide window in the second display area.

[0019] As can be seen from the above technical solutions, the display device and multi-channel projection window display method provided in some embodiments of this application can respond to a projection connection command sent by a first terminal device, create a first projection window, and release a first guide window. The first projection window is used to display the first projection screen of the first terminal device. Then, the first projection session sent by the first terminal device is acquired, and the screen resolution and the reference resolution of the guide page are detected. A first display area and a second display area are calculated based on the screen resolution, the first resolution, and the reference resolution. The height of the first display area and the second display area are equal. The display is then controlled to display the first projection window in the first display area and to display the second guide window in the second display area. This method can calculate the display area of ​​each window based on the screen resolution, the projection screen, and the resolution of the guide page, while displaying multiple windows at the same height, so that the projection screen can be displayed to the maximum extent in the multi-channel projection interface, improving the user experience. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating the usage scenarios of the display device in some embodiments of this application;

[0021] Figure 2 Hardware configuration block diagrams of display devices provided in some embodiments of this application;

[0022] Figure 3 Hardware configuration block diagrams of control devices provided in some embodiments of this application;

[0023] Figure 4 This is a schematic diagram of the software configuration in a display device provided in some embodiments of this application;

[0024] Figure 5 These are schematic diagrams illustrating screen mirroring provided for some embodiments of this application;

[0025] Figure 6 These are schematic diagrams illustrating push-to-screen projection provided in some embodiments of this application;

[0026] Figure 7 This is a schematic diagram of a multi-channel projection interface provided in some embodiments of this application;

[0027] Figure 8 This is a schematic diagram of the functional modes of a display device provided in some embodiments of this application;

[0028] Figure 9 A schematic diagram of a guide page provided for some embodiments of this application;

[0029] Figure 10 A flowchart illustrating a multi-channel projection window display method provided for some embodiments of this application;

[0030] Figure 11 A schematic diagram illustrating the display effect of a multi-channel screen mirroring interface when one channel is mirrored, provided for some embodiments of this application;

[0031] Figure 12 A flowchart illustrating the display of a second projection window provided in some embodiments of this application;

[0032] Figure 13 A schematic diagram illustrating the display effect of a multi-channel projection interface during two-channel projection, provided for some embodiments of this application;

[0033] Figure 14 This is a schematic diagram illustrating the display effect of a multi-channel projection interface during two-channel projection, provided in some embodiments of this application.

[0034] Figure 15 This is a schematic diagram illustrating the display effect of the floating display playback control function control provided in some embodiments of this application;

[0035] Figure 16 A schematic diagram illustrating the display effect of the edge area display control function control provided in some embodiments of this application;

[0036] Figure 17 This is a schematic diagram illustrating the display effect of a compressed projection window provided in some embodiments of this application. Detailed Implementation

[0037] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0038] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0039] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0040] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0041] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0042] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, users can operate the display device 200 via touch operation, mobile terminal 300, and control device 100. The control device 100 can be a remote control, stylus, etc.

[0043] In some embodiments, the mobile terminal 300 may install software applications with the display device 200 to establish a connection and communication via a network communication protocol, thereby achieving one-to-one control operation and data communication. Alternatively, audio and video content displayed on the mobile terminal 300 can be transmitted to the display device 200 to achieve synchronous display.

[0044] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0045] In addition to providing broadcast television reception functions, the display device 200 can also be equipped with intelligent network television functions that provide computer support, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0046] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0047] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0048] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0049] In some embodiments, the display 260 includes a display screen component for presenting an image, a driving component for driving image display, a component for receiving image signals output from a controller, and a user control UI interface, etc.

[0050] In some embodiments, the communicator 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communicators 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communicator 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communicator 220 with Bluetooth functionality.

[0051] The communicator 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0052] In some embodiments, the display device 200 can also support establishing communication connections with multiple external devices simultaneously. These multiple external devices can be the same, for example, terminal devices 500 such as mobile phones or tablets. The first terminal device 510 and the second terminal device 520 can establish communication connections with the display device 200 by accessing the wireless local area network (WLAN) where the display device 200 is located. The multiple external devices can also connect to the display device 200 using different types of connection methods. For example, the first terminal device 510 connects to the display device 200 via a WLAN; the second terminal device 520 connects to the display device 200 via Bluetooth.

[0053] To establish a communication connection between the display device 200 and the terminal device 500, both devices are equipped with communicators supporting the same communication type. For example, if the display device 200 has a communicator 220 containing a WiFi module, the terminal device 500 should also have a communicator containing a WiFi module. Furthermore, in addition to having the same type of communicator, the display device 200 and the terminal device 500 also require a specific communication transmission protocol for data transfer. Examples include WiFi, Bluetooth, ZigBee, and NFC protocols.

[0054] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a ROM, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0055] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0056] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0057] In some embodiments, user interface 280 is an interface that can be used to receive control input.

[0058] Figure 3 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of the central control device. (Example) Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0059] The control device 100 is configured to control the display device 200, and to receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0060] In some embodiments, the control device 100 may be an intelligent device. For example, the control device 100 may be equipped with various applications for controlling the display device 200 according to user needs.

[0061] In some embodiments, such as Figure 1 As shown, the mobile terminal 300 or other smart electronic devices can perform similar functions to the control device 100 after installing the application of the control display device 200.

[0062] The controller 110 includes a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation of the control device 100, as well as the communication and cooperation between internal components and the external and internal data processing functions.

[0063] Under the control of the controller 110, the communication interface 130 enables communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of other near-field communication modules such as WiFi chip 131, Bluetooth module 132, and NFC module 133.

[0064] User input / output interface 140, wherein the input interface includes at least one of other input interfaces such as microphone 141, touchpad 142, sensor 143, and button 144.

[0065] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, which can encode user input commands via the WiFi, Bluetooth, or NFC protocol and send them to the display device 200.

[0066] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can also store various control signal instructions input by the user.

[0067] The power supply 180 is used to provide operating power support for the various components of the control device 100 under the control of the controller.

[0068] like Figure 4 In some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Layer"), and the kernel layer.

[0069] In some embodiments, at least one application runs in the application layer. These applications may be Windows programs, system settings programs, or clock programs that come with the operating system; they may also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0070] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0071] like Figure 4 As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0072] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling display window changes (e.g., shrinking the display window, shaking the display, distorting the display, etc.).

[0073] In some embodiments, the system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to perform.

[0074] In some embodiments, the kernel layer is a layer between hardware and software. For example... Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0075] After establishing a communication connection with the terminal device 500, the aforementioned display device 200 can establish a projection connection channel and transmit projection data based on the projection connection, so that the display device 200 can display the projection screen of the terminal device 500. In some embodiments, in order to establish a projection connection, the user can send a projection connection request through the terminal device 500 and send the projection connection request to the display device 200 via a WiFi network. The display device 200 then completes the transmission protocol configuration according to the projection connection request, thereby establishing a transmission channel for projection data with the terminal device 500.

[0076] The display device 200 can also establish a projection connection with the terminal device 500 through different connection methods. For example, when the display device 200 and the terminal device 500 are connected to the same wireless local area network, a projection connection can be established based on the WiFi network. As another example, when both the display device 200 and the terminal device 500 are equipped with NFC (Near Field Communication) components, a projection connection can be established through the NFC components. Obviously, the display device 200 and the terminal device 500 can also use other wired or wireless connection methods to establish a projection connection, such as RF radio frequency connection, infrared connection, cellular network, and other connection methods that those skilled in the art can conceive of based on the connection methods provided in the above embodiments.

[0077] After establishing a screen mirroring connection, the display device 200 can receive screen mirroring-related data from the terminal device 500 through the screen mirroring data channel, thereby playing the screen mirroring-related content on the display device 200. Under different screen mirroring data transmission protocols, the form of screen mirroring data obtained by the display device 200 varies. For example, the display device 200 and the terminal device 500 can establish a screen mirroring connection through protocols such as the Digital Living Network Alliance (DLNA) protocol, Miracast protocol, AirPlay push protocol, WiDi screen mirroring, NFC mirroring, and LeBao screen mirroring.

[0078] DLNA is a protocol used to enable media resource sharing between devices. This protocol utilizes network technology (wired or wireless) to interconnect various devices, exchanging data through standard protocols to allow media resources to be shared between them. Based on the UPnP protocol, DLNA enables devices to discover and control each other, achieving seamless transmission and sharing of media resources. DLNA supports various devices, including televisions, audio equipment, mobile phones, and computers, allowing users to seamlessly switch and share media content across different devices.

[0079] The Miracast protocol is a wireless display standard developed by the Wi-Fi Alliance, based on Wi-Fi Direct. The Miracast screen mirroring protocol is compatible with Android devices, meaning Android devices can share video feeds via Miracast. Android smart devices running Android 5.0 and above have the Miracast protocol built-in, allowing for direct wireless screen mirroring between the mirroring device and the receiving device's Wi-Fi network card without installing any software. Furthermore, Miracast screen mirroring supports simultaneous audio and video transmission.

[0080] AirPlay is a wireless playback technology developed by Apple, suitable for wireless screen mirroring on iOS and Mac systems. Apple devices have built-in AirPlay mirroring functionality. When used with a screen mirroring device that supports AirPlay server service, the effect is clear and stable, and the protocol includes a built-in screen mirroring code. However, the Apple device and the screen mirroring device must be on the same local area network (same network segment) and cannot be used across network segments or VLANs.

[0081] WiDi screen mirroring is a wireless playback technology spearheaded by Intel, based on WiFi Direct technology. Windows 8 / 10 / 11 laptops natively support WiDi screen mirroring, allowing direct screen mirroring without installing any software, and also support USB reverse control functionality.

[0082] In some embodiments, after the display device 200 establishes a screen projection connection with the terminal device 500, screen projection data can be transmitted through mirroring mode and push mode. Mirroring mode refers to the display device 200 synchronously displaying the screen content displayed by the terminal device 500. For example, ... Figure 5 As shown, display device 200 displays the main interface of terminal device 500, and when the user performs interactive operations on terminal device 500, causing changes in the content of the main interface, the content displayed on display device 200 also changes accordingly. Therefore, terminal device 500 can generate a video projection data stream based on the displayed content and send the projection data stream to display device 200 for display.

[0083] Push mode refers to the display device 200 accessing a network link pushed by the terminal device 500 to obtain the corresponding projected screen. For example, ... Figure 6 As shown, when the terminal device 500 is playing video media, a screen mirroring control is provided in the upper right corner of the playback interface. After the user clicks the screen mirroring control and selects the display device 200 as the device to display the screen mirrored image, the terminal device 500 can send the URL address of the currently playing video media to the display device 200. After receiving the URL address, the display device 200 accesses the URL address to obtain the video media data and plays the video media.

[0084] Among the screen mirroring protocols supported by display device 200 and terminal device 500, some protocols are based on mirroring mode, while others are based on push mode. For example, under the DLNA protocol, display device 200 can obtain the link address of a resource in terminal device 500 through the screen mirroring data channel, and then obtain media asset content data by accessing that link address, thereby playing the media asset data. Simultaneously, display device 200 can also obtain control commands sent by terminal device 500 through the screen mirroring data channel while playing media asset data, such as fast forward, pause, and stop screen mirroring. For example, after establishing a connection through the communication protocol, terminal device 500 (such as a mobile phone) and display device 200 will negotiate device functions and network conditions to select a suitable audio-visual transmission format. After establishing the audio-visual streaming connection protocol (Session), a series of Real-Time Streaming Protocol (RTSP) control commands are then used to control the playback and termination of the audio-visual streaming.

[0085] Under other screen mirroring protocols such as Miracast, AirPlay, NFC mirroring, and LeBao screen mirroring, display device 200 can directly obtain video data streams from terminal device 500. This video data stream can be the video data stream corresponding to the final display screen of the terminal device 500's display unit; that is, the video data stream includes media content, UI interface, and other video content that can be displayed on the display unit of the terminal device 500. Alternatively, the video data stream can simply be the video data that can be obtained from the terminal device 500, such as currently playing media video data. In this case, terminal device 500 can act as a signal source for display device 200. Display device 200 can continuously obtain video data through the screen mirroring data channel and display it in real time on monitor 260, achieving a synchronized display effect with terminal device 500.

[0086] It should be noted that the above-mentioned screen mirroring protocol can not only obtain the video data stream from the terminal device 500, but also the audio data stream from the terminal device 500, so that the video data and audio data can be sent to the display device 200 simultaneously, enabling the display device 200 to play the video and audio synchronously.

[0087] Terminal device 500 can obtain different screen mirroring data based on different screen mirroring protocols. For example, for system applications or screen mirroring directly at the system level, terminal device 500 can establish a screen mirroring data channel using Miracast, AirPlay, or WiDi screen mirroring protocols. Since the Miracast and AirPlay screen mirroring protocols require system-level application permissions, they are only applicable to system applications, or for display devices 200 and terminal device 500 from the same manufacturer to establish a screen mirroring connection directly through built-in rules in the operating system.

[0088] The display device 200 can establish a projection connection with the terminal device 500 based on any of the projection methods provided in the above embodiments. In some embodiments, the display device 200 can also establish projection connections with multiple communication terminals through one or more of the above projection methods. For example, the display device 200 can simultaneously establish projection connections with the first terminal device 510 and the second terminal device 520, and divide the display screen into display areas for displaying the projection screen shared by the first terminal device 510 and the second terminal device 520 respectively.

[0089] To enable the display device 200 to display a shared projection screen from multiple terminal devices 500, in some embodiments, when the display device 200 establishes a projection connection with multiple terminal devices 500, the display device 200 can display multiple projection interfaces, which may include multiple projection windows. Each projection window displays the projection screen from a corresponding terminal device 500 within a single projection connection channel. For example, ... Figure 7 As shown, the multi-channel projection interface can include two projection windows: a first projection window and a second projection window. The first projection window displays the projection screen of the first terminal device 510; the second projection window displays the projection screen of the second terminal device 520.

[0090] It should be noted that the casting window in the multi-channel casting interface does not necessarily have a strong binding relationship with the terminal device 500. That is, in some embodiments, the content displayed in the casting window can be determined according to the order in which the terminal device 500 and the display device 200 establish casting connections. For example, when the first terminal device 510 establishes a casting connection with the display device 200 before the second terminal device 520, the first casting window on the left side of the multi-channel casting interface displays the casting screen of the first terminal device 510. Conversely, when the first terminal device 510 establishes a casting connection with the display device 200 after the second terminal device 520, the first casting window on the left side of the multi-channel casting interface displays the casting screen of the second terminal device 520.

[0091] To achieve better image display and improve the coordination of multi-screen projection interfaces, in some embodiments, the display device 200 can also configure the layout of the projection windows. For this purpose, such as... Figure 8 As shown, the display device 200 can integrate multiple functional modules within the same application, such as a multi-channel projection application. The multi-channel projection application includes: a data management module, a message processing module, a service management module, an algorithm processing module, a view management module, an application interaction processing module, a location management module, a substitution management module, a broadcast control management module, a time management module, a focus management module, and a projection management module, etc.

[0092] The data management module primarily manages all message data involved in the current template and synchronizes and updates the data information. The message processing module mainly handles the transmission of relevant messages between different modules, such as transmitting data changes to the View layer for refresh display, or synchronizing View changes to the data management module for broadcast control data processing. The service management module is responsible for starting and stopping services throughout the application and handling customized processes, such as starting specific services in specific scenarios. The algorithm processing module is responsible for all layout and position calculations throughout the application. The View management module is primarily responsible for all UI displays of the entire template application, receiving data and refreshing the View display.

[0093] The application interaction processing module is responsible for handling interactions with external applications. For example, any entry point to launch the current template application needs to interact with this module, and different customized management is performed based on the different entry points. The position management module is responsible for the rules of displaying the projection position in the left or right side windows, such as whether it's on the left or right. The substitution management module is responsible for the substitution rules when the number of projection channels exceeds the maximum display limit, such as how to substitute when there is focus or not. The playback control management module is responsible for the playback control processing of different media asset types in the projection, such as fast forward, rewind, play / pause, etc. The time management module is responsible for the focus display timing processing in the projection, whether it's the large frame focus or the focus at the playback control position. The focus management module is responsible for the focus display and button processing throughout the application. The projection management module is responsible for projection protocol interaction, client list management, and handling the status information of the window displayed content.

[0094] It should be noted that the multi-channel projection interface can be based on a multi-channel projection viewing application to display content. This application, capable of displaying multi-protocol, multi-channel mirroring and push content, is a higher-level application within the display device 200. The framework layer of the display device 200 provides Java interfaces for hardware interfaces. The multi-channel projection viewing application uses these Java interfaces to interface with lower layers and obtain information. Other layers such as Nativie, Runtime, HAL, and the Linux kernel are standard Android architecture layers, and these layers interact according to business processes.

[0095] The display device 200 first starts the application through the service management module and performs some initialization preparations, such as starting the listening image or pushing the corresponding protocol service; the view management module initializes the view displayed in the application layer; the data management module creates some data for the data processing module; the algorithm processing module performs algorithm layout calculations on the projected content; and the position management module is responsible for locating the left and right positions of the projected content.

[0096] The service management module can launch the screen mirroring application and register a Session during the screen mirroring connection establishment process; it can also listen for Session Connection Listener, Session Prepare Listener, Session Display Listener, and Session Event Listener. A Session represents a single conversation between the client and the server and can store information from the client. The Session Connection Listener listens for Session connection and disconnection events. The Session Prepare Listener interacts with the client during screen mirroring preparation. The Session Display Listener monitors playback information, status changes, and client-side status changes. The Session Event Listener handles client-side control information.

[0097] After the View is initialized, the display device 200 will display a default onboarding window when entering the application. This onboarding window displays a setup page, which shows device information such as the device name and corresponding network information of the current display device 200. The setup page can include multiple standard system TextViews and ImageViews. The display device 200 can control the TextViews and ImageViews and their corresponding layouts to display the setup page in a symmetrical horizontal layout. Figure 9 As shown. The guide page includes a reference resolution; when the display device 200 displays the guide page through the guide window, it needs to scale the guide page according to the reference resolution.

[0098] In some embodiments, the guide window and the multi-screen projection interface share a root node, meaning they share the same root node. For example, after a user runs the multi-screen projection application on the display device 200, the display device 200 controls the monitor 260 to display the multi-screen projection interface in full screen and displays the guide window in the corresponding position on the multi-screen projection interface. The multi-screen projection interface is the main interface of the multi-screen projection application, and the guide window uses the same root node as the main interface of the multi-screen projection application.

[0099] After the guide window is displayed, the display device 200 can listen to and receive the screen projection connection command sent by the terminal device 500, and establish a screen projection connection channel according to the screen projection connection command, thereby displaying the screen projection screen of the terminal device in the screen projection window.

[0100] Some embodiments of this application provide a display device 200, including a display 260, a communicator 220, and a controller 250. The display 260 is configured to display a multi-channel projection interface, which includes a first guide window and a second guide window. The first and second guide windows are used to display a guide page, and both the first and second guide windows share the same root node as the multi-channel projection interface. The communicator 220 is configured to establish a communication connection with a terminal device 500. Figure 10 As shown, the controller 250 is configured to perform the following program steps:

[0101] S100: In response to the screen mirroring connection command sent by the first terminal device, create a first screen mirroring window, release the first guide window, and retain the display of the second guide window on the multi-channel screen mirroring interface.

[0102] After displaying the default guide window, the user can establish a communication connection between the terminal device 200 and the display device 200 based on the prompts in the guide window. When the user sends a screen mirroring connection command to the display device 200 through the first terminal device 510, the display device 200 responds to the screen mirroring connection command and creates a first screen mirroring window. This first screen mirroring window is used to display the first screen mirroring image, and the first screen mirroring window shares the same root node as the multi-channel screen mirroring interface; that is, the first screen mirroring window also uses the same root node as the multi-channel screen mirroring interface.

[0103] For example, the screen projection management module of the display device 200 is always ready for the access of a new session. When the first terminal device 510Client1 is ready to start the connection, the screen projection management module listens for the onConnect connection of the new session through UseSessionConnectionListener and then executes the creation of the first screen projection window. The first screen projection window may include a playback control SurfaceView / TextureView, and the display device 200 can set the first screen projection screen to the corresponding position of the SurfaceView / TextureView.

[0104] While creating the first projection window, the display device 200 also releases the first guide window, preventing it from appearing in the multi-projection interface. Simultaneously, the second guide window remains displayed in the multi-projection interface. When releasing the first guide window, the display device 200 can also release the resources associated with the window object corresponding to the first guide window.

[0105] To facilitate the management of the display orientation of the projection window, in some embodiments, the display device 200 sets the orientation identifier of the first projection window through the position management module when creating the first projection window. The orientation identifier represents the display orientation of the projection window, and the display device 200 can display the first projection window according to the orientation identifier. When setting the orientation identifier, the display device 200 can set the orientation identifier of the projection window according to the default priority order and the projection order of the terminal device 500. The orientation identifier can be different numbers or English characters. For example, the display orientation of the projection window includes left and right sides, and the default priority order is left > right, with the orientation identifier of the left side being 1 and the orientation identifier of the right side being 2. After receiving the projection connection command from the first terminal device 510, the position management module sets the orientation identifier of the first projection window to 1 according to the default priority order, i.e., sets mViewPosition:1.

[0106] To facilitate the management of the first projection window and the first projection session between the first projection window and the first terminal device 510, in some embodiments, the display device 200 further includes a management object MultiCastPlayView. This MultiCastPlayView can be associated with information such as Session, SurfaceView / TextureView, and mViewPosition. For example, for the first terminal device 510, the data corresponding to its Client1 can be represented as: Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1.

[0107] S200: Obtain the first screen mirroring session sent by the first terminal device.

[0108] After creating the first projection window, the display device 200 obtains the first projection session sent by the first terminal device 510, and uses this session to acquire projection data information based on the first terminal device 510. This projection data information includes information associated with the projection connection between the first terminal device and the display device, such as the resolution of the first projection screen; that is, the first projection session includes the first resolution of the first projection screen. By obtaining the first projection session (SessionClient1), the display device 200 can obtain the first resolution of the first projection screen and then calculate the display area of ​​the first projection window based on this first resolution.

[0109] For example, once the display device 200 has prepared the first projection window view, it can notify the first terminal device Client1 via UsePrepareListener to record or send the video stream. When the terminal device 500 is ready to send video information, the display device 200 learns through UseDisplayListener's onFirstFrame() method that the first projection screen is about to be played. Therefore, it obtains the first resolution of the first projection screen corresponding to Client1 from SessionClient1, i.e., the width and height of the video stream. It then determines the current video stream direction and calculates the current view's display position and size using an algorithm.

[0110] S300: Detects the screen resolution and the baseline resolution of the boot page.

[0111] Since the display device 200 releases the first guide window and creates the first projection window, the windows displayed in the multi-projection interface include the first projection window and the second guide window. Therefore, after obtaining the first resolution of the first projection screen in the first projection session, the display device 200 detects the screen resolution of the display device 200 and the reference resolution of the guide page to determine the display areas of the first projection window and the second guide window through the screen resolution, the first resolution of the first projection screen, and the reference resolution of the guide page.

[0112] The screen resolution and the reference resolution of the guide page are pre-stored information in the display device 200. The display device 200 can detect its own screen resolution and the reference resolution of the guide page in a specified storage location. For example, the reference resolution of the guide page can be stored in the layout file of a multi-screen projection application. The display device 200 can detect the layout information of the guide page in the layout file to obtain the reference resolution.

[0113] S400: Calculate the window display area based on the screen resolution, the first resolution, and the reference resolution.

[0114] After obtaining the screen resolution, the first resolution, and the reference resolution, the display device 200 can calculate the display area of ​​each window according to the aforementioned resolutions. The window display area includes a first display area and a second display area. The first display area is used to display the first projection window, and the second display area is used to display the second guide window. To improve the user's visual experience, the first and second display areas are of equal height, ensuring that all windows maintain a consistent height and avoiding the problem of users repeatedly switching viewing heights when viewing multiple windows simultaneously.

[0115] In some embodiments, when calculating the window display area based on the screen resolution, a first resolution, and a reference resolution, the display device 200 reads the horizontal and vertical pixel values ​​of the screen resolution. It then calculates a first aspect ratio based on the ratio of the horizontal to vertical pixel values ​​in the first resolution, and a second aspect ratio based on the ratio of the resolution to the vertical pixel values ​​in the reference resolution. For example, the screen resolution of the display device 200 is 1920*1080, the first resolution of the first projected image is 1920*1080, and the reference resolution of the guide page is 954*538. The horizontal pixel value in the screen resolution is 1920, and the vertical pixel value is 1080; the horizontal pixel value in the first resolution is 1920, and the vertical pixel value is 1080. Calculating their ratio yields a first aspect ratio of 16 / 9; the horizontal pixel value in the reference resolution is 954, and the vertical pixel value is 1080. Calculating their ratio yields a second aspect ratio of 477 / 269, which is close to 16 / 9.

[0116] Next, set the first layout parameters and the second layout parameters. The first layout parameters include a first layout width and a first layout height, and the second layout parameters include a second layout width and a second layout height. The first layout height is equal to the second layout height; the sum of the first and second layout widths is equal to the horizontal pixel value of the screen resolution; the first and second layout heights are less than or equal to the vertical pixel value of the screen resolution; the ratio of the first layout width to the first layout height is equal to the first aspect ratio; and the ratio of the second layout width to the second layout height is equal to the second aspect ratio. Then, generate a first display area according to the first layout parameters and a second display area according to the second layout parameters.

[0117] To improve the standardization of window layouts, in some embodiments, the multi-channel projection interface of the display device 200 includes an edge region. This edge region is the display area in the multi-channel projection interface excluding the display window and the guide window, and can be referred to as a white space. Furthermore, the edge region can also be used to display playback control functions or prompts. Therefore, the sum of the layout height and the edge region height is set to equal the vertical pixel value of the screen.

[0118] In some embodiments, when setting the first layout parameters, the display device 200 reads a first preset height of the edge region. The edge region includes the display area in the multi-channel projection interface excluding the first display window and the second guide window. The sum of the first layout height and the first preset height is then set to equal the vertical pixel value of the screen resolution to generate the first layout height based on the first preset height.

[0119] Taking a display device 200 with a screen resolution of 1920*1080 as an example, when setting the first and second layout parameters, the display device 200 sets Client 1: width1, height1; Guide (guide page): width_guide, height_guide, according to the first resolution and the base resolution. Here, width1 and height1 correspond to the width and height of the first layout, respectively; width_guide and height_guide correspond to the width and height of the second layout, respectively. The height of the first projection window from the top edge of the screen, i.e., the first preset height, is set to top. Since height1 and height_guide are the same height, the same variable height can be used instead. The first aspect ratio ratio1 of the first projection screen and the second aspect ratio ratio ratio_guide of the guide page are both known constants. The following combined calculations can be used to calculate width1, top, height, and width_guide: width1 + width_guide = 1920; top * 2 + height = 1080; width1 / height = ratio1; width_guide / height = ratio_guide.

[0120] To facilitate differentiation between windows, in some embodiments, a white space is provided between the first projection window and the second guide window, and / or, a white space is provided between the first projection window and the second guide window from the left and right edges of the screen. When setting layout parameters, the display device 200 also sets the sum of the layout height and the width of the white space to be equal to the horizontal pixel value of the screen. For example, the width of the projection screen from the left edge is `left`, the width of the guide page from the right edge is `right`, and the white space between the first projection window and the guide window is 12 pixels. If the screen resolution of the display device 200 is 1920*1080, then `width1 + width_guide + left + right + 12 = 1920` is set.

[0121] S500: Control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area.

[0122] After calculating the first display area and the second display area, the display device 200 displays the first projection window in the first display area and the second display window in the second display area. By calculating the first display area and the second display area using the above algorithm, and displaying the first projection window in the first display area and the second guide window in the second display area, the first projection window can be displayed to the maximum extent while the two windows are at the same height, thus standardizing the layout of each window and improving the user's visual experience.

[0123] In some embodiments, when the display device 200 displays the first projection window in the first display area, it generates the layout position coordinates of the first projection window according to the first layout parameters, and records the first layout parameters and the layout position coordinates. Then, it draws the first projection window according to the first layout parameters and the layout position coordinates.

[0124] Similarly, in some embodiments, when the display device 200 displays the second guide window in the second display area, it generates the layout position coordinates of the second guide window according to the second layout parameters and records the second layout parameters and layout position coordinates. Then, it draws the second projection window according to the second layout parameters and layout position coordinates.

[0125] For example, the display device 200 has a screen resolution of 1920*1080, the first resolution of the first projection screen is 512*1088, the base resolution of the guide page is 954*538, and the width of the blank area between the first projection window and the second guide window is 12 pixels. The first and second layout parameters are calculated based on the algorithm: width1 + width_guide = 1920; top*2 + height = 1080; width1 / height = ratio1; width_guide / height = ratio_guide, where top = 116. The first layout parameters are calculated to have a height of 848 and a width of 400. The coordinates of the top left corner of the first projection window (0, 116) and the top left corner of the guide window (412, 116) are obtained according to these layout parameters. These layout parameters and top left corner coordinates are then stored in the data management module and transmitted to the View management module via the message processing module, so that the View management module can draw the first projection window and the second guide window according to the layout parameters and top left corner coordinates.

[0126] The data management module of display device 200 can organize and store layout parameters and layout position coordinates before transmitting the data. This includes: Session: SessionClient1, View: SurfaceView / TextureView, MView_ewPosition:1, Position:(0, 116), Width:400, Height:848; Session:View:layout, MViewPosition:Position:(412, 116), Width:1508, Height:848. After receiving the data, the View management module of display device 200 sets the display parameters through LayoutParams, including the coordinates of the top-left corner, as well as the corresponding layout width and height, and then displays the position using the setLayoutParams method. That is:

[0127] FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0);lp1.width=400;lp1.height=848;lp1.leftMargin=0;lp1.topMargin=116;mPlayerView.setLayoutPa rams(lp1);

[0128] FrameLayout.LayoutParamslp1=newFrameLayout.LayoutParams(width,height,0);lp1.width=1508;lp1.height=848;lp1.leftMargin=412;lp1.topMargin=116;mPlayerView.setLayoutPa rams(lp1).

[0129] By setting the display positions of the SurfaceView / TextureView and the onboarding page layout through the View management module, you can obtain the following: Figure 11 The display effect of the multi-channel projection interface is shown.

[0130] Display device 200 in display Figure 11 After displaying the multi-screen projection interface, it can also receive projection connection commands sent by other terminal devices 500 and establish a projection connection with those terminal devices. Therefore, as shown... Figure 12 As shown, in some embodiments, the display device 200 further responds to the screen mirroring connection command sent by the second terminal device 520 by creating a second screen mirroring window and releasing a second guide window. The second screen mirroring window is used to display the second screen mirroring screen of the second terminal device 520, and the root node of the second screen mirroring window is the same as that of the multi-channel screen mirroring interface. Then, the second screen mirroring session sent by the second terminal device 520 is acquired. The second screen mirroring session includes the second resolution of the second screen mirroring screen. The screen resolution and the first resolution are then detected, and the window display area is calculated based on the screen resolution, the first resolution, and the second resolution. The window display area includes a third display area and a fourth display area, and the heights of the third and fourth display areas are equal. Then, the display 260 is controlled to display the first screen mirroring window in the third display area and the second screen mirroring window in the fourth display area.

[0131] In some embodiments, when the display device 200 calculates the window display area based on the screen resolution, a first resolution, and a second resolution, it calculates a third aspect ratio based on the ratio of horizontal pixel values ​​to vertical pixel values ​​in the second resolution. A third layout parameter and a fourth layout parameter are set. The third layout parameter includes a third layout width and a third layout height; the fourth layout parameter includes a fourth layout width and a fourth layout height; the third layout height is equal to the fourth layout height; the sum of the third layout width and the fourth layout width is equal to the horizontal pixel value of the screen resolution; the third layout height and the third layout height are less than or equal to the vertical pixel value of the screen resolution; the ratio of the third layout width to the third layout height is equal to the first aspect ratio; and the ratio of the fourth layout width to the fourth layout height is equal to the third aspect ratio.

[0132] For example, the screen resolution of display device 200 is 1920*1080, and the first resolution of the first projected screen is 512*1088. After display device 200 displays the projected window of the first terminal device Client1, and then listens for a new projected connection command from the second terminal device Client2, it synchronously creates the second projected window View corresponding to the second terminal device 520. After the View is successfully created, the position management module is started to set and manage the display position of SessionClient2, and the position is processed and obtained through the position management module. Since the first projected window is already displayed on the left, the orientation marker of the second projected window is set to the right, that is, the mViewPosition of Client2 is marked as 2. The View management data of the managed object includes: Session: SessionClient2; View: SurfaceView / TextureView; MViewPosition: 2. If the second resolution of the second projection screen is obtained as 1552*1088, then the layout parameters and layout position coordinates of the first projection window and the second projection window are calculated according to the above algorithm, and the obtained layout parameters and layout position coordinates are stored in the data management module, including: Session:SessionClient1; View:SurfaceView / TextureView; MViewPosition:1; Position:(0,37); Width:473; Height:1006;

[0133] Session: SessionClient2; View: SurfaceView / TextureView; MViewPosition: 2; Position: (485, 37); Width: 1435; Height: 1006.

[0134] After the data management template records the above data information, it transmits the data information to the view management module through the message management module. The view management module then displays the position according to the transmitted data information using the setLayoutParams method. That is:

[0135] FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0); lp1.width=473; lp1.height=1006; lp1.leftMargin=0; lp1.topMargin=37; mPlayerView.set LayoutParams(lp1);

[0136] FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0); lp1.width=1435; lp1.height=1006; lp1.leftMargin=485; lp1.topMargin=37; mPlayerView.setLayoutParams(lp1).

[0137] By setting the display positions of the first and second projection windows corresponding to the SurfaceView / TextureView respectively through the View management module, you can obtain the following: Figure 13 The display effect of the multi-channel projection interface is shown. The display device 200 displays the first projection window and the second projection window at the same height, and at the same time, it can make both windows display to the maximum extent in the multi-channel projection interface.

[0138] After display device 200 displays the projection window of terminal device 500, it can also respond to the corresponding termination projection command of terminal device 500 to terminate the projection connection between display device 200 and terminal device 500 and remove the projection window of terminal device 500. In some embodiments, the termination projection command can be input from the display device, input from the terminal device, or automatically generated due to external factors such as network issues.

[0139] Therefore, in some embodiments, the display device 200 listens to the screen mirroring connection session of the terminal device 500. Upon receiving a termination command for the first screen mirroring session, in response to the termination command, the first screen mirroring window is removed, and the first guide window is invoked. Since the first guide window is released when the first screen mirroring window is displayed, it needs to be invoked again after the display device 200 removes the first screen mirroring window. For example, the relevant resources of the window object corresponding to the first guide window are invoked again. The screen resolution and the second resolution are then detected, and the window display area is calculated based on the screen resolution, the second resolution, and the reference resolution. The window display area includes a fifth display area and a sixth display area, and the heights of the fifth and sixth display areas are equal. After calculating the fifth and sixth display areas, the display 260 is controlled to display the first guide window in the fifth display area and to display the second screen mirroring window in the sixth display area.

[0140] In some embodiments, when the display device 200 calculates the window display area based on the screen resolution, the second resolution, and the reference resolution, it sets a fifth layout parameter and a sixth layout parameter. The fifth layout parameter includes a fifth layout width and a fifth layout height, and the sixth layout parameter includes a sixth layout width and a sixth layout height. The fifth layout height is equal to the sixth layout height. The sum of the fifth layout width and the sixth layout width is equal to the horizontal pixel value of the screen resolution. The fifth layout height and the sixth layout height are less than or equal to the vertical pixel value of the screen resolution. The ratio of the fifth layout width to the fifth layout height is equal to a third aspect ratio, and the ratio of the sixth layout width to the sixth layout height is equal to a second aspect ratio.

[0141] It is understandable that the display device 200 uses the same algorithm to calculate the fifth and sixth layout parameters as it does to calculate the first, second, third, and fourth layout parameters, and will not be elaborated here.

[0142] Because the terminal device 500 can be placed in different states depending on its usage, such as smartphones, it may be in portrait or landscape mode. Portrait mode refers to a display where the width of the screen is less than its height, while landscape mode refers to a display where the width is greater than its height. In these different placement states, the terminal device 500 can present different screen layouts.

[0143] Based on this, in some embodiments, the resolution of the projected image is different when the display device 200 and terminal device 500 are placed in different states; that is, the projected image will present different aspect ratios under different placement states. For example, when the first terminal device 510 is in landscape mode, the resolution of the first projected image is 1920*1080. When the first terminal device 510 is switched to portrait mode, the size of the first projected image is 1080*1920.

[0144] Obviously, when the resolution of the projected image changes, the display device 200 also needs to recalculate the display area of ​​the projected window according to the changed resolution. Therefore, in some embodiments, when only the first projected window is displayed in the multi-projection interface, the display device 200 also listens for changes in the first projected session. When a change in the first resolution is detected, the changed resolution of the first projected image is detected. Here, the changed resolution is the resolution of the first projected image after the change event is executed. Then, the window display area is calculated based on the screen resolution, the changed resolution, and the base resolution. The window display area includes a seventh display area and an eighth display area, and the heights of the seventh and eighth display areas are equal. Then, the display 260 is controlled to display the first projected window in the seventh display area and to display the second guide window in the eighth display area.

[0145] Similarly, in some embodiments, when only the second projection window is displayed in the multi-display interface, the display device 200 also listens for changes in the second projection session. When a change event for the second resolution is detected, the changed resolution of the second projection screen is detected. The changed resolution is the resolution of the second projection screen after the change event is executed. The window display area is then calculated based on the screen resolution, the changed resolution, and the baseline resolution. The window display area includes a ninth display area and a tenth display area, and the heights of the ninth and tenth display areas are equal. The display 260 is then controlled to display the first guide window in the ninth display area and to display the second projection window in the tenth display area.

[0146] In some embodiments, when multiple display interfaces simultaneously display a first projection interface and a second projection interface, the display device 200 listens for change events in the first projection session and the second projection session. When a change event of the first resolution and / or the second resolution is detected, the window display area is calculated according to the refresh resolution and screen resolution after the change event. That is, when the display device 200 detects a resolution change event in any projection session, it recalculates the window display area to display the projection window according to the newly calculated window display area.

[0147] For example, after the display device 200 establishes a screen projection connection with the first terminal device 510 and the second terminal device 520, the display will show, for example... Figure 13 The multi-screen projection interface shown uses the `onVideoChanged()` method of `UsessionDisplayListener` in the projection management module to listen to the first projection session. When the first terminal device 510 changes from portrait to landscape mode (i.e., the first resolution changes), the message processing module notifies the algorithm processing module of the changed first resolution. The algorithm processing module then recalculates the layout parameters and position coordinates of each window using the algorithm provided in the above embodiment. After calculation, the algorithm processing module notifies the View management module of the data information via the message processing module. The View management module then re-layouts the first and second projection windows according to the transmitted data information, displaying the result as shown. Figure 14 The effect shown.

[0148] To fully utilize the display space of the multi-channel projection interface and reduce large blank areas, in some embodiments, after the display device 200 displays the first projection window, it calculates the aspect ratio of the first projection window based on the first layout width and the first layout height, and compares the window aspect ratio with the first aspect ratio. If the window aspect ratio is not equal to the first aspect ratio, the first projection image in the first projection window is scaled. That is, when the aspect ratio of the projection window is inconsistent with the aspect ratio of the projection image, it indicates that there are blank areas in the projection window besides the display area of ​​the projection image. The display device 200 can then scale the projection image in the projection image to reduce the blank areas within the projection window.

[0149] To facilitate user interaction with the projection window, in some embodiments, when displaying a multi-projection interface, the display device 200 can also display playback control function controls within the multi-projection interface. Users can use the playback control function to perform interactive operations, input control commands, and control the display device 200 to execute interactive responses. For example, since the projection content of the display device 200 and the terminal device 500 is diverse, including mirrored video streams, pushed URL addresses, and different media asset types such as images and music, the playback control function controls are also different for different media asset types. For example, mirrored video projection does not support fast forward / rewind or play / pause operations; pushed URL addresses can perform fast forward, rewind, play, and pause operations; music can perform fast forward, rewind, play, and pause operations; while images do not support fast forward, rewind, play, and pause operations.

[0150] In some embodiments, the playback control function controls of the display device 200 can be displayed in a floating manner above the projection screen, such as... Figure 15 As shown. Alternatively, the playback control function controls of the display device 200 can also be displayed in the edge area of ​​the multi-channel projection interface, such as... Figure 16As shown. For example, the display device 200 can display the aforementioned broadcast control function controls when it detects a focus event on the projection window.

[0151] To balance the display area of ​​the projection window and the display effect of the playback control function controls, in some embodiments, for scenarios where the playback control function controls are displayed in the edge area, after the display device 200 displays the first projection window in the first display area, the display device 200 responds to the focus event of the first projection window and controls the display 260 to display the playback control function controls of the first projection window in the edge area. A second preset height of the edge area is read. The second preset height is greater than the first preset height. The sum of the first layout height and the second preset height is then set to equal the vertical pixel value of the screen resolution, so as to generate the first layout height based on the second preset height. In other words, when the display device 200 displays the playback control function controls in the edge area, the display area of ​​the projection window or guide window can also be compressed proportionally, thereby increasing the edge area of ​​the multi-channel projection interface to facilitate the display of the playback control function controls.

[0152] Alternatively, the display device 200 can respond to the focus event of the first projection window, control the display 260 to display the playback control function controls of the first projection window in the edge area, and detect the maximum display height when displaying the playback control function controls. Then, the first layout height of the first projection window is calculated based on the maximum display height.

[0153] For example, after the display device 200 establishes a screen projection connection with the first terminal device 510 and the second terminal device 520, the display will show, for example... Figure 13The multi-channel projection interface shown has a layout height of 1006 for the first and second projection windows. When the user clicks the first projection window, the display device 200 detects a maximum display height of 864 for the playback control function. Therefore, the layout parameters of the first and second projection windows need to be reset, changing their layout height from 1006 to 864. Simultaneously, the first and second projection windows are proportionally compressed, setting Client 1 to width1 and height1, and Client 2 to width2 and height2. Through the following combined calculations, width1, top, height, and width_guide can be calculated: width1 / height1 = ratio1; width2 / height2 = ratio2. Resetting height1 and height2 to 864 allows the acquisition of width1 and width2 based on the aforementioned ratios. At this time, the data stored in the data management module includes: Session: SessionClient1; View: SurfaceView / TextureView; ViewPosition: 1; Position: (0, 108); Width: 406; Height: 864.

[0154] Session: SessionClient2; View: SurfaceView / TextureView; MViewPosition: 2; Position: (418, 108); Width: 1232; Height: 864.

[0155] The data management module notifies the View management module of the above data information, and sets the display positions of the SurfaceView / TextureView corresponding to the first and second projection windows respectively, so as to obtain the following: Figure 17 The display effect of the multi-channel projection interface is shown.

[0156] Based on the aforementioned display device 200, some embodiments of this application also provide a method for displaying a multi-channel projection window, applied to the display device 200 of the above embodiments. The display device 200 includes a display 260, a communicator 220, and a controller 250. The display 260 is configured to display a multi-channel projection interface, which includes a first guide window and a second guide window. The first guide window and the second guide window are used to display a guide page. The first guide window and the second guide window are the same as the root node of the multi-channel projection interface. The communicator 220 is configured to establish a communication connection with a terminal device. Figure 10As shown, the method includes the following procedural steps:

[0157] S100: In response to the screen mirroring connection command sent by the first terminal device, create a first screen mirroring window, release the first guide window, and retain the display of the second guide window on the multi-channel screen mirroring interface; the first screen mirroring window is used to display the first screen mirroring screen of the first terminal device, and the first screen mirroring window is the same as the root node of the multi-channel screen mirroring interface;

[0158] S200: Obtain the first screen projection session sent by the first terminal device, wherein the first screen projection session includes the first resolution of the first screen projection screen;

[0159] S300: Detect the screen resolution and the reference resolution of the guide page;

[0160] S400: Calculate the window display area based on the screen resolution, the first resolution, and the reference resolution. The window display area includes a first display area and a second display area, and the heights of the first display area and the second display area are equal.

[0161] S500: Control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area.

[0162] As can be seen from the above technical solutions, the display device and multi-channel projection window display method provided in some embodiments of this application can respond to a projection connection command sent by a first terminal device, create a first projection window, and release a first guide window. The first projection window is used to display the first projection screen of the first terminal device. Then, the first projection session sent by the first terminal device is acquired, and the screen resolution and the reference resolution of the guide page are detected. A first display area and a second display area are calculated based on the screen resolution, the first resolution, and the reference resolution. The height of the first display area and the second display area are equal. The display is then controlled to display the first projection window in the first display area and to display the second guide window in the second display area. This method can calculate the display area of ​​each window based on the screen resolution, the projection screen, and the resolution of the guide page, while displaying multiple windows at the same height, so that the projection screen can be displayed to the maximum extent in the multi-channel projection interface, improving the user experience.

[0163] The same or similar parts among the various embodiments in this specification can be referred to mutually, and will not be repeated here.

[0164] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or certain parts of the embodiments of the present invention.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0166] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that, include: The display is configured to show a multi-channel projection interface, which includes a first guide window and a second guide window. The first guide window and the second guide window are used to display a guide page. The first guide window and the second guide window are the same as the root node of the multi-channel projection interface. A communicator, configured to establish a communication connection with a terminal device; The controller is configured as follows: In response to the screen mirroring connection command sent by the first terminal device, a first screen mirroring window is created and the first guide window is released, while the second guide window is retained in the display of the multi-channel screen mirroring interface; the first screen mirroring window is used to display the first screen mirroring screen of the first terminal device, and the first screen mirroring window is the same as the root node of the multi-channel screen mirroring interface; Obtain the first screen projection session sent by the first terminal device, wherein the first screen projection session includes the first resolution of the first screen projection screen; Detect the screen resolution and compare it with the reference resolution of the onboarding page; The window display area is calculated based on the screen resolution, the first resolution, and the reference resolution. The window display area includes a first display area and a second display area, and the heights of the first display area and the second display area are equal. The display is controlled to display the first projection window in the first display area, and the display is controlled to display the second guide window in the second display area.

2. The display device according to claim 1, characterized in that, The controller is also configured to: In response to the screen mirroring connection command sent by the second terminal device, a second screen mirroring window is created and the second guide window is released; the second screen mirroring window is used to display the second screen mirroring screen of the second terminal device, and the root node of the second screen mirroring window is the same as that of the multi-channel screen mirroring interface; Obtain the second screen projection session sent by the second terminal device, wherein the second screen projection session includes the second resolution of the second screen projection screen; Detect the screen resolution and compare it with the first resolution; The window display area is calculated based on the screen resolution, the first resolution, and the second resolution. The window display area includes a third display area and a fourth display area, and the heights of the third display area and the fourth display area are equal. The display is controlled to display the first projection window in the third display area, and the display is controlled to display the second projection window in the fourth display area.

3. The display device according to claim 2, characterized in that, The controller is also configured to: In response to the termination command of the first screen mirroring session, the first screen mirroring window is removed, and the first guide window is invoked; Detect the screen resolution and the second resolution; The window display area is calculated based on the screen resolution, the second resolution, and the reference resolution. The window display area includes a fifth display area and a sixth display area, and the heights of the fifth display area and the sixth display area are equal. The display is controlled to display the first guide window in the fifth display area, and the display is controlled to display the second projection window in the sixth display area.

4. The display device according to claim 1, characterized in that, The controller is configured to calculate the window display area based on the screen resolution, the first resolution, and the reference resolution, and is set to: Read the horizontal and vertical pixel values ​​of the screen resolution; A first aspect ratio is calculated based on the ratio of horizontal pixel values ​​to vertical pixel values ​​in the first resolution, and a second aspect ratio is calculated based on the ratio of horizontal pixel values ​​to vertical pixel values ​​in the reference resolution. A first layout parameter and a second layout parameter are set. The first layout parameter includes a first layout width and a first layout height. The second layout parameter includes a second layout width and a second layout height. The first layout height is equal to the second layout height. The sum of the first layout width and the second layout width is equal to the horizontal pixel value of the screen resolution. The first layout height and the second layout height are less than or equal to the vertical pixel value of the screen resolution. The ratio of the first layout width to the first layout height is equal to the first aspect ratio. The ratio of the second layout width to the second layout height is equal to the second aspect ratio. The first display area is generated according to the first layout parameters, and the second display area is generated according to the second layout parameters.

5. The display device according to claim 4, characterized in that, The controller is configured to control the display to show the first projection window in the first display area, and is also configured to: The layout position coordinates of the first projection window are generated based on the first layout parameters; Record the first layout parameters and the layout position coordinates; The first projection window is drawn according to the first layout parameters and the layout position coordinates.

6. The display device according to claim 4, characterized in that, The controller is also configured to: Read the first preset height of the edge region, the edge region including the display area in the multi-channel projection interface other than the first projection window and the second guide window; The sum of the first layout height and the first preset height is set to equal the vertical pixel value of the screen resolution, so as to generate the first layout height based on the first preset height.

7. The display device according to claim 6, characterized in that, After the controller executes the command to control the display to show the first projection window in the first display area, it is further configured to: In response to the focus event of the first projection window, the display is controlled to show the playback control function controls in the edge area; Read the second preset height of the edge region, where the second preset height is greater than the first preset height; The sum of the first layout height and the second preset height is set to equal the vertical pixel value of the screen resolution, so as to generate the first layout height based on the second preset height.

8. The display device according to claim 4, characterized in that, After the controller executes the command to control the display to show the first projection window in the first display area, it is further configured to: The aspect ratio of the first projection window is calculated based on the width and height of the first layout. Compare the aspect ratio of the window with the first aspect ratio; If the aspect ratio of the window is not equal to the first aspect ratio, the first projection screen in the first projection window is scaled.

9. The display device according to claim 1, characterized in that, The controller is also configured to: Listen for changes in the first screen mirroring session; When the change event of the first resolution is detected, the changed resolution of the first screen projection screen is detected, wherein the changed resolution is the resolution of the first screen projection screen after the change event is executed; The window display area is calculated based on the screen resolution, the changed resolution, and the base resolution. The window display area includes a seventh display area and an eighth display area, and the heights of the seventh display area and the eighth display area are equal. The display is controlled to display the first projection window in the seventh display area, and the display is controlled to display the second guide window in the eighth display area.

10. A method for displaying a multi-channel projection window, characterized in that, The invention is applied to a display device, which includes a display, a communicator, and a controller. The display is configured to display a multi-channel projection interface, which includes a first guide window and a second guide window. The first guide window and the second guide window are used to display a guide page. The first guide window and the second guide window are the same as the root node of the multi-channel projection interface. The communicator is configured to establish a communication connection with a terminal device; the method includes: In response to the screen mirroring connection command sent by the first terminal device, a first screen mirroring window is created and the first guide window is released, while the second guide window is retained in the display of the multi-channel screen mirroring interface; the first screen mirroring window is used to display the first screen mirroring screen of the first terminal device, and the first screen mirroring window is the same as the root node of the multi-channel screen mirroring interface; Obtain the first screen projection session sent by the first terminal device, wherein the first screen projection session includes the first resolution of the first screen projection screen; Detect the screen resolution and compare it with the reference resolution of the onboarding page; The window display area is calculated based on the screen resolution, the first resolution, and the reference resolution. The window display area includes a first display area and a second display area, and the heights of the first display area and the second display area are equal. The display is controlled to display the first projection window in the first display area, and the display is controlled to display the second guide window in the second display area.

Images