Display device and display method of multi-path screen projection window

Abstract

Some embodiments of the invention provide a display device and a method for displaying multiple screen projection windows, and the method can create a first screen projection window and release a first guide window in response to a screen projection connection instruction sent by a first terminal device. Obtaining a first screen projection session sent by the first terminal device, and detecting the screen resolution and the reference resolution of the guide page; calculating a first display area and a second display area according to the screen resolution, the first resolution and the reference resolution; wherein the height of the first display area is equal to that of the second display area. And then controlling the display to display a first screen projection window in the first display area and display a second guide window in the second display area. According to the method, the display area of each window can be calculated according to the screen resolution, the screen projection picture and the resolution of the guide page on the premise that a plurality of windows are displayed at the same height, so that the screen projection picture can be displayed in a multi-path screen projection interface to the greatest extent, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a display method for multiple projection windows. Background Art

[0002] Display devices are intelligent devices that can present user interfaces and support user interaction. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users. Display devices can establish communication connections with terminal devices such as mobile phones, and based on specific screen projection protocols, obtain the screen projection data of terminal devices, so as to display the screen projection images of terminal devices through display devices.

[0003] Some display devices can establish communication connections with multiple terminal devices at the same time and receive projection data from multiple terminal devices to form a multi-channel projection scenario. However, in a multi-channel projection scenario, the display device can only display the projection window of a single terminal device. If projection windows for multiple terminal devices are created at the same time, since the projection windows cannot communicate directly with each other, the projection windows on the display device may block each other. Summary of the invention

[0004] The present application provides a display device and a display method for multiple screen projection windows to solve the problem of screen projection windows of the display device blocking each other.

[0005] In a first aspect, some embodiments of the present application provide a display device, including a display, a communicator, and a controller. The display is configured to display a multi-channel screen projection interface, the multi-channel screen projection interface includes a first guide window and a second guide window, the first guide window and the second guide window are used to display a guide page; the first guide window, the second guide window and the root node of the multi-channel screen projection interface are the same;

[0006] A communicator configured to establish a communication connection with a terminal device;

[0007] The controller is configured as:

[0008] In response to the screen projection connection instruction sent by the first terminal device, a first screen projection window is created, the first guide window is released, and the display of the second guide window on the multi-way screen projection interface is retained; the first screen projection window is used to display the first screen projection screen of the first terminal device, and the first screen projection window is the same as the root node of the multi-way screen projection interface;

[0009] Obtain a first screen projection session sent by the first terminal device, where the first screen projection session includes a first resolution of a first screen projection picture;

[0010] Detecting the screen resolution and the base resolution of the guide page;

[0011] Calculate a window display area according to the screen resolution, the first resolution and the reference resolution, the window display area includes a first display area and a second display area, and the first display area and the second display area have the same height;

[0012] Control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area.

[0013] In a second aspect, some embodiments of the present application further provide a method for displaying a multi-way screen projection window, using the display device provided in the first aspect, the display device comprising a display, a communicator and a controller; the display is configured to display a multi-way screen projection interface, the multi-way screen projection interface comprising a first guide window and a second guide window, the first guide window and the second guide window are used to display a guide page; the first guide window, the second guide window and the root node of the multi-way screen projection interface are the same; the communicator is configured to establish a communication connection with a terminal device; the method comprises:

[0014] In response to the screen projection connection instruction sent by the first terminal device, a first screen projection window is created, the first guide window is released, and the display of the second guide window on the multi-way screen projection interface is retained; the first screen projection window is used to display the first screen projection screen of the first terminal device, and the first screen projection window is the same as the root node of the multi-way screen projection interface;

[0015] Obtain a first screen projection session sent by the first terminal device, where the first screen projection session includes a first resolution of a first screen projection picture;

[0016] Detecting the screen resolution and the base resolution of the guide page;

[0017] Calculate a window display area according to the screen resolution, the first resolution and the reference resolution, the window display area includes a first display area and a second display area, and the first display area and the second display area have the same height;

[0018] Control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area.

[0019] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a display method for multiple projection windows. The method can create a first projection window and release the first guide window in response to a projection connection instruction sent by a first terminal device. Among them, the first projection window is used to display the first projection screen of the first terminal device. Then obtain the first projection session sent by the first terminal device, and detect the screen resolution and the reference resolution of the guide page. Calculate the first display area and the second display area according to the screen resolution, the first resolution and the reference resolution. Among them, the height of the first display area is equal to that of the second display area. Then control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area. The method can calculate the display area of ​​each window according to the screen resolution, the projection screen and the resolution of the guide page on the premise of displaying multiple windows at the same height, so that the projection screen can be displayed in the multi-projection interface to the greatest extent, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram showing usage scenarios of a display device in some embodiments of the present application;

[0021] Figure 2 A hardware configuration block diagram of a display device provided in some embodiments of the present application;

[0022] Figure 3 A hardware configuration block diagram of a control device provided in some embodiments of the present application;

[0023] Figure 4 A schematic diagram of software configuration in a display device provided in some embodiments of the present application;

[0024] Figure 5 A schematic diagram of mirror screen projection provided in some embodiments of the present application;

[0025] Figure 6 A schematic diagram of push screen projection provided in some embodiments of the present application;

[0026] Figure 7 A schematic diagram of a multi-screen projection interface provided in some embodiments of the present application;

[0027] Figure 8 A schematic diagram of a functional mode of a display device provided in some embodiments of the present application;

[0028] Fig. 9 A schematic diagram of a guide page provided for some embodiments of the present application;

[0029] Fig.10 A schematic flow chart of a multi-screen projection window display method provided in some embodiments of the present application;

[0030] Fig.11 A schematic diagram of the display effect of a multi-screen projection interface when projecting a screen on one channel provided in some embodiments of the present application;

[0031] Fig.12 A schematic diagram of a process for displaying a second projection window provided in some embodiments of the present application;

[0032] Fig.13 A schematic diagram of the display effect of a multi-channel screen projection interface when two-channel screen projection is provided in some embodiments of the present application;

[0033] Fig.14 A schematic diagram of the display effect of a multi-channel screen projection interface during another two-channel screen projection provided in some embodiments of the present application;

[0034] Fig.15 A schematic diagram of the display effect of the floating display play control function control provided in some embodiments of the present application;

[0035] Fig.16 A schematic diagram of the display effect of displaying a playback control function control in an edge area provided in some embodiments of the present application;

[0036] Fig.17 A schematic diagram of the display effect after compressing the projection window provided in some embodiments of the present application. DETAILED DESCRIPTION

[0037] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0038] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0039] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0040] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0041] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0042] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 may be a remote controller, a stylus pen, or the like.

[0043] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0044] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0045] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0046] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0047] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0048] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0049] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0050] In some embodiments, the communicator 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communicators 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communicator 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communicator 220 including a Bluetooth function.

[0051] The communicator 220 can connect the display device 200 to the external device or the server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 to the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 to the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0052] In some embodiments, the display device 200 may also support establishing a communication connection relationship with multiple external devices at the same time. Multiple external devices may be connected through the same. For example, the external device is a terminal device 500 such as a mobile phone or a tablet computer. The first terminal device 510 and the second terminal device 520 may establish a communication connection with the display device 200 by accessing the wireless local area network where the display device 200 is located. Multiple external devices may also be connected to the display device 200 through different types of connection methods. For example, the first terminal device 510 is connected to the display device 200 via a wireless local area network; the second terminal device 520 is connected to the display device 200 via Bluetooth.

[0053] In order to realize the communication connection between the display device 200 and the terminal device 500, the display device 200 and the terminal device 500 are respectively provided with communicators supporting the same communication type. For example, when the display device 200 is provided with a communicator 220 including a WiFi module, the terminal device 500 should also be provided with a communicator including a WiFi module. In addition, when the same type of communicators are provided, a specific communication transmission protocol is also required between the display device 200 and the terminal device 500 for data transmission. For example, WiFi protocol, Bluetooth connection protocol, ZigBee protocol, NFC protocol, etc.

[0054] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0055] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0056] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0057] In some embodiments, the user interface 280 is an interface that can be used to receive control input.

[0058] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0059] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0060] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0061] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0062] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0063] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0064] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0065] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, which can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0066] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0067] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0068] like Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0069] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0070] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0071] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0072] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0073] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0074] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0075] After establishing a communication connection relationship with the terminal device 500, the display device 200 can establish a screen projection connection channel, and transmit the projection data based on the screen projection connection, so that the display device 200 can display the projection screen of the terminal device 500. In some embodiments, in order to establish a screen projection connection relationship, the user can send a screen projection connection request through the terminal device 500, and send the screen projection connection request to the display device 200 through the WiFi network. The display device 200 then completes the transmission protocol configuration according to the screen projection connection request, thereby establishing a transmission channel for the projection data with the terminal device 500.

[0076] The display device 200 can also achieve a screen projection connection with the terminal device 500 through different connection methods. For example, when the display device 200 and the terminal device 500 are connected to the same wireless local area network, a screen projection connection can be established based on a WiFi network. For another example, when both the display device 200 and the terminal device 500 are provided with NFC (Near Field Communication) components, a screen projection connection relationship can be established through the NFC component. Obviously, other wired or wireless connection methods can also be used between the display device 200 and the terminal device 500 to establish a screen projection connection relationship, such as RF radio frequency connection, infrared connection, cellular network, and other connection methods that can be associated with those skilled in the art based on the connection methods provided in the above embodiments.

[0077] After the screen projection connection is established, the display device 200 can receive the screen projection related data of the terminal device 500 through the screen projection data channel, so as to play the screen projection related content in the display device 200. Under different screen projection data transmission protocols, the screen projection data obtained by the display device 200 is in different forms. For example, the display device 200 and the terminal device 500 can establish a screen projection connection through the Digital Living Network Alliance (DLNA) protocol, Miracast protocol, AirPlay push protocol, WiDi screen projection, NFC mirror screen projection, LeBo screen projection and other protocols.

[0078] Among them, the DLNA protocol is a protocol for sharing media resources between devices. The protocol uses network technology (wired or wireless) to interconnect various devices and exchange data through standard protocols, so that media resources can be shared between devices. The DLNA protocol is based on the UPnP protocol, which enables devices to discover and control each other and achieve seamless transmission and sharing of media resources. The DLNA protocol supports a variety of devices, including TVs, speakers, mobile phones, computers, etc., allowing users to seamlessly switch and share media content between different devices.

[0079] Miracast protocol is a wireless display standard based on Wi-Fi direct connection and developed by the Wi-Fi Alliance. Miracast screen projection protocol is applicable to Android devices, that is, Android devices can share video images through Miracast screen projection. Smart devices with Android 5.0 or above have built-in Miracast protocol, so you can directly wirelessly project the screen through the WiFi P2P function between the projection device and the WiFi network card of the receiving device without installing any software. In addition, Miracast screen projection also supports synchronous transmission of audio and video.

[0080] Airplay protocol is a wireless playback technology launched by Apple, which is applicable to the wireless screen projection protocol of iOS and Mac systems. Apple devices have built-in Airplay screen projection function. When used with a screen projection device that supports AirPlay server service, the effect is clear and stable. The protocol comes with a screen projection code function. However, it is necessary to ensure that the Apple device and the screen projection device are used in the same LAN (same network segment), and cannot be used across network segments or VLANs.

[0081] WiDi screen projection is a wireless playback technology led by Intel, and is also based on WiFi Direct technology. Windows 8 / 10 / 11 laptops natively support WiDi screen projection, which can be directly projected without installing any software, and supports USB reverse control function.

[0082] In some embodiments, after the display device 200 establishes a screen projection connection with the terminal device 500, the screen projection data can be transmitted through the mirror mode and the push mode. Among them, the mirror mode means that the display device 200 synchronously displays the screen content displayed by the terminal device 500. For example, Figure 5 As shown, the display device 200 displays the main interface of the terminal device 500, and when the user performs an interactive operation on the terminal device 500, resulting in a change in the content of the main interface, the content displayed by the display device 200 also changes. To this end, the terminal device 500 can generate a screen projection data stream in the form of a video based on the displayed content, and send the screen projection data stream to the display device 200 for display.

[0083] The push mode means that the display device 200 accesses the network link according to the network link pushed by the terminal device 500 to obtain the corresponding projection screen. Figure 6 As shown, when the terminal device 500 plays the video media, a screen projection control is provided in the upper right corner of the play interface. After the user clicks the screen projection control and selects the display device 200 as the device for displaying the screen projection image, the terminal device 500 can send the URL address of the currently played video media to the display device 200. After receiving the URL address, the display device 200 accesses the URL address to obtain the video media data and plays the video media.

[0084] Among the screen projection protocols supported by the display device 200 and the terminal device 500, some protocols are based on the mirror mode, and some protocols are based on the push mode. For example, under the DLNA protocol, the display device 200 can obtain the link address of a resource in the terminal device 500 through the screen projection data channel, and obtain the media content data by accessing the link address, thereby playing the media data. At the same time, the display device 200 can also obtain control instructions sent by the terminal device 500 through the screen projection data channel when playing media data, such as fast forward, pause, stop screen projection, etc. For example, after establishing a connection through the communication protocol, the terminal device 500 such as a mobile phone and the display device 200 will negotiate the device functions and network conditions to select a suitable audio and video transmission format. After establishing the online protocol (Session) of the audio and video streaming, a series of real-time streaming protocol (RTSP) control commands are used to control the playback and termination of the audio and video streaming.

[0085] Under other screen projection connection protocols such as Miracast, Airplay, NFC mirroring, and LeTV, the display device 200 can directly obtain the video data stream from the terminal device 500. The video data stream can be the video data stream corresponding to the final display screen of the display unit of the terminal device 500, that is, the video data stream includes the media content screen, UI interface, and other video screen content that can be displayed in the display unit of the terminal device 500. The video data stream can also be only the video data that can be obtained in the terminal device 500, such as the media video data being played. At this time, the terminal device 500 can be used as a signal source for the display device 200, and the display device 200 can continuously obtain video data through the screen projection data channel, and display it in real time on the display 260, so as to achieve the effect of synchronous display with the terminal device 500.

[0086] It should be noted that the above-mentioned screen projection protocol can not only obtain the video data stream in the terminal device 500, but also obtain the audio data stream in the terminal device 500, so that the video data and audio data can be sent to the display device 200 synchronously, so that the display device 200 can play the video and audio synchronously.

[0087] The terminal device 500 can obtain different projection data according to different projection protocols. For example, for system applications or direct projection at the system level, the terminal device 500 can use Miracast projection, Airplay projection or WiDi projection protocol to establish a projection data channel. Since the above-mentioned Miracast projection protocol and Airplay projection protocol require system-level application permissions, the Miracast projection protocol and Airplay projection protocol are only applicable to system applications, or the display device 200 and the terminal device 500 of the same manufacturer directly establish a projection connection through the built-in rules of the operating system.

[0088] The display device 200 can establish a screen projection connection with the terminal device 500 based on any of the screen projection methods provided in the above embodiments. In some embodiments, the display device 200 can also establish a screen projection connection relationship with multiple communication terminals through one or more of the above screen projection methods. For example, the display device 200 can simultaneously establish a screen projection connection with the first terminal device 510 and the second terminal device 520, and divide the display area in the display screen to display the projection screen shared by the first terminal device 510 and the second terminal device 520 respectively.

[0089] In order to display the projection screen shared by multiple terminal devices 500 on the display device 200, in some embodiments, when the display device 200 establishes a projection connection with multiple terminal devices 500, the display device 200 can display a multi-channel projection interface, which can include multiple projection windows. The projection window is used to display the projection screen of the corresponding terminal device 500 under a projection connection channel. For example, Figure 7 As shown, the multi-channel screen projection interface may include two screen projection windows, namely a first screen projection window and a second screen projection window. The first screen projection window is used to display the projection screen of the first terminal device 510; the second screen projection window is used to display the projection screen of the second terminal device 520.

[0090] It should be noted that the projection window in the multi-channel projection interface may not be strongly bound to the terminal device 500, that is, in some embodiments, the display content of the projection window may be determined according to the order in which the terminal device 500 establishes a projection connection with the display device 200. For example, when the first terminal device 510 establishes a projection connection with the display device 200 before the second terminal device 520, the first projection window on the left side of the multi-channel projection interface displays the projection screen of the first terminal device 510. When the first terminal device 510 establishes a projection connection with the display device 200 later than the second terminal device 520, the first projection window on the left side of the multi-channel projection interface displays the projection screen of the second terminal device 520.

[0091] In order to obtain a better picture display effect and improve the coordination of the multi-channel projection interface, in some embodiments, the display device 200 can also set the layout of the projection window. Figure 8 As shown, the display device 200 can have multiple functional modules built into the same application, such as a multi-channel screen projection application. The multi-channel screen projection application includes: a data management module, a message processing module, a service management module, an algorithm processing module, a View management module, an application interaction processing module, a location management module, a replacement management module, a broadcast control management module, a time management module, a focus management module, and a screen projection management module.

[0092] Among them, the data management module mainly manages all message data involved in the current template and updates the data information synchronously. The message processing module mainly transmits related messages between different modules, such as transmitting data changes to the View layer for refresh display, or synchronizing View changes to the data management module for change processing of broadcast control data, etc. The service management module is responsible for opening and closing the services of the entire application and some customized processing, such as starting specific services in specific scenarios. The algorithm processing module is responsible for all layout position calculations of the entire application. The View management module is mainly responsible for all UI displays of the entire template application, and receives data to refresh the View display.

[0093] The application interaction processing module is responsible for the processing of interactions with external applications. For example, any entry to start the current template application needs to interact with this module, and different customized management is performed according to the differences in the startup entries. The position management module is responsible for the rules for displaying the projection position in the left and right windows, such as on the left or on the right. The replacement management module is responsible for the replacement rules when the number of projection paths exceeds the maximum display number, such as how to replace when there is focus or no focus. The broadcast control management module is responsible for the broadcast control processing of different media types in the projection, such as fast forward, rewind, play pause, etc. The time management module is responsible for the focus display timing processing in the projection, whether it is the large frame focus or the broadcast control position focus. The focus management module is responsible for the focus display and button processing in the entire application. The projection management module is responsible for the projection protocol interaction and the client list management and window display content status information processing.

[0094] It should be noted that the multi-channel screen projection interface can realize screen display based on the multi-channel screen projection viewing application. The multi-channel screen projection viewing application is an application that can display multi-protocol multi-channel mirror projection and push content, and is an upper-level application inside the display device 200. The framework layer of the display device 200 can provide a Java interface for docking with hardware, etc. The multi-channel screen projection viewing application realizes docking with the lower layer and obtains information through the Java interface of the Framework layer. Other National, Runtime, HAL, and Linux kernel layers are all standard layers of the Android architecture, and each layer interacts according to the business process.

[0095] The display device 200 first starts the application through the service management module and performs some initial preparations, such as starting the monitoring mirror or pushing the corresponding protocol service; the View management module initializes the View displayed by the application layer; the data management module creates some data for the data processing module; the algorithm processing module performs algorithmic layout calculations on the projection content, and the position management module is responsible for positioning the left and right positions of the projection content.

[0096] The service management module can start the screen projection application and register the Session during the process of establishing the screen projection connection; Usession Connection Listener, Usession Prepare Listener, Usession DisplayListener and Usession Event Listener listen. Among them, Session represents a conversation between the client and the server, which can save the information of the Client. Uses ession Connection Listener is used to monitor the connection and disconnection events of the Session. Usession Prepare Listener interacts with the screen projection preparation of the Client. UsessionDisplay Listener monitors the changes in playback information and status and the status changes of the Client. Usession EventListener processes information such as Client control.

[0097] After the initialization of View is completed, the display device 200 will display the default guide window when entering the application. The guide window is used to display the guide page. The guide page will display some device information, such as the device name of the current display device 200 and the corresponding network information, that is, the guide page includes TextView and ImageView that can include multiple system standards. The display device 200 can display the guide page in equal proportions on the left and right by controlling the TextView and ImageView controls and the corresponding Layout, such as Fig. 9 The guide page includes a reference resolution, and when the display device 200 displays the guide page through the guide window, it is necessary to scale the guide page according to the reference resolution.

[0098] In some embodiments, the guide window and the multi-channel projection interface share a root, that is, the root node of the guide window and the multi-channel projection interface is the same. For example, after the user runs the multi-channel projection application in the display device 200, the display device 200 controls the display 260 to display the multi-channel projection interface in full screen, and displays the guide window in the corresponding position of the multi-channel projection interface. Among them, the multi-channel projection interface is the homepage interface of the multi-channel projection application, and the guide window and the homepage interface of the multi-channel projection application use the same root.

[0099] After displaying the guide window, the display device 200 can monitor and receive the screen projection connection instruction sent by the terminal device 500, and establish a screen projection connection channel according to the screen projection connection instruction, thereby displaying the projection screen of the terminal device in the screen projection window.

[0100] Some embodiments of the present application provide a display device 200, including a display 260, a communicator 220 and a controller 250. The display 260 is configured to display a multi-channel screen projection interface, and the multi-channel screen projection interface includes a first guide window and a second guide window. The first guide window and the second guide window are used to display a guide page, and the first guide window, the second guide window and the root node of the multi-channel screen projection interface are the same; the communicator 220 is configured to establish a communication connection with the terminal device 500. Fig.10 As shown, the controller 250 is configured to perform the following program steps:

[0101] S100: In response to the screen projection connection instruction sent by the first terminal device, create a first screen projection window, release the first guide window, and keep the second guide window displayed in the multi-way screen projection interface.

[0102] After the display device 200 displays the default guide window, the user can establish a communication connection between the terminal device 200 and the display device 200 based on the prompt information in the guide window. When the user sends a screen projection connection instruction to the display device 200 through the first terminal device 510, the display device 200 responds to the screen projection connection instruction and creates a first screen projection window. The first screen projection window is used to display the first screen projection picture, and the first screen projection window is the same as the root node of the multi-channel screen projection interface, that is, the first screen projection window also uses the same root as the multi-channel screen projection interface.

[0103] For example, the screen projection management module of the display device 200 is always ready to access a new Session. When the first terminal device 510Client1 is ready to start a connection, the screen projection management module listens to the onConnect connection of the new Session through UsessionConnectionListener, and then executes the creation of the first projection window. The first projection window can include the playback control SurfaceView / TextureView, and the display device 200 can set the first projection screen to the corresponding position of SurfaceView / TextureView.

[0104] When the display device 200 creates the first projection window, it also releases the first guide window so that it is not presented in the multi-screen projection interface. At the same time, the display of the second guide window in the multi-screen projection interface is retained. When the display device 200 releases the first guide window, the related resources of the window object corresponding to the first guide window can be released.

[0105] In order to facilitate the management of the display orientation of the projection window, in some embodiments, when the display device 200 creates the first projection window, it also sets the orientation identifier of the first projection window through the position management module. Among them, the orientation identifier is used to characterize the display orientation of the projection window, and the display device 200 can display the first projection window according to the orientation identifier. When setting the orientation identifier, the display device 200 can set the orientation identifier of the projection window according to the projection order of the terminal device 500 according to the default priority order. The orientation identifier can be different numbers or English characters. For example, the display orientation of the projection window includes the left side and the right side, and the default priority order is left side>right side, the orientation identifier of the left side is 1, and the orientation identifier of the right side is 2; after receiving the projection connection instruction of the first terminal device 510, the position management module sets the orientation identifier of the first projection window to 1 according to the default priority order, that is, sets mViewPosition:1.

[0106] In order to facilitate the management of the first projection window and the first projection session of the first terminal device 510, in some embodiments, the display device 200 is further provided with a management object MultiCastPlayView, wherein the management object MultiCastPlayView can be associated with information such as Session, SurfaceView / TextureView, and mViewPosition. For example, for the first terminal device 510, the data corresponding to its Client1 can be represented as: Session:SessionClient1, View:SurfaceView / TextureView, M ViewPosition:1.

[0107] S200: Obtain a first screen projection session sent by a first terminal device.

[0108] After completing the creation of the first projection window, the display device 200 obtains the first projection session sent by the first terminal device 510, so as to obtain the projection data information based on the projection data sent by the first terminal device 510 through the first projection session. The projection data information includes information associated with the projection connection between the first terminal device and the display device, such as the resolution of the first projection screen and other attributes, that is, the first projection session includes the first resolution of the first projection screen. The display device 200 can obtain the first resolution of the first projection screen by obtaining the first projection session SessionClient1, and then calculate the display area of ​​the first projection window based on the first resolution.

[0109] For example, when the display device 200 prepares the first projection window view, it can notify the first terminal device Client1 through UsesionPrepareListener to record or send the video stream. When the terminal device 500 is ready to send video information, the display device 200 knows that the first projection screen will be played through onFirstFrame() of UsesionDisplayListener, so it obtains the first resolution of the current Client1 corresponding to the first projection screen from SessionClient1, that is, the width and height of the video stream. Then determine the direction of the current video stream, and calculate the display position and size of the current view through an algorithm.

[0110] S300: Detecting the screen resolution and the reference resolution of the guide page.

[0111] After the display device 200 releases the first guide window and creates the first projection window, the windows displayed in the multi-channel projection interface include the first projection window and the second guide window. Therefore, after the display device 200 obtains the first resolution of the first projection screen in the first projection session, it detects the screen resolution of the display device 200 and the base resolution of the guide page, so as to determine the display areas of the first projection window and the second guide window through the screen resolution, the first resolution of the first projection screen and the base resolution of the guide page.

[0112] The screen resolution and the reference resolution of the guide page are information pre-stored in the display device 200, and the display device 200 can detect the screen resolution of the display device 200 and the reference resolution of the guide page at a designated storage location. For example, the reference resolution of the guide page can be stored in a layout file of a multi-channel screen projection application, and the display device 200 can detect the layout information of the guide page in the layout file to obtain the reference resolution.

[0113] S400: Calculating a window display area according to the screen resolution, the first resolution and the reference resolution.

[0114] After obtaining the screen resolution, the first resolution and the reference resolution, the display device 200 can calculate the display area of ​​each window according to the above resolutions. Among them, the window display area includes a first display area and a second display area, the first display area is used to display the first projection window, and the second display area is used to display the second guide window. In order to improve the user's visual experience, the height of the first display area and the second display area are equal, so that each window can maintain a consistent height, avoiding the problem of users repeatedly switching visual heights when viewing multiple windows at the same time.

[0115] In some embodiments, when the display device 200 calculates the window display area according to the screen resolution, the first resolution and the reference resolution, the horizontal pixel value and the vertical pixel value of the screen resolution are read. Then the first aspect ratio is calculated according to the ratio of the horizontal pixel value to the vertical pixel value in the first resolution, and the second aspect ratio is calculated according to the ratio of the resolution to the vertical pixel value in the reference resolution. For example, the screen resolution of the display device 200 is 1920*1080, the first resolution of the first projection screen is 1920*1080, and the reference resolution of the guide page is 954*538. The horizontal pixel value in the screen resolution is 1920, and the vertical pixel value is 1080; the horizontal pixel value of the first resolution is 1920, and the vertical pixel value is 1080. The ratio of the two is calculated to obtain a first aspect ratio of 16 / 9; the horizontal pixel value of the reference resolution is 954, and the vertical pixel value is 1080. The ratio of the two is calculated to obtain a second aspect ratio of 477 / 269, which is close to 16 / 9.

[0116] Then set the first layout parameter and the second layout parameter. The first layout parameter includes the first layout width and the first layout height, and the second layout parameter includes the second layout width and the second layout height; the first layout height is equal to the second layout height, the sum of the first layout width and the second layout width is equal to the horizontal pixel value of the screen resolution, the first layout height and the second layout height are less than or equal to the vertical pixel value of the screen resolution, the ratio of the first layout width to the first layout height is equal to the first aspect ratio, and the ratio of the second layout width to the second layout height is equal to the second aspect ratio. Then generate the first display area according to the first layout parameters, and generate the second display area according to the second layout parameters.

[0117] In order to improve the standardization of each window layout, in some embodiments, the multi-channel projection interface of the display device 200 includes an edge area, which is the display area other than the display window and the guide window in the multi-channel projection interface, and can be called a blank area. In addition, the edge area can also be used to display the playback control function controls or prompt information. Therefore, the sum of the layout height and the edge area height is set to be equal to the vertical pixel value of the screen.

[0118] In some embodiments, when setting the first layout parameter, the display device 200 reads the first preset height of the edge area. The edge area includes the display area other than the first display window and the second guide window in the multi-channel projection interface. Then, the sum of the first layout height and the first preset height is set to be equal to the vertical pixel value of the screen resolution to generate the first layout height based on the first preset height.

[0119] Taking the screen resolution of the display device 200 as 1920*1080 as an example, when the display device 200 sets the above-mentioned first layout parameters and the second layout parameters, according to the first resolution and the reference resolution, set Client 1: width1, height1; Guide (guide page): width_guide, height_guide. Among them, width1 and height1 correspond to the first layout width and the first layout height respectively; width_guide and height_guide correspond to the second layout width and the second layout height respectively. The height of the first projection window from the upper edge of the screen, that is, the first preset height is set to top. Since height1 and height_guide are the same height, the same variable height can be used instead. The first aspect ratio ratio1 of the first projection screen and the second aspect ratio ratio_guide of the guide page are both known constants. Through the following combined calculations, width1, top, height, and width_guide can be calculated; that is, width1+width_guide=1920; top*2+height=1080; width1 / height=ratio1; width_guide / height=ratio_guide.

[0120] In order to facilitate the distinction between each window, in some embodiments, a blank area is provided between the first projection window and the second guide window, and / or a blank area is provided between the first projection window and the second guide window at the left and right edges of the screen. When setting the layout parameters, the display device 200 also sets the sum of the layout height and the width of the blank area to be equal to the horizontal pixel value of the screen. For example, the width of the projection screen from the left edge is left, the width of the guide page from the right edge is right, and the blank space between the first projection window and the guide window is 12 pixels. If the screen resolution of the display device 200 is 1920*1080, width1+width_guide+left+right+12=1920 is set.

[0121] S500: Control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area.

[0122] After calculating the first display area and the second display area, the display device 200 displays the first projection window in the first display area and displays the second display window in the second display area. By calculating the first display area and the second display area through the above algorithm, and displaying the first projection window in the first display area and the second guide window in the second display area, the first projection window can be displayed to the maximum extent under the premise that the two windows are of equal height, and the layout of each window can be standardized, thereby improving the user's visual experience.

[0123] In some embodiments, when the display device 200 displays the first projection window in the first display area, it generates the layout position coordinates of the first projection window according to the first layout parameters, and records the first layout parameters and the layout position coordinates, and then draws the first projection window according to the first layout parameters and the layout position coordinates.

[0124] Similarly, in some embodiments, when the display device 200 displays the second guide window in the second display area, the layout position coordinates of the second guide window are generated according to the second layout parameters, and the second layout parameters and the layout position coordinates are recorded, and then the second projection window is drawn according to the second layout parameters and the layout position coordinates.

[0125] For example, the screen resolution of the display device 200 is 1920*1080, the first resolution of the first projection screen is 512*1088, the base resolution of the guide page is 954*538, and the width of the blank area between the first projection window and the second guide window is 12 pixels. The first layout parameter and the second layout parameter are calculated based on the algorithm width1+width_guide=1920; top*2+height=1080; width1 / height=ratio1; width_guide / height=ratio_guide, where top=116. The first layout height in the first layout parameter is calculated to be 848 and the first layout width is 400. According to the above layout parameters, the upper left corner coordinates (0, 116) of the first projection window and the upper left corner coordinates (412, 116) of the guide window are obtained, and then the above layout parameters and the upper left corner coordinates are stored in the data management module, and the above data is transmitted to the View management module through the message processing module, so that the View management module draws the first projection window and the second guide window according to the layout parameters and the upper left corner coordinates.

[0126] That is, the data management module of the display device 200 can sort and store the layout parameters and layout position coordinates, and then transmit the data, including: Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1, Position: (0, 116), Width: 400, Height: 848; Session: View: layout, MViewPosition: Position: (412, 116), Width: 1508, Height: 848. After receiving the data information, the View management module of the display device 200 sets the display parameters through LayoutParams, including the upper left corner coordinates, and the corresponding layout width and layout height, and then displays the position through the setLayoutParams method. That is:

[0127] FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0);lp1.width=400;lp1.height=848;lp1.leftMargin=0;lp1.topMargin=116;mPlayerView.setLayoutPa rams(lp1);

[0128] FrameLayout.LayoutParamslp1=newFrameLayout.LayoutParams(width,height,0);lp1.width=1508;lp1.height=848;lp1.leftMargin=412;lp1.topMargin=116;mPlayerView.setLayoutPa rams(lp1).

[0129] By setting the display position of SurfaceView / TextureView and the display position of the guide page Layout respectively through the View management module, you can get the following Fig.11 The display effect of the multi-channel screen projection interface is shown.

[0130] The display device 200 is displaying Fig.11 After the multi-channel screen projection interface shown, it can also receive the screen projection connection instruction sent by other terminal devices 500 and establish a screen projection connection relationship with the terminal device. Fig.12 As shown, in some embodiments, the display device 200 also creates a second projection window and releases the second guide window in response to the projection connection instruction sent by the second terminal device 520, wherein the second projection window is used to display the second projection screen of the second terminal device 520, and the second projection window is the same as the root node of the multi-way projection interface. Then obtain the second projection session sent by the second terminal device 520. Among them, the second projection session includes the second resolution of the second projection screen. Then detect the screen resolution and the first resolution, and calculate the window display area according to the screen resolution, the first resolution and the second resolution, the window display area includes a third display area and a fourth display area, and the height of the third display area is equal to that of the fourth display area. Then control the display 260 to display the first projection window in the third display area, and control the display 260 to display the second projection window in the fourth display area.

[0131] In some embodiments, when the display device 200 calculates the window display area according to the screen resolution, the first resolution, and the second resolution, the third aspect ratio is calculated according to the ratio of the horizontal pixel value to the vertical pixel value in the second resolution. The third layout parameter and the fourth layout parameter are set. Among them, the third layout parameter includes the third layout width and the third layout height, the fourth layout parameter includes the fourth layout width and the fourth layout height, and the third layout height is equal to the fourth layout height; the sum of the third layout width and the fourth layout width is equal to the horizontal pixel value of the screen resolution, the third layout height and the third layout height are less than or equal to the vertical pixel value of the screen resolution, the ratio of the third layout width to the third layout height is equal to the first aspect ratio, and the ratio of the fourth layout width to the fourth layout height is equal to the third aspect ratio.

[0132] For example, the screen resolution of the display device 200 is 1920*1080, and the first resolution of the first projection screen is 512*1088. After the display device 200 displays the projection window of the first terminal device Client1, and then listens to the projection connection instruction of the new second terminal device Client2, it will synchronously create the second projection window View corresponding to the second terminal device 520. After the View is successfully created, the position management module is started to set and manage the display position of SessionClient2, and it is processed and obtained through the position management module. Since the first projection window has been displayed on the left, the orientation mark of the second projection window is set to the right, that is, the mViewPosition of Client2 is marked as 2. The View management data of the management object includes: Session: SessionClie nt2; View: SurfaceView / TextureView; MViewPosition: 2. The second resolution of the second projection screen is obtained to be 1552*1088, then the layout parameters and layout position coordinates of the first projection window and the second projection window are calculated according to the above algorithm, and the obtained layout parameters and layout position coordinates are stored in the data management module, including: Session: SessionClient1; View: SurfaceView / TextureView; MViewPosition: 1; Position: (0, 37); Width: 473; Height: 1006;

[0133] Session: SessionClient2; View: SurfaceView / TextureView; MViewPosition: 2; Position: (485, 37); Width: 1435; Height: 1006.

[0134] After the data management template records the above data information, it transmits the data information to the View management module through the message management module. The View management module then displays the position according to the transmitted data information through the setLayoutParams method. That is:

[0135] FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0); lp1.width=473; lp1.height=1006; lp1.leftMargin=0; lp1.topMargin=37; mPlayerView.set LayoutParams(lp1);

[0136] FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0); lp1.width=1435; lp1.height=1006; lp1.leftMargin=485; lp1.topMargin=37; mPlayerView.setLayoutParams(lp1).

[0137] The display positions of the first projection window and the second projection window corresponding to SurfaceView / TextureView can be set by the View management module respectively, so as to obtain the following: Fig.13 The display effect of the multi-screen projection interface shown. The display device 200 displays the first screen projection window and the second screen projection window at the same height, and at the same time, both windows can be displayed in the multi-screen projection interface to the maximum extent.

[0138] After the display device 200 displays the projection window of the terminal device 500, it can also respond to the corresponding projection termination instruction of the terminal device 500, terminate the projection connection relationship between the display device 200 and the terminal device 500, and remove the projection window of the terminal device 500. In some embodiments, the projection termination instruction can be input from the display device, can be based on the input from the terminal device, and can also be automatically generated due to external reasons such as the network.

[0139] Therefore, in some embodiments, the display device 200 monitors the screen projection connection session of the terminal device 500. After monitoring the termination screen projection instruction of the first screen projection session, in response to the termination screen projection instruction of the first screen projection session, the first screen projection window is removed, and the first guide window is called. Since the first guide window is released when the first screen projection window is displayed, the first guide window needs to be re-called after the display device 200 removes the first screen projection window. For example, the relevant resources of the window object corresponding to the first guide window are re-called. The screen resolution and the second resolution are then detected, and the window display area is calculated according to the screen resolution, the second resolution and the reference resolution. Among them, the window display area includes a fifth display area and a sixth display area, and the height of the fifth display area is equal to that of the sixth display area. After calculating the fifth display area and the sixth display area, the display 260 is controlled to display the first guide window in the fifth display area, and the display 260 is controlled to display the second screen projection window in the sixth display area.

[0140] In some embodiments, when the display device 200 calculates the window display area according to the screen resolution, the second resolution, and the reference resolution, the fifth layout parameter and the sixth layout parameter are set. The fifth layout parameter includes the fifth layout width and the fifth layout height, the sixth layout parameter includes the sixth layout width and the sixth layout height, and the fifth layout height is equal to the sixth layout height; the sum of the fifth layout width and the sixth layout width is equal to the horizontal pixel value of the screen resolution, the fifth layout height and the fifth layout height are less than or equal to the vertical pixel value of the screen resolution, the ratio of the fifth layout width to the fifth layout height is equal to the third aspect ratio, and the ratio of the sixth layout width to the sixth layout height is equal to the second aspect ratio.

[0141] It is understandable that the display device 200 uses the same algorithm as the algorithm used to calculate the first layout parameter, the second layout parameter, the third layout parameter, and the fourth layout parameter when calculating the fifth layout parameter and the sixth layout parameter, which will not be described in detail here.

[0142] Since the terminal device 500 may be placed in different states according to different usage modes, for example, a terminal device 500 such as a smart phone may be in a portrait state and a landscape state during use. The portrait state refers to a state in which the width of the displayed image is less than the height, and the landscape state refers to a state in which the width of the displayed image is greater than the height. In different placement states, the terminal device 500 may present different screen layouts.

[0143] Based on this, in some embodiments, when the display device 200 and the terminal device 500 are placed in different states, the resolution of the projection screen is also different, that is, the projection screen will present different screen ratios in different placement states. For example, when the first terminal device 510 is in a horizontal screen state, the resolution of the first projection screen is 1920*1080, and when the first terminal device 510 is converted to a vertical screen state, the size of the first projection screen is 1080*1920.

[0144] Obviously, when the resolution of the projection screen changes, the display device 200 also needs to recalculate the display area of ​​the projection window according to the resolution of the projection screen after the change. Therefore, in some embodiments, when only the first projection window is displayed in the multi-channel projection interface, the display device 200 also listens for the change event of the first projection session. When the change event of the first resolution is monitored, the changed resolution of the first projection screen is detected. Among them, the changed resolution is the resolution of the first projection screen after the change event is executed. Then calculate the window display area according to the screen resolution, the changed resolution and the reference resolution. The window display area includes the seventh display area and the eighth display area, and the height of the seventh display area is equal to that of the eighth display area. Then control the display 260 to display the first projection window in the seventh display area, and control the display 260 to display the second guide window in the eighth display area.

[0145] Similarly, in some embodiments, when only the second projection window is displayed in the multi-channel display interface, the display device 200 also monitors the change event of the second projection session. When the change event of the second resolution is monitored, the changed resolution of the second projection screen is detected. Among them, the changed resolution is the resolution of the second projection screen after the change event is executed. Then the window display area is calculated according to the screen resolution, the changed resolution and the reference resolution. The window display area includes the ninth display area and the tenth display area, and the height of the ninth display area is equal to that of the tenth display area. Then control the display 260 to display the first guide window in the ninth display area, and control the display 260 to display the second projection window in the tenth display area.

[0146] In some embodiments, when the first projection interface and the second projection interface are displayed simultaneously on multiple display interfaces, the display device 200 monitors the change events of the first projection session and the second projection session. When the change event of the first resolution and / or the second resolution is monitored, the window display area is calculated according to the refresh resolution and the screen resolution after the change event is executed, that is, when the display device 200 monitors the resolution change event in any projection session, the window display area is recalculated to display the projection window according to the newly calculated window display area.

[0147] For example, after the display device 200 establishes a screen projection connection relationship with the first terminal device 510 and the second terminal device 520, the following is displayed: Fig.13 The multi-channel screen projection interface shown in the figure can monitor the first screen projection session through the onVideoChanged() method of UsesionDisplayListener through the screen projection management module, and monitor the change of the first terminal device 510 from the portrait screen state to the landscape screen state, that is, when the first resolution changes, the message processing module notifies the algorithm processing module of the changed first resolution, so that the algorithm processing module recalculates the layout parameters and layout position coordinates of each window through the algorithm provided in the above embodiment. After the calculation is completed, the algorithm processing module notifies the View management module of the data information through the message processing module, and the View management module then re-layouts the first projection window and the second projection window according to the transmitted data information, and displays as shown. Fig.14 The effect shown.

[0148] In order to make full use of the display space of the multi-channel projection interface and reduce large blank areas, in some embodiments, after the display device 200 displays the first projection window, it also calculates the window aspect ratio of the first projection window based on the first layout width and the first layout height, and compares the window aspect ratio with the first aspect ratio. If the window aspect ratio is not equal to the first aspect ratio, the first projection screen in the first projection window is scaled. In other words, when the aspect ratio of the projection window is inconsistent with the aspect ratio of the projection screen, it means that in addition to the display area of ​​the projection screen in the projection window, there are other blank areas. The display device 200 can then scale the projection screen in the projection screen, thereby reducing the blank area in the projection window.

[0149] In order to facilitate the user to interact with the screen projection window, in some embodiments, when the display device 200 displays the multi-channel screen projection interface, it can also display the broadcast control function control in the multi-channel screen projection interface, and the user can perform interactive operations through the broadcast control function control to input control instructions and control the display device 200 to perform interactive responses. For example, since the screen projection content of the display device 200 and the terminal device 500 is diverse, there are mirrored video streams, pushed URL addresses, and different media types such as pictures and music, the broadcast control function controls of different media types are also different. For example, the mirrored video screen projection does not support fast forward and rewind operations nor does it support play and pause operations; the pushed URL address can fast forward, rewind, play, pause, etc.; the music can fast forward, rewind, play, pause, etc.; the picture does not support fast forward, rewind, play, pause, etc.

[0150] In some embodiments, the playback control function control of the display device 200 can be displayed above the projection screen in a floating manner, such as Fig.15 Alternatively, the playback control function control of the display device 200 can also be displayed in the edge area of ​​the multi-channel projection interface, such as Fig.16For example, the display device 200 may display the above-mentioned playback control function control when detecting a focus event of the projection window.

[0151] In order to take into account the display area of ​​the projection window and the display effect of the broadcast control function control, for the scene where the broadcast control function control is displayed in the edge area, in some embodiments, after the display device 200 displays the first projection window in the first display area, the display device 200 responds to the focus event of the first projection window and controls the display 260 to display the broadcast control function control of the first projection window in the edge area. Read the second preset height of the edge area. Among them, the second preset height is greater than the first preset height. Then set the sum of the first layout height and the second preset height equal to the vertical pixel value of the screen resolution to generate the first layout height based on the second preset height. In other words, when the display device 200 displays the broadcast control function control in the edge area, it can also compress the display area of ​​the projection window or the guide window in proportion, thereby increasing the edge area of ​​the multi-channel projection interface to facilitate the display of the broadcast control function control.

[0152] Alternatively, the display device 200 may control the display 260 to display the playback control function control of the first projection window in the edge area in response to the focus event of the first projection window, and detect the maximum display height when the playback control function control is displayed, and then calculate the first layout height of the first projection window according to the maximum display height.

[0153] For example, after the display device 200 establishes a screen projection connection relationship with the first terminal device 510 and the second terminal device 520, the following is displayed: Fig.13In the multi-channel projection interface shown, the layout height of the first projection window and the second projection window is 1006. At this time, the user clicks on the first projection window, and the display device 200 detects that the maximum display height when displaying the broadcast control function control is 864, then it is necessary to reset the layout parameters of the first projection window and the second projection window, and set the layout height of the first projection window and the second projection window from 1006 to 864, and compress the first projection window and the second projection window in equal proportion, and set Client 1: width1, height1; Client2: width2, height2; through the following combined calculation, width1, top, height, width_guide can be calculated; that is, width1 / height1=ratio1; width2 / height2=ratio2; reset height1 and height 2 to 864, and then width1 and width2 can be obtained according to the above ratio. At this time, the data stored in the data management module includes: Session: SessionClient1; View: SurfaceView / TextureView; MViewPosition: 1; Position: (0, 108); Width: 406; Height: 864,

[0154] Session: SessionClient2; View: SurfaceView / TextureView; MViewPosition: 2; Position: (418, 108); Width: 1232; Height: 864.

[0155] The data management module notifies the View management module of the above data information, and sets the display positions of the first projection window and the second projection window corresponding to the SurfaceView / TextureView, respectively, so as to obtain the following: Fig.17 The display effect of the multi-channel screen projection interface is shown.

[0156] Based on the above-mentioned display device 200, some embodiments of the present application also provide a method for displaying a multi-channel screen projection window, which is applied to the display device 200 of the above-mentioned embodiment, wherein the display device 200 includes a display 260, a communicator 220 and a controller 250; the display 260 is configured to display a multi-channel screen projection interface, and the multi-channel screen projection interface includes a first guide window and a second guide window, and the first guide window and the second guide window are used to display a guide page; the first guide window and the second guide window are the same as the root node of the multi-channel screen projection interface; the communicator 220 is configured to establish a communication connection with a terminal device; Fig.10As shown, the method comprises the following procedural steps:

[0157] S100: In response to the screen projection connection instruction sent by the first terminal device, create a first screen projection window, release the first guide window, and retain the display of the second guide window on the multi-way screen projection interface; the first screen projection window is used to display the first screen projection picture of the first terminal device, and the first screen projection window is the same as the root node of the multi-way screen projection interface;

[0158] S200: Acquire a first screen projection session sent by the first terminal device, where the first screen projection session includes a first resolution of a first screen projection picture;

[0159] S300: Detecting the screen resolution and the reference resolution of the guide page;

[0160] S400: Calculating a window display area according to the screen resolution, the first resolution, and the reference resolution, the window display area including a first display area and a second display area, the first display area and the second display area having the same height;

[0161] S500: Control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area.

[0162] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a display method for multiple projection windows. The method can create a first projection window and release the first guide window in response to a projection connection instruction sent by a first terminal device. Among them, the first projection window is used to display the first projection screen of the first terminal device. Then obtain the first projection session sent by the first terminal device, and detect the screen resolution and the reference resolution of the guide page. Calculate the first display area and the second display area according to the screen resolution, the first resolution and the reference resolution. Among them, the height of the first display area is equal to that of the second display area. Then control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area. The method can calculate the display area of ​​each window according to the screen resolution, the projection screen and the resolution of the guide page on the premise of displaying multiple windows at the same height, so that the projection screen can be displayed in the multi-projection interface to the greatest extent, thereby improving the user experience.

[0163] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0164] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0166] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: A display, configured to display a multi-channel screen projection interface, the multi-channel screen projection interface comprising a first guide window and a second guide window, the first guide window and the second guide window are used to display a guide page; the first guide window, the second guide window and the root node of the multi-channel screen projection interface are the same; A communicator configured to establish a communication connection with a terminal device; The controller is configured as: In response to the screen projection connection instruction sent by the first terminal device, a first screen projection window is created, the first guide window is released, and the display of the second guide window on the multi-way screen projection interface is retained; the first screen projection window is used to display the first screen projection screen of the first terminal device, and the first screen projection window is the same as the root node of the multi-way screen projection interface; Obtain a first screen projection session sent by the first terminal device, where the first screen projection session includes a first resolution of a first screen projection picture; Detecting the screen resolution and the base resolution of the guide page; Calculate a window display area according to the screen resolution, the first resolution and the reference resolution, the window display area includes a first display area and a second display area, and the first display area and the second display area have the same height; Control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area.

2. The display device according to claim 1, characterized in that The controller is also configured to: In response to the screen projection connection instruction sent by the second terminal device, create a second screen projection window, and release the second guide window; the second screen projection window is used to display the second screen projection screen of the second terminal device, and the second screen projection window is the same as the root node of the multi-channel screen projection interface; Acquire a second screen projection session sent by the second terminal device, where the second screen projection session includes a second resolution of the second screen projection picture; detecting the screen resolution and the first resolution; Calculate a window display area according to the screen resolution, the first resolution, and the second resolution, wherein the window display area includes a third display area and a fourth display area, and the third display area and the fourth display area have the same height; Control the display to display the first projection window in the third display area, and control the display to display the second projection window in the fourth display area.

3. The display device according to claim 2, characterized in that The controller is also configured to: In response to a screen projection termination instruction of the first screen projection session, removing the first screen projection window and calling the first guide window; detecting the screen resolution and the second resolution; Calculate the window display area according to the screen resolution, the second resolution and the reference resolution, the window display area includes a fifth display area and a sixth display area, and the fifth display area and the sixth display area have the same height; The display is controlled to display a first guide window in the fifth display area, and the display is controlled to display the second projection window in the sixth display area.

4. The display device according to claim 1, characterized in that The controller calculates the window display area according to the screen resolution, the first resolution and the reference resolution, and is configured to: Read the horizontal pixel value and the vertical pixel value of the screen resolution; Calculating a first aspect ratio according to a ratio of horizontal pixel values ​​to vertical pixel values ​​in the first resolution, and calculating a second aspect ratio according to a ratio of horizontal pixel values ​​to vertical pixel values ​​in the reference resolution; Setting first layout parameters and second layout parameters, wherein the first layout parameters include a first layout width and a first layout height, and the second layout parameters include a second layout width and a second layout height; the first layout height is equal to the second layout height, the sum of the first layout width and the second layout width is equal to the horizontal pixel value of the screen resolution, the first layout height and the second layout height are less than or equal to the vertical pixel value of the screen resolution, the ratio of the first layout width to the first layout height is equal to the first aspect ratio, and the ratio of the second layout width to the second layout height is equal to the second aspect ratio; The first display area is generated according to the first layout parameters, and the second display area is generated according to the second layout parameters.

5. The display device according to claim 4, characterized in that The controller controls the display to display the first projection window in the first display area, and is further configured to: Generate the layout position coordinates of the first projection window according to the first layout parameters; Recording the first layout parameter and the layout position coordinates; Draw the first projection window according to the first layout parameters and the layout position coordinates.

6. The display device according to claim 4, characterized in that The controller is also configured to: Reading a first preset height of an edge area, wherein the edge area includes a display area of ​​the multi-channel screen projection interface except the first display window and the second guide window; The sum of the first layout height and the first preset height is set to be equal to the vertical pixel value of the screen resolution, so as to generate the first layout height based on the first preset height.

7. The display device according to claim 6, characterized in that After the controller controls the display to display the first projection window in the first display area, the controller is further configured to: In response to a focus event of the first projection window, controlling the display to display a playback control function control in the edge area; Reading a second preset height of the edge area, wherein the second preset height is greater than the first preset height; The sum of the first layout height and the second preset height is set to be equal to the vertical pixel value of the screen resolution, so as to generate the first layout height based on the second preset height.

8. The display device according to claim 4, characterized in that After the controller controls the display to display the first projection window in the first display area, the controller is further configured to: Calculate the window aspect ratio of the first projection window according to the first layout width and the first layout height; comparing the window aspect ratio with the first aspect ratio; If the window aspect ratio is not equal to the first aspect ratio, scaling is performed on the first projection picture in the first projection window.

9. The display device according to claim 1, characterized in that: The controller is also configured to: Monitoring the change event of the first screen projection session; When a change event of the first resolution is monitored, detecting a changed resolution of the first projection screen, where the changed resolution is a resolution of the first projection screen after the change event is executed; Calculating the window display area according to the screen resolution, the changed resolution and the reference resolution, the window display area including a seventh display area and an eighth display area, the seventh display area and the eighth display area having the same height; The display is controlled to display the first projection window in the seventh display area, and the display is controlled to display the second guide window in the eighth display area.

10. A method for displaying a multi-channel projection window, characterized in that: Applied to a display device, the display device includes a display, a communicator and a controller; the display is configured to display a multi-channel screen projection interface, the multi-channel screen projection interface includes a first guide window and a second guide window, the first guide window and the second guide window are used to display a guide page; the first guide window, the second guide window and the root node of the multi-channel screen projection interface are the same; The communicator is configured to establish a communication connection with a terminal device; the method comprises: In response to the screen projection connection instruction sent by the first terminal device, a first screen projection window is created, the first guide window is released, and the display of the second guide window on the multi-way screen projection interface is retained; the first screen projection window is used to display the first screen projection screen of the first terminal device, and the first screen projection window is the same as the root node of the multi-way screen projection interface; Obtain a first screen projection session sent by the first terminal device, where the first screen projection session includes a first resolution of a first screen projection picture; Detecting the screen resolution and the base resolution of the guide page; Calculate a window display area according to the screen resolution, the first resolution and the reference resolution, the window display area includes a first display area and a second display area, and the first display area and the second display area have the same height; Control the display to display the first projection window in the first display area, and control the display to display the second guide window in the second display area.

Images