Display device

Abstract

The embodiment of the invention provides display equipment. In the embodiment of the invention, the display equipment receives a selection instruction of a multi-window template under the condition that the currently played media asset is a screen projection media asset picture, and supports the screen projection media asset which is played previously to be continuously played based on the screen projection instance by multiplexing the screen projection instance in a single task mode in a task stack. And supporting screen projection media resources indicated by the screen projection instruction sent by the user later based on the screen projection instance. And the multi-window layout in the multi-window playing mode is adjusted based on the screen projection instance, so that mutual shielding among the multiple windows is avoided. On the basis of reuse of the screen projection instance, the problem of playing interruption of previous screen projection media resources caused by screen projection signal conflicts in a multi-window mode is effectively avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display devices, in particular to a display device. BACKGROUND

[0002] The display device can display media content pictures based on media content data transmitted by a physical signal source, or display media content pictures based on media content data indicated by a screen projection signal source. In addition, based on the increasing demand of users for watching, for example, when a user wants to watch multiple media content pictures at the same time, the display device needs to display multiple media content playing windows at the same time.

[0003] The display device can display multiple media content playing windows at the same time, and the multiple media content playing windows can correspond to combinations of a physical signal source + a screen projection signal source, a screen projection signal source + a screen projection signal source, and the like. In the case where both signal sources are screen projection signal sources, the display device will stop the initial media content playing picture due to the conflict of the screen projection signal. SUMMARY

[0004] An embodiment of the present application provides a display device, which includes a display and a controller.

[0005] The controller is configured to:

[0006] in response to a first screen projection instruction sent by an external device, playing an initial screen projection media content picture on the display in full screen;

[0007] in response to a selection instruction for selecting a multi-window template, invoking a screen projection instance from a current task stack; the screen projection instance is an instance corresponding to the initial screen projection media content picture, and the running mode of the screen projection instance is a single task mode;

[0008] setting an initial playing window in a first window position of a first layer through the screen projection instance, and setting a to-be-projected window in a second window position of the first layer through the screen projection instance; the initial playing window is a window playing the initial screen projection media content picture, the to-be-projected window is a playing window newly added to the first layer in a multi-window playing mode, and the initial playing window and the to-be-projected window have no intersection area;

[0009] controlling the display to continue playing the initial screen projection media content picture in the initial playing window, and controlling the display to display a waiting picture in the to-be-projected window.

[0010] In this way, by multiplexing the screen projection instance in the task stack, the screen projection instance can support both online playing of the media content picture and offline playing of the media content picture, thereby avoiding the problem of interruption of online playing of the media content picture due to change of the screen projection instance in the multi-window mode. In addition, the position of the multi-window can be adjusted based on the screen projection instance, thereby effectively avoiding the problem of poor viewing effect due to obstruction between playing windows.

[0011] In some possible implementation manners, the controller, in the case that the initial screen projection media content picture is played in full screen on the display, before executing the operation of invoking the screen projection instance from the current task stack in response to the selection instruction for selecting the multi-window template, is further configured to:

[0012] In response to the start instruction for starting the multi-window template list, the controller controls the display to display the multi-window template list. The multi-window template list includes at least one multi-window playing mode corresponding multi-window template.

[0013] In this way, the user can send the start instruction for starting the multi-window template list to the display device through a pre-set triggering manner, and then select the required multi-window template in the multi-window template list. Based on the setting of the multi-window template list, various viewing needs of the user can be met, and based on the selection of the multi-window template by the user, the related parameters in the subsequent screen projection instance can be dynamically adjusted, so as to control the layout of the multi-window.

[0014] In some possible implementation manners, the controller, after executing the operation of invoking the screen projection instance from the current task stack in response to the selection instruction for selecting the multi-window template, is further configured to:

[0015] create a window container through the screen projection instance;

[0016] set a view type corresponding to the window container through the screen projection instance; the view type of the to-be-screen-projected window is different from the view type of the initial playing window.

[0017] In this way, by creating an independent window container and setting a differentiated rendering type, control conflict between the previously played media content picture and the subsequently played media content picture can be effectively avoided.

[0018] In some possible implementation manners, the controller, in the operation of invoking the screen projection instance from the current task stack in response to the selection instruction for selecting the multi-window template, is specifically configured to:

[0019] based on the multi-window template pointed by the selection instruction, set a multi-window template identifier for the screen projection instance; the multi-window template identifier is used to indicate a pre-set size adjustment parameter;

[0020] arouse the screen projection instance from the current task stack, so that the screen projection instance adjusts the window position of the initial playing window and the to-be-projected window based on the preset size adjustment parameter.

[0021] In this way, based on the selection of the multi-window template by the user, the corresponding multi-window template identifier can be set for the screen projection instance, and then based on the multi-window template identifier, the layout, size and other parameters of the multi-window can be adjusted to ensure the reasonable distribution and efficient cooperation of each playing window in the screen.

[0022] In some possible implementation manners, the controller performs adjusting, by the screen projection instance, the initial playing window to the first window position of the first layer, and determining, by the screen projection instance, the second window position of the to-be-projected window in the first layer, and is specifically configured to:

[0023] The preset size adjustment parameter is obtained by the screen projection instance based on the multi-window template identifier. The preset size adjustment parameter includes a preset coordinate system, a resolution of the display, and a size adjustment ratio.

[0024] The first window size of the initial playing window and the second window size of the to-be-projected window are calculated by the screen projection instance based on the resolution of the display and the size adjustment ratio.

[0025] The first window coordinates of the initial playing window are determined by the screen projection instance based on the first window size and the preset coordinate system, and the second window coordinates of the to-be-projected window are determined by the screen projection instance based on the second window size and the preset coordinate system.

[0026] The initial playing window is set at the first window position by the screen projection instance based on the first window coordinates, and the to-be-projected window is set at the initial to-be-projected window position by the screen projection instance based on the second window coordinates.

[0027] In this way, based on the multi-window template identifier, the preset size adjustment parameter corresponding to the multi-window template identifier can be called, and then different window layouts, sizes and positions can be adjusted. Based on the application of a single screen projection instance, the interaction between applications can be effectively reduced, thereby simplifying the multi-window control process and improving the control efficiency of the multi-window playing process.

[0028] In some possible implementation manners, the controller performs controlling the display to continue playing the initial media content picture in the initial playing window, and after controlling the display to display a waiting picture in the to-be-projected window, is further configured to:

[0029] In response to a second screen projection instruction sent by an external device, a display attribute of screen projection media content data is obtained from the second screen projection instruction; the display attribute at least includes a resolution of the screen projection media content data.

[0030] The third window size of the to-be-cast window is calculated according to a resolution of the cast media data by the cast instance; the third window size is a window size when the to-be-cast window plays a media picture corresponding to the cast media data.

[0031] The third window coordinates of the to-be-cast window and fourth window coordinates of the initial playing window are determined based on the third window size and a preset coordinate system by the cast instance; the fourth window coordinates correspond to a fourth window size when the first initial playing window continues to play the initial media picture.

[0032] The to-be-cast window is set at a third window position of the first layer based on the third window coordinates by the cast instance, and the initial playing window is set at a fourth window position of the first layer based on the fourth window coordinates by the cast instance.

[0033] The display is controlled to play the media picture corresponding to the cast instruction in the to-be-cast window, and the display is controlled to continue to play the initial media picture in the first initial playing window.

[0034] In this way, in the multi-window playing mode, after receiving the cast instruction, the window size required by the new cast media can be dynamically acquired based on the cast application, and the window layout can be adaptively adjusted, so that the multiple playing windows can adapt to diversified cast media and do not block each other in the playing process.

[0035] In a second aspect, the present application provides a display device, comprising a display and a controller.

[0036] The controller is configured to:

[0037] In a case where the display plays the physical signal source media picture in the initial playing window full screen, a multi-window control application is pulled up in response to a selection instruction for selecting a multi-window template.

[0038] A fifth window position of the to-be-cast window in the first layer is determined by the multi-window control application, and the fifth window position is sent to the cast instance. In addition, a sixth window position of the initial playing window in the second layer is determined by the multi-window control application, and the sixth window position is sent to the physical signal source application.

[0039] The to-be-cast window is set at the fifth window position of the first layer by the cast application, and the initial playing window is adjusted to the sixth window position of the second layer by the physical signal source application. The first layer is located above the second layer, and the first layer is a transparent layer. The second initial playing window has no intersection area with the to-be-cast window.

[0040] The display is controlled to continue playing the physical signal source media content in the initial playing window, and the display is controlled to display a waiting picture in the to-be-cast window.

[0041] In this way, through the cooperation of multiple applications, the layout adjustment of the position and size of the playing window between layers can be realized. In particular, based on the setting of the physical signal source application, the position and size of the initial playing window in the second layer can be effectively adjusted to ensure that the initial playing window located in the second layer is not blocked by the first layer in the multi-window mode, and the to-be-cast window can also be independently displayed in the first layer without blocking each other.

[0042] In some possible embodiments, when the signal source type is a physical signal source, the controller determines a fifth window position of the to-be-cast window in the first layer and adjusts the initial playing window to a sixth window position of the second layer through the physical signal source application, and is specifically configured to:

[0043] A preset size adjustment parameter is obtained. The preset size adjustment parameter includes a preset coordinate system, a resolution of the display, and a size adjustment ratio.

[0044] The fifth window size of the to-be-cast window and the sixth window size of the initial playing window are calculated based on the resolution of the display and the size adjustment ratio.

[0045] The fifth window coordinates of the to-be-cast window are determined based on the fifth window size and the preset coordinate system, and the sixth window coordinates of the initial playing window are determined based on the sixth window size and the preset coordinate system.

[0046] The fifth window coordinates are sent to the casting instance, and the sixth window coordinates are sent to the physical signal source application.

[0047] The to-be-cast window is set at the fifth window position of the first layer based on the fifth window coordinates through the casting instance, and the second initial playing window is set at the sixth window position of the second layer based on the sixth window coordinates through the physical signal source application.

[0048] In this way, the multi-window control application can obtain the preset size adjustment parameter set in advance, dynamically calculate the size and coordinates of each window according to the actual resolution of the display, and then distribute the size and coordinates of the window to the physical signal source application and the casting instance, so that the physical signal source application and the casting instance adjust the corresponding windows, respectively, thereby realizing the control of multiple layers and multiple windows, and ensuring that the layout of each window in different layers is reasonable and does not block each other.

[0049] In some possible embodiments, when the display plays the media content in the full-screen playing mode, after the multi-window control application is pulled up in response to the selection instruction for selecting the multi-window template, the controller is further configured to:

[0050] The multi-window control application determines the window position of the control window based on the window positions of the initial playing window and the to-be-cast window. The control window is drawn on the third layer based on the window position of the control window.

[0051] The display is controlled to display the control window.

[0052] The third layer is above the first layer, and the third layer is a transparent layer. The display area of the control window includes a control control used for controlling the initial playing window or the to-be-cast window.

[0053] In this way, the multi-window control application can further determine the position of the control window according to the positional relationship between the initial playing window and the to-be-cast window, and then draw the transparent superimposed control window in the third layer, so as to ensure that the control window does not shield the underlying picture and is convenient for user operation.

[0054] In some possible implementation manners, after the controller executes the multi-window control application in response to the selection instruction for selecting the multi-window template, the controller is further configured to, in a case where the display plays the media picture based on the full-screen playing mode:

[0055] The multi-window control application determines the focus box drawing parameter based on the window positions of the initial playing window and the to-be-cast window.

[0056] The focus box is drawn based on the focus box drawing parameter in response to the selection instruction for selecting the playing window.

[0057] The display is controlled to display the focus box at the playing window pointed by the selection instruction.

[0058] In this way, the multi-window control application can draw the focus box according to the positions of the initial playing window and the to-be-cast window, and control the focus box to be displayed at the corresponding playing window in a case where the user selects the window. Based on the multi-window control application, the focus box can be dynamically drawn to adapt to the change of the window, so as to improve the operation flexibility and interaction accuracy in the multi-window playing mode.

[0059] In a third aspect, an embodiment of the present application provides a multi-playing window control method, which can be applied to the display device in the first aspect, and the method comprises the following steps:

[0060] In a case where the display plays the initial cast media picture in full screen, the cast instance is invoked from the current task stack in response to the selection instruction for selecting the multi-window template. The cast instance is an instance supporting playing of the initial cast media picture, and the running mode of the cast instance is a single-task mode.

[0061] The initial playing window is adjusted to the first window position of the first layer through the screen projection instance, and the second window position of the first layer of the to-be-projected window is determined through the screen projection instance. The to-be-projected window is a playing window newly added to the first layer in the multi-window playing mode. The first initial playing window and the to-be-projected window have no intersection area.

[0062] The display is controlled to continue playing the initial media content picture in the first initial playing window, and the display is controlled to display a waiting picture in the to-be-projected window.

[0063] In a fourth aspect, the embodiments of the present application provide a multi-playing window control method, which can be applied to the display device in the second aspect, and the method comprises the following steps:

[0064] In a case where the display plays the physical signal source media content picture in the full screen based on the initial playing window, the multi-window control application is pulled up in response to a selection instruction for selecting a multi-window template.

[0065] The fifth window position of the to-be-projected window in the first layer is determined through the multi-window control application, and the fifth window position is sent to the screen projection instance. The sixth window position of the initial playing window in the second layer is determined through the multi-window control application, and the sixth window position is sent to the physical signal source application.

[0066] The to-be-projected window is set in the fifth window position of the first layer through the screen projection application, and the initial playing window is adjusted to the sixth window position of the second layer through the physical signal source application. The first layer is located above the second layer, and the first layer is a transparent layer. The second initial playing window and the to-be-projected window have no intersection area.

[0067] The display is controlled to continue playing the physical signal source media content picture in the initial playing window, and the display is controlled to display a waiting picture in the to-be-projected window. BRIEF DESCRIPTION OF DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, other drawings can also be obtained by those skilled in the art without any creative effort on the premise of not paying any creative effort.

[0069] Figure 1 The schematic diagram of the display device operation scene provided by some embodiments of the present application;

[0070] Figure 2 The hardware configuration schematic diagram of the display device provided by some embodiments of the present application;

[0071] Figure 3 The software configuration schematic diagram of the display device provided by some embodiments of the present application;

[0072] Figure 4 A multi-window mode schematic diagram provided for some embodiments of the present application;

[0073] Figure 5 A flowchart of starting a multi-window mode when the current media asset is a screen projection media asset provided for some embodiments of the present application;

[0074] Figure 6 A timing diagram of starting a multi-window mode when the current media asset is a screen projection media asset provided for some embodiments of the present application;

[0075] Figure 7 A schematic diagram of an initial play window continuing to play after entering a multi-window mode provided for some embodiments of the present application;

[0076] Figure 8 A schematic diagram of entering a multi-window mode according to a user-selected multi-window template provided for some embodiments of the present application;

[0077] Figure 9 A timing diagram of waking up a screen projection instance provided for some embodiments of the present application;

[0078] Figure 10 A flowchart of a screen projection instance adjusting a multi-window layout provided for some embodiments of the present application;

[0079] Figure 11 A window container creation flowchart provided for some embodiments of the present application;

[0080] Figure 12 A flowchart of a screen projection instance adjusting a multi-window layout when a screen projection media asset changes provided for some embodiments of the present application;

[0081] Figure 13 A multi-window layout self-adaptive adjustment schematic diagram provided for some embodiments of the present application;

[0082] Figure 14 A multi-window mode schematic diagram when the current play media asset is a physical signal source media asset picture provided for some embodiments of the present application;

[0083] Figure 15 A multi-window mode starting flowchart when the current media asset is a physical signal source media asset picture provided for some embodiments of the present application;

[0084] Figure 16 A multi-application cooperative control multi-window layout schematic diagram provided for some embodiments of the present application;

[0085] Figure 17 A flowchart of multi-application cooperative control multi-window layout provided for some embodiments of the present application;

[0086] Figure 18 Schematic diagram of control window provided for some embodiments of this application;

[0087] Figure 19 Flowcharts are drawn for the control windows provided in some embodiments of this application;

[0088] Figure 20 A flowchart is drawn to illustrate the focus box provided in some embodiments of this application;

[0089] Figure 21 A schematic diagram of the initial position of the focus frame provided for some embodiments of this application;

[0090] Figure 22 This is a schematic diagram of the initial position of the second focus box provided for some embodiments of this application. Detailed Implementation

[0091] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0092] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0093] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0094] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0095] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0096] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software codes that can perform the function related to the element.

[0097] In some embodiments of the present application, the display device 200 refers to a device with the ability of picture display and data processing. For example, the display device 200 includes, but is not limited to, a smart television, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0098] Figure 1 The schematic diagram of the operation scenario between the display device and the control device is provided for some embodiments of the present application. As shown in Figure 1 The user can operate the display device 200 through the touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus, a handle, etc.

[0099] The mobile terminal 300 can be used as a kind of control device to perform the human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a kind of communication device to establish a communication connection with the display device 200 and perform data interaction. In some embodiments, the mobile terminal 300 can install a software application with the display device 200, implement the connection communication through a network communication protocol, and achieve the purpose of one-to-one control operation and data communication. The mobile terminal 300 can also transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to achieve the function of synchronous display.

[0100] As shown in Figure 1 It is also shown in

[0101] The display device 200 can provide the function of receiving a broadcast television and can additionally provide the function of a smart network television with computer support, including but not limited to a network television, a smart television, an Internet protocol television (IPTV), etc.

[0102] Figure 2 The hardware configuration block diagram of the display device 200 is provided for some embodiments of the present application. Figure 1

[0103] In some embodiments, the display device 200 can include at least one of a tuning demodulator 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, a user input interface, etc.

[0104] ​In some embodiments, the detector 230 is configured to collect signals of the external environment or the external interaction. For example, the detector 230 can include a millimeter wave radar, which can be configured to detect whether a user is included in a preset range. The detector 230 can also include a sound collector, which is configured to collect voice instructions input by the user.

[0105] In some embodiments, the display 260 includes a display functional component configured to present a picture, and a driving component configured to drive the image display. The display 260 is configured to receive an image signal output from the controller 250 for display. For example, the display 260 can be configured to display video content, image content, components of a menu control interface, a user control UI interface, and the like.

[0106] In some embodiments, the communication device 220 is a component configured to communicate with the external device or the server 400 according to various communication protocol types. The display device 200 can be provided with multiple communication devices 220 according to different supported communication manners. For example, when the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0107] The communication device 220 can be configured to connect the display device 200 to the external device or the server 400 in a wireless or wired manner. The wired connection can be achieved by connecting the display device 200 to the external device through a data line, an interface, or the like. The wireless connection can be achieved by connecting the display device 200 to the external device through a wireless signal or a wireless network. The display device 200 can be directly connected to the external device, or can be indirectly connected to the external device through a gateway, a router, a connection device, or the like.

[0108] In some embodiments, the controller 250 can include at least one of a central processor, a video processor, an audio processor, a graphics processor, a power supply processor, a first interface to an n-th interface for input / output, and the like. The controller 250 can control the operation of the display device and respond to the user's operation by controlling various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0109] In some embodiments, the controller 250 and the tuner demodulator 210 can be located in different split devices, i.e., the tuner demodulator 210 can also be located in an external device of the main body device where the controller 250 is located, such as an external set-top box, or the like.

[0110] In some embodiments, the user can input user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the graphical user interface (GUI).

[0111] In some embodiments, the audio output device 270 can be a native loudspeaker of the display device 200, or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 can further be provided with an external audio output terminal, and the audio output device can be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0112] In some embodiments, the user input interface 280 can be used to receive instructions from user input. The user input interface 280 can include at least one of a microphone, a touchpad, a sensor, a remote control, and the like. In turn, the display device 200 can execute interactive functions with the user based on the instructions received by the user input interface 280 from the user input.

[0113] In order to perform user interaction, in some embodiments, the display device 200 can run an operating system. The operating system is a computer program used to manage and control hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0114] It should be noted that the operating system can be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0115] The operating system can be divided into different modules or levels according to the functions implemented, for example, as shown in FIG. 1, in some embodiments, the system is divided into four layers, from top to bottom, the application layer (referred to as "application layer" for short), the application framework layer (referred to as "framework layer" for short), the system library layer and the kernel layer. Figure 3

[0116] ​In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0117] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0118] like Figure 3 As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0119] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0120] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0121] In some embodiments, the kernel layer is a functional layer between the hardware and the software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, memory management, etc. For example, as shown in FIG. 2, the kernel layer can be configured with a hardware driver. The driver contained in the kernel layer can be at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a WIFI driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power supply driver, etc. Figure 3

[0122] Based on the hardware configuration and the application program architecture provided, Figure 2 , Figure 3 According to the display device 200 in the embodiments of the present application, the multi-window mode can be implemented. In the multi-window mode, the display 260 can display multiple playing windows. Generally, the newly added playing window is used to play the media data indicated by the user's screen projection instruction. Therefore, in the embodiments of the present application, the window existing before the multi-window mode is started is referred to as an initial playing window, and the window newly added after the multi-window mode is started is referred to as a to-be-projected window. Therefore, in the multi-window mode, the signal sources of the media data corresponding to the multiple windows can be different.

[0123] However, based on the different signal sources corresponding to the playing windows, the layers in which the playing windows are located are also different. Therefore, in the multi-window mode, the multiple windows can easily be blocked. In addition, in the multi-window mode, the size and the position of the window correspond to the multi-window template. In the case where the template clearly indicates the window layout, the size of the window cannot be changed, which results in that the size of the window cannot adapt to the size required by the media picture, and thus the user experience is reduced.

[0124] In addition, in the case where the data sources of the media pictures corresponding to the multiple windows in the multi-window mode are different, the various multi-window templates also have individual problems based on the control architecture corresponding to the playing of the templates.

[0125] As shown in FIG. 2, in the multi-window mode, two playing windows are taken as an example. One of the two playing windows can be an initial playing window corresponding to the projected media, and the other can be a to-be-projected window newly started in the multi-window mode. In this way, after the user starts the multi-window mode, the user can watch the media picture played in the initial playing window and project the new media picture in the to-be-projected window to watch multiple pictures at the same time. Figure 4

[0126] ​​However, in the multi-window mode of the window corresponding to the screen projection media asset and the window to be projected, the screen projection application is prone to signal conflict because it needs to support both the online playing of the media asset picture and the playing of the media asset picture played later, thereby causing the online playing of the media asset picture to be interrupted.

[0127] Based on the above problems, an embodiment of the present application provides a display device 200 to solve the problem that in the multi-window mode of the window corresponding to the screen projection media asset and the window to be projected, the online playing of the media asset picture is prone to interruption due to the conflict of the screen projection signals in sequence.

[0128] The display device 200 includes a display 260, which can display a playing interface. The playing interface can display an initial playing window and a playing window to be projected in the case where the multi-window mode is started.

[0129] As shown in FIG. 1, the controller 250 is configured to: Figure 5

[0130] S100: In response to a first screen projection instruction sent by an external device, playing an initial screen projection media asset picture in full screen on the display.

[0131] In some embodiments, the external device can be a mobile phone, a tablet computer, or other electronic device with a screen projection function. Thus, the user can send a screen projection instruction to the display device 200 by operating such an electronic device, so that the display device 200 acquires screen projection media asset data according to the screen projection instruction and plays the corresponding screen projection media asset picture.

[0132] In the case where the display device 200 receives a screen projection instruction only once and plays the screen projection media asset picture corresponding to the screen projection instruction, the display device 200 can play the screen projection media asset picture in full screen playing mode.

[0133] In some embodiments, the display device 200 provides a multi-window playing mode, so that the user can start the multi-window playing mode through a preset trigger operation.

[0134] S200: In response to a selection instruction for selecting a multi-window template, invoking a screen projection instance from a current task stack.

[0135] It should be noted that the full screen playing of the initial screen projection media asset picture by the display 260 means that the initial playing window currently displayed by the display 260 is playing the screen projection media asset picture indicated by the screen projection instruction received in advance. In this way, the display device 200 needs to create a screen projection instance and load the screen projection instance into the task stack in the case where the screen projection instruction is received for the first time, so as to support the playing of the screen projection media asset picture corresponding to the screen projection instruction based on the screen projection instance.

[0136] ​The selection instruction for selecting the multi-window template refers to an instruction triggered by a preset key or in a preset operation area by the user. After the user triggers the selection instruction, the display device 200 can enter the multi-window playing mode based on the multi-window template selected by the user. The multi-window playing mode refers to that the display device 200 displays multiple windows in the display interface at the same time.

[0137] In some embodiments, the number of windows displayed by the display device 200 after entering the multi-window playing mode is 2, one of which is an initial playing window, and the other is a newly added to-be-screened playing window.

[0138] It should be noted that, in the embodiments of the present application, the newly added playing window of the display device 200 after entering the multi-window playing mode is used to undertake a new screen media picture playing task, rather than other media playing tasks. Therefore, in the process of entering the multi-window playing mode, the display device 200 can call the screen instance that has been loaded into the task stack from the current task stack, so as to support the screen media picture playing in the newly added to-be-screened playing window based on the reuse of the screen instance, thereby avoiding the interruption of the playing of the media picture in sequence due to the conflict of the screen signal caused by the re-creation of the screen instance.

[0139] It should be noted that, in order to enable the screen instance to have the ability to be reused, the running mode corresponding to the screen instance can be set to a single task mode, i.e., a single task mode, in the configuration stage of the screen instance. Further, the screen instance started based on the single task mode can always maintain a unique instance state in the task stack and can be effectively reused.

[0140] For example, in the case where the display device 200 receives a screen instruction again based on the single task mode started screen instance, the screen instance can receive an event callback of “onNewIntent”, and then undertake the screen task corresponding to the new screen instruction based on the processing branch corresponding to the onNewIntent callback event. In this way, the situation that the screen instance needs to be restarted to receive the subsequent screen instruction in the scenario where the screen instance receives multiple screen instructions in sequence is effectively avoided, and the situation that the online screen media playing is interrupted due to the restart of the screen instance is effectively avoided.

[0141] As Figure 6As shown, when the display device 200 receives the selection instruction, it acquires the layer currently located at the top of the interface through a system framework (for example, a framework). If the system framework acquires the layer as an OSD layer (On-Screen Display layer), it can perform an operation of invoking an existing screen mirroring instance from the current task stack and delivering a new screen mirroring task to the instance through an onNewIntent mode. At this time, the system does not need to create a new screen mirroring instance, but only needs to load a new screen mirroring media content screen in the existing instance, so as to realize that, in the multi-window playing mode, while the media content of the initial playing window is continued, the newly added to-be-screen-mirrored playing window can independently load new screen mirroring content, and the two windows do not interfere with each other. By reusing the screen mirroring instance in the single-task mode, the system effectively avoids screen interruption or delay caused by instance reconstruction, and guarantees the continuity and stability of the previous playing task.

[0142] It can be understood that, when the user starts the multi-window playing mode, after initiating a new screen mirroring instruction, the system will automatically deliver a new screen mirroring task to an existing screen mirroring instance in the current task stack through an onNewIntent callback mechanism, and then the screen mirroring instance can play screen mirroring media content data indicated by the new screen mirroring task in the newly added to-be-screen-mirrored window.

[0143] S300: setting, by the screen mirroring instance, the initial playing window at a first window position of a first layer and setting, by the screen mirroring instance, the to-be-screen-mirrored window at a second window position of the first layer.

[0144] It can be understood that the initial playing window is a playing window corresponding to a previous screen mirroring instruction and is used to play a previous media content screen. Since the initial playing window is used to play a screen mirroring media content screen, the layer where the initial playing window is located is the first layer, that is, an OSD layer. In addition, the newly added to-be-screen-mirrored window is also used to play a screen mirroring media content screen, so the layer of the to-be-screen-mirrored window is also the first layer. That is, the screen mirroring instance allocates the first window position and the second window position of the initial playing window and the to-be-screen-mirrored window in the first layer.

[0145] In some embodiments, the confirmation and drawing of the first window position and the second window position are uniformly scheduled and executed by the screen mirroring instance under the reusable mechanism of the screen mirroring instance, so as to effectively reduce the interaction between modules, save the occupation of system resources, and effectively improve the starting efficiency of the multi-window mode.

[0146] S400: controlling the display to continue playing the initial media content screen in the initial playing window and controlling the display to display a waiting screen in the to-be-screen-mirrored window.

[0147] As Figure 7As shown, in some embodiments, after the display device 200 enters the multi-window playback mode, it controls the display 260 to continue playing the initial projection media image in the initial playback window. Since the projection instance is the same instance in the single-task mode, the initial projection media image played in the initial playback window will not be interrupted.

[0148] Furthermore, the display device 200 can control the monitor 260 to display a waiting screen in the screen-casting window. The waiting screen can be a pre-defined animation or prompt, such as "Your new screen-cast content will play in this window," to guide the user to further actions. Alternatively, the waiting screen can be blank to avoid interfering with the user's selection before they have confirmed the new screen-cast content.

[0149] like Figure 8 As shown, users can select a multi-window template to put the display device 200 into a corresponding multi-window playback mode. For example, if the media asset being played first is a screen-casting media asset, the device can enter a dual-window playback mode, playing two screen-casting media asset images through two separate playback windows. A corresponding display list can be set for the multi-window templates to show the user the available multi-window templates. Users can first access the display list of multi-window templates and then select a multi-window template.

[0150] Specifically, before the controller executes the selection instruction for choosing a multi-window template and invokes the projection instance from the current task stack when the monitor is displaying the initial projection media in full screen, it is also configured as follows:

[0151] In response to a launch command used to initiate a multi-window template list, the system controls the display to show the multi-window template list. The multi-window template list includes at least one multi-window template corresponding to a multi-window playback mode.

[0152] In some embodiments, the remote control used with the display device 200 may have a preset trigger button corresponding to the multi-window template list. Then, after the user presses the button, the display device 200 can receive the key value sent by the remote control, and if the key value is parsed as being used to activate the multi-window template list, control the display 260 to display the multi-window template list.

[0153] The multi-window template list can float above the current full-screen playback screen as a window, displaying multiple selectable multi-window templates. That is, the multi-window template list is located on the topmost layer, but the display device 200 will hide the multi-window template list before receiving a start command triggered by user key input to avoid obstructing media playback.

[0154] It should be noted that, based on the design that the multi-window template list can be set in the form of a floating window, the display device 200 does not interrupt the screen mirroring media content currently being played in full screen when displaying the multi-window template list, thereby ensuring that the media content is not interrupted due to the selection and starting process of the multi-window template, and further ensuring the viewing quality of the user.

[0155] In some embodiments, the display device 200 is configured to set a shortcut button for each selectable multi-window template in the layout space of the remote controller. The user can trigger the display device to perform the corresponding mode switching operation by directly pressing the shortcut button corresponding to the target multi-window template, without the need to open the list and then select, thereby further improving the operation efficiency.

[0156] As shown in FIG. 13, the controller 250 is further configured to, after the screen mirroring instance is woken up from the current task stack in the case that the signal source type is a screen mirroring signal source: Figure 9

[0157] S210: Create a window container through the screen mirroring instance.

[0158] S220: Set the view type corresponding to the window container through the screen mirroring instance. The view type of the to-be-screen-mirrored window is different from the view type of the initial playing window.

[0159] In some embodiments, after the screen mirroring instance is woken up, it can perform multiple steps such as creating a window container and adjusting the window position. When receiving a previous screen mirroring instruction, the screen mirroring instance can create a window container and load a SurfaceView based on the window container to realize the playing of the initial screen mirroring media content, thereby effectively ensuring the playing effect of the initial screen mirroring media content.

[0160] For the to-be-screen-mirrored window, the screen mirroring instance can create a window container, such as a Fragment, for loading the to-be-screen-mirrored window, and set a TextView view type for the to-be-screen-mirrored window based on the Fragment. In this way, based on the manner of setting different view types for different playing windows, the mutual influence between the windows can be avoided, and the over-occupation of system resources in the case of using multiple same SurfaceViews can be avoided, and the problem of window blocking caused by multiple SurfaceViews can be effectively avoided. Further, based on the manner of setting different view types for different windows, the playing stability and continuity of the initial playing window and the to-be-screen-mirrored window in the multi-window playing mode can be effectively improved.

[0161] ​It can be understood that the view type of the to-be-screened window corresponding to the Fragment can also be adaptively adjusted according to requirements to adapt to more media playing scenarios, as long as the to-be-screened window and the initial playing window do not have control and playing conflicts caused by the view type.

[0162] As shown in Figure 10 The screen projection instance needs to adjust the first window position of the initial playing window and the second window position of the to-be-screened window in the process in which the display device 200 enters the multi-window playing mode. Different multi-window playing templates have different layouts and sizes between multi-windows, and thus a mapping relationship between part of the multi-window templates and corresponding window layout parameters can be preset, so that the screen projection instance directly calls the mapping relationship to adjust the first window position and the second window position in the process in which the multi-window mode is started.

[0163] The controller 250 executes the screen projection instance from the current task stack in response to a selection instruction for selecting a multi-window template, and is specifically configured to:

[0164] S201: Set a multi-window template identifier for the screen projection instance based on the multi-window template pointed by the selection instruction.

[0165] S202: Call the screen projection instance with the multi-window template identifier from the current task stack, so that the screen projection instance adjusts the window positions of the initial playing window and the to-be-screened window based on the preset size adjustment parameter.

[0166] In some embodiments, the display device 200 can determine the multi-window template identifier according to the multi-window template selected by the user after receiving the selection instruction of the user, and the multi-window template identifier can be previously mapped with parameters such as window position, size, and proportion. That is, the multi-window template identifier can be used to indicate the preset size adjustment parameter.

[0167] Further, the screen projection instance can call the corresponding preset size adjustment parameter based on the multi-window template identifier in the starting process of the multi-window playing mode, so as to configure the first window position and the second window position of the display device 200 when initially entering the multi-window playing mode.

[0168] After the user selects the multi-window template in which the initial playing window and the to-be-screened window are both used to play the screen projection media, the system framework can set the identifier corresponding to the multi-window template to the screen projection instance, and then the screen projection instance can obtain the corresponding preset size adjustment parameter according to the multi-window template identifier when adjusting the first window position and the second window position.

[0169] For example, the system framework can set the multi-window template identifier to 1, allowing the projection instance to read the preset size adjustment parameters corresponding to the multi-window template identifier being 1. The first and second window positions are then adjusted according to these preset size adjustment parameters. Specifically, when the preset size adjustment parameters indicate a 1:1 size ratio between the first and second windows, the projection instance can adjust the sizes of the first and second windows in a 1:1 ratio.

[0170] Understandably, the display device 200 can listen to the user's selection command for a multi-window template in the multi-window template list through the UI module. The UI module then passes the listened-upon event to the system framework, which determines whether the playback screen currently at the top layer corresponds to a projection media asset or another type of media asset. If it is confirmed to be a projection media asset, the system framework sets the corresponding multi-window template identifier for the projection instance and then invokes the projection instance in the task stack.

[0171] In this way, by setting corresponding multi-window template identifiers for the projection instance based on the user's selection of multi-window templates, a clear layout configuration basis can be provided for the projection instance, thereby improving the efficiency of the projection instance in adjusting the position of the first window and the second window, which is conducive to improving the response speed and user experience of the multi-window playback mode.

[0172] The display device 200 can adjust the layout of the initial playback window and the window to be projected through the projection instance. When both the initial playback window and the window to be projected are located in the OSD layer, the projection instance determines the first window position of the initial playback window and the second window position of the window to be projected in the OSD layer respectively, so as to ensure that the initial playback window and the window to be projected do not obstruct each other.

[0173] like Figure 11 As shown, controller 250 adjusts the initial playback window to the first window position of the first layer through the screen casting instance, and determines the second window position of the window to be cast to in the first layer through the screen casting instance, specifically configured as follows:

[0174] S301: Obtain preset size adjustment parameters based on the multi-window template identifier through the screen projection instance.

[0175] S302: Calculate the first window size of the initial playback window and the second window size of the window to be projected based on the screen projection instance, according to the monitor's resolution and size adjustment ratio.

[0176] S303: Determine the first window coordinates of the first playback window based on the first window size and the preset coordinate system through the screen casting instance, and determine the second window coordinates of the window to be cast based on the second window size and the preset coordinate system.

[0177] S304: setting the initial playing window at the first window position based on the first window coordinates and setting the to-be-projected window at the initial to-be-projected window position based on the second window coordinates through the projection instance.

[0178] In some embodiments, the preset size adjustment parameters include a preset coordinate system, a resolution of the display 260, and a size adjustment ratio. The preset size adjustment parameters can have a mapping relationship with the multi-window template identifier, so that the projection instance can directly call the preset size adjustment parameters according to the multi-window template identifier.

[0179] In addition, after obtaining the preset size adjustment parameters, the projection instance can set the windows according to the parameters. Taking the size adjustment ratio as 1:1, the resolution of the display 260 as 1920x1080, the upper-left corner coordinates of the display 260 as (0, 0), and the lower-right corner coordinates as (1920, 1080) as an example, the first window position and the second window position cannot exceed the coordinate range and need to be ensured not to block each other, that is, there is no boundary.

[0180] In the process of confirming the window positions, the projection instance needs to ensure that the sizes of the first window position and the second window position are the same based on the size adjustment ratio 1:1, and also needs to ensure that the first window position and the second window position are located on the same horizontal line and are distributed side by side in order to ensure the viewing effect.

[0181] In addition, the projection instance can adjust the specific size of the window based on the resolution and other parameters. The specific size adjustment process is not limited in the embodiments of the present application, and can be ensured that the sizes of the multi-windows do not affect the normal viewing experience of the user.

[0182] In this way, the projection instance can first calculate the sizes of the initial playing window and the to-be-projected window, and then determine the window coordinates of the initial playing window and the to-be-projected window in combination with the preset coordinate system, that is, confirm the first window position and the second window position of the initial playing window and the to-be-projected window. Moreover, the projection instance can draw the corresponding initial playing window and to-be-projected window based on the determined first window position and second window position, so that the display 260 displays the initial playing window and the to-be-projected window at the same time.

[0183] In the embodiments of the present application, the mapping relationship between the multi-window template identifier and the preset size adjustment parameters can enable the projection instance to flexibly call the preset size adjustment parameters according to the multi-window template selected by the user, and thus improve the window adjustment efficiency of the projection instance.

[0184] It can be understood that the initial playing window has a corresponding window container, such as a surface view, in the OSD layer. Therefore, a corresponding window container also needs to be created for the to-be-projected window in the case of adding the to-be-projected window.

[0185] As shown in FIG. 13, after the display device 200 enters the multi-window playing mode, the display 260 has played different media content through two different windows at the same time. At this time, the user can still input a new screen projection instruction to make the display device 200 receive a screen projection media picture corresponding to the new screen projection instruction based on the newly added screen projection window. Figure 12 Figure 13 As shown in FIG. 13, after the display device 200 enters the multi-window playing mode, the display 260 has played different media content through two different windows at the same time. At this time, the user can still input a new screen projection instruction to make the display device 200 receive a screen projection media picture corresponding to the new screen projection instruction based on the newly added screen projection window.

[0186] The controller 250, which performs control on the display to continue playing the initial media picture in the initial playing window and control on the display to display the waiting picture in the screen projection window, is further configured to:

[0187] S410: In response to the second screen projection instruction sent by the external device, obtaining a display attribute of the screen projection media data from the second screen projection instruction.

[0188] S420: Calculating, by the screen projection instance, a third window size of the screen projection window according to the resolution of the screen projection media data. The third window size is the window size of the screen projection window when playing the media picture corresponding to the screen projection media data.

[0189] S430: Determining, by the screen projection instance, a third window coordinate of the screen projection window and a fourth window coordinate of the initial playing window based on the third window size and the preset coordinate system. The fourth window size corresponding to the fourth window coordinate is the window size of the first initial playing window when continuing to play the initial media picture.

[0190] S440: Setting, by the screen projection instance, the screen projection window at a third window position of the first layer based on the third window coordinate, and setting, by the screen projection instance, the initial playing window at a fourth window position of the first layer based on the fourth window coordinate.

[0191] S450: Controlling the display to play the media picture corresponding to the screen projection instruction in the screen projection window, and controlling the display to continue playing the initial media picture in the first initial playing window.

[0192] It can be understood that, in the case that the display device 200 receives the second screen projection instruction, the display device 200 can obtain the display attribute of the screen projection media data from the second screen projection instruction, such as the resolution of the screen projection media data. In this way, the display device 200 can adjust the size and position of the initial playing window and the screen projection window in combination with the resolution of the screen projection media data through the screen projection instance.

[0193] In some embodiments, the external device corresponding to the second screen projection instruction can be the external device that sends the first screen projection instruction.

[0194] In other embodiments, the external device corresponding to the second screen projection instruction is different from the external device corresponding to the first screen projection instruction.​

[0195] It should be noted that since the initial screen mirroring media picture is relatively stable and continues to be played in the initial playing window, the display device 200 needs to adapt to the size change and position change of the to-be-screened window in priority when receiving a new screen mirroring instruction.

[0196] For example, the display device 200 can obtain the to-be-screened window adjustment parameter, such as the resolution of the screen mirroring media data, from the screen mirroring instruction through the screen mirroring instance. Further, the screen mirroring instance can adjust the size and position of the current to-be-screened window according to the resolution of the screen mirroring media data, so that the to-be-screened window is completely adapted to the screen mirroring media data corresponding to the new screen mirroring instruction.

[0197] It can be understood that the size change of the to-be-screened window will affect the original multi-window layout ratio, and therefore the screen mirroring instance also needs to further adjust the size and position of the initial playing window based on the adjusted size and position of the to-be-screened window. This is to avoid the problem of occlusion between multi-windows due to the size and position change of the to-be-screened window.

[0198] Such an adjustment manner can effectively adapt to high-frequency screen mirroring instructions in the multi-window playing mode, so that the to-be-screened window dynamically adapts to the screen mirroring task with changing size and resolution requirements, and at the same time, based on the dynamic adjustment of the screen mirroring instance, it is ensured that there is no occlusion between multi-windows, thereby ensuring the use experience of multi-window playing.

[0199] In some embodiments, the screen mirroring instance can also adaptively adjust the position of the control window in the case of multi-window layout change. The control window is a window containing multiple control controls. The user can control the multi-window playing based on the multiple control controls displayed by the control window, such as volume adjustment, brightness adjustment, fast forward and fast backward, pause, and multiple types of control.

[0200] Among them, the control window and the multi-window also follow the design principle of not occluding each other, and therefore the screen mirroring instance can redraw the control window according to the finally determined multi-window layout in the case of multi-window layout change, so as to ensure that the control window can effectively display the control controls and will not occlude the window playing media data.

[0201] In some embodiments, the user needs to select a playing window before controlling the playing window in the multi-window playing mode, and therefore the screen mirroring instance can also draw a focus box to display the focus box at the playing window when the user selects any playing window, so as to indicate the currently controlled window, and further facilitate the user to adjust the playing parameters such as sound and brightness of the window.

[0202] In this way, in a case where the display device 200 displays a previously played media content picture as a screen mirroring media content picture, the display device 200, in a process of entering the multi-window mode, based on multiplexing of the screen mirroring instance, on one hand, ensures that the previously played screen mirroring media content picture can be continued to play, and on the other hand, based on execution of the multi-window adjustment by the single screen mirroring application, reduces occupation of system resources and reduces the number of interactions between applications, thereby facilitating improvement of multi-window control efficiency.

[0203] As shown in FIG. 1, in a process in which a user uses the display device 200, the user can externally connect an external device to the display device 200, so as to realize interaction between the display device 200 and the external device. For example, the display device 200 can be connected with a game console through an HDMI (High-Definition Multimedia Interface), so that the display device 200 can play a game picture transmitted by the game console in a full-screen playing mode. Meanwhile, the user can start a multi-window playing mode of the display device 200, so as to play a screen mirroring media content, such as a game strategy, through a newly added window, thereby forming a use scenario of “playing while watching”. Figure 14

[0204] It can be understood that the HDMI is a kind of physical signal source, and therefore, media content data transmitted through the HDMI is displayed as a media content picture in a Video layer of the display device 200, so as to be distinguished from a screen mirroring media content picture displayed in an OSD layer, thereby avoiding signal conflict. However, in this case, it is more likely that multiple windows in multiple layers are blocked.

[0205] Embodiments of the present application provide a display device 200 to solve the above problems. The display device 200 includes a display 260 and a controller 250. The display 260 can display a media content picture formed by media content data transmitted by a physical signal source, and can also display a screen mirroring media content picture indicated by a screen mirroring instruction.

[0206] As shown in FIG. 1, the controller 250 is configured to: Figure 15

[0207] S10: In a case where the display displays a physical signal source media content picture in a full-screen playing mode based on an initial playing window, in response to a selection instruction for selecting a multi-window template, the multi-window control application is pulled up.

[0208] In some embodiments, in a case where the display device 200 plays a physical signal source media content picture in a full-screen mode, the display device 200 receives a selection instruction of the user, and a system framework of the display device 200 can acquire a signal type corresponding to a media content picture currently located in a top layer interface. In a case where the system framework acquires that the signal type corresponding to the media content picture is a physical signal source, a corresponding multi-window playing mode processing branch can be executed. For example, the display device 200 can pull up a multi-window control application. ​​

[0209] As Figure 16 shown, the multi-window control application can include a drawing module for drawing the control window and the focus frame, a split bar position listening module, and a window layout module. The drawing module is used to draw the control window and the focus frame, which will be described in subsequent embodiments. The split bar position listening module is used to listen to the split bar position change triggered by the screen projection event. The window layout module is used to calculate the window size and coordinate of the initial play window and the to-be-projected window.

[0210] In the case that the display device 200 enters the multi-window play mode for the first time, the multi-window control application can draw multiple windows to adapt to the multi-window play mode.

[0211] S20: The multi-window control application determines the fifth window position of the to-be-projected window in the first layer and sends the fifth window position to the screen projection application. The multi-window control application also determines the sixth window position of the initial play window in the second layer and sends the sixth window position to the physical signal source application.

[0212] In some embodiments, the screen projection application and the physical signal source application are also started in the startup stage of the display device 200 and run in the background. Therefore, they can both receive the window position information sent by the multi-window control application and perform the subsequent window position setting operation.

[0213] In some embodiments, the to-be-projected window is used to display the screen projection media content indicated by the screen projection task, i.e., the to-be-projected window is set in the OSD layer. The initial play window is used to continue playing the physical signal source media content, i.e., the initial play window is set in the Video layer.

[0214] Therefore, the embodiments of the present application additionally create a physical signal source application based on the multi-window control application to control the size and position of the play window in the Video layer based on the physical signal source application. After determining the size and coordinate of the initial play window in the multi-window play mode, the multi-window control application sends the size and coordinate of the initial play window to the physical signal source application, so that the physical signal source application controls the position and size of the initial play window in the Video layer according to the received size and coordinate, i.e., controls the initial play window to be displayed in the sixth window position of the second layer.

[0215] In this way, the physical signal source application can directly control the change of the play window in the Video layer without going through the system framework, which can improve the adjustment efficiency of the initial play window and further improve the startup efficiency of the multi-window play mode.

[0216] And the multi-window control application can also send the size and coordinates of the to-be-screened window to the screen application, and the screen application can create a screen instance to create and layout the to-be-screened window in the OSD layer through the screen instance, that is, control the to-be-screened window to be displayed in the fifth window position of the first layer.

[0217] In this way, based on the way that the multi-window control application uniformly calculates and determines the fifth window position and the sixth window position, the window position determination process is integrated into the multi-window control application, which is conducive to the unified layout of the multi-window position. In combination with the cooperation between multiple applications, the multi-window control located in multiple layers can be effectively realized, and the window position relationship can be effectively coordinated.

[0218] S30: The to-be-screened window is set in the fifth window position of the first layer by the screen application, and the initial play window is adjusted to the sixth window position of the second layer by the physical signal source application.

[0219] In some embodiments, the first layer is an OSD layer, and the second layer is a Video layer, that is, the first layer is located above the second layer. In order to ensure that the first layer does not block the second layer, the first layer can be set as a transparent layer. In this way, the initial play window located in the second layer can still be viewed through the first layer.

[0220] It can be understood that based on the arrangement of the multi-window control application, there is no intersection area between the initial play window and the to-be-screened window, that is, the initial play window and the to-be-screened window do not block each other, thereby effectively ensuring the viewing effect.

[0221] S40: The display device continues to play the physical signal source media picture in the initial play window, and displays a waiting picture in the to-be-screened window.

[0222] In some embodiments, after the display device 200 enters the multi-window play mode, the display 260 can simultaneously display the to-be-screened window in the first layer and the initial play window in the second layer, continue to play the physical signal source media picture in the initial play window, and display a waiting picture in the to-be-screened window.

[0223] In the multi-window play mode, a transparent first layer is superimposed on the second layer, so it does not interfere with the play of the media in the original second layer, thereby ensuring the normal play of the physical signal source media picture.

[0224] In addition, the display device 200 has not received the screen projection instruction in the case of just entering the multi-window playing mode, and thus the to-be-projected window can display a preset prompt picture or text to prompt the user, such as “please send a screen projection request” or “waiting for device connection”, so as to enable the user to know the current state and perform subsequent screen projection operation. The specific display manner of the waiting picture can also refer to the foregoing embodiments, which will not be described herein again.

[0225] As shown in FIG. 8, after the multi-window control application is pulled up, the size and position of the initial playing window and the to-be-projected window can be adjusted according to preset size adjustment parameters. The controller 250 is configured to: Figure 17

[0226] S21: Obtain preset size adjustment parameters. The preset size adjustment parameters include a preset coordinate system, a resolution of the display, and a size adjustment ratio.

[0227] S22: Calculate a fifth window size of the to-be-projected window and a sixth window size of the initial playing window based on the resolution of the display and the size adjustment ratio.

[0228] S23: Determine a fifth window coordinate of the to-be-projected window based on the fifth window size and the preset coordinate system, and determine a sixth window coordinate of the initial playing window based on the sixth window size and the preset coordinate system.

[0229] S24: Send the fifth window coordinate to the screen projection instance, and send the sixth window coordinate to the physical signal source application.

[0230] S25: Set the to-be-projected window at a fifth window position of the first layer based on the fifth window coordinate by the screen projection instance, and set the initial playing window at a sixth window position of the second layer based on the sixth window coordinate by the physical signal source application.

[0231] It can be understood that the calculation manner of the size and coordinate of the initial playing window and the to-be-projected window can refer to the calculation manner of the size and coordinate of the initial playing window and the to-be-projected window by the screen projection application in the foregoing embodiments, which will not be described herein again.

[0232] ​In the case that the multi-window playing mode corresponds to the combination scenario of the physical signal source media picture + the screen projection media picture, the calculation of the size and coordinates of the initial playing window and the to-be-projected window is uniformly executed by the multi-window control application and then distributed to the physical signal source application and the screen projection instance, and the unified execution based on the calculation can effectively guarantee the effectiveness of the calculation and the accuracy of the coordinate allocation, and avoid window overlapping or layout deviation caused by independent calculation. Through the distribution of the size and coordinates, the physical signal source application and the screen projection instance can accurately render the window position in the respective layers, and realize the coordinated and unified picture layout. The centralized calculation and distributed execution architecture can effectively improve the efficiency and reliability of multi-window management, and ensure the coherence of the user visual experience and the consistency of the operation logic, so that the system response remains accurate and stable in a complex scenario.

[0233] After the display device 200 enters the multi-window playing mode, the physical signal source media picture can be played through the initial playing window, and the screen projection media picture indicated by the screen projection instruction issued by the user can be played through the to-be-projected window.

[0234] However, the user can issue a new screen projection instruction to replace the screen projection media picture according to the demand. When the to-be-projected window receives different screen projection media pictures indicated by different screen projection instructions, the size of the to-be-projected window needs to be adaptively changed, and the position of the to-be-projected window also needs to be adaptively changed.

[0235] Therefore, the split bar position listening module in the multi-window control application can listen to the window change event, and then notify the window layout module to adjust the window size and position according to the demand, so as to guarantee the complete presentation of the media picture and the coordination of the visual layout.

[0236] In some embodiments, when the user initiates the screen projection instruction, the screen projection instance in the display device 200 can receive the screen projection instruction, analyze the resolution and other parameters of the screen projection media picture indicated by the screen projection instruction, and report these parameters to the split bar position listening module.

[0237] After receiving the resolution of the screen projection media picture, the split bar position listening module can further report these parameters to the window layout module and trigger the window re-layout process of the window layout module. Then, the window layout module can re-calculate the target size and coordinate position of the to-be-projected window according to the resolution and other parameters, and distribute the calculation result to the physical signal source application and the screen projection instance.

[0238] In some embodiments, the split-screen bar monitoring module, when obtaining the screen projection media content picture parameters reported by the screen projection instance, can compare the current screen projection media content picture parameters. If the parameters are the same, the screen projection media content picture parameters do not need to be reported to the window layout module, and the window layout module does not need to be triggered to redraw the window. If the parameters are different, the screen projection media content picture parameters can be reported to the window layout module, and the window layout module can be triggered to draw the window.

[0239] It should be noted that the split-screen bar will not be directly displayed in the display interface, but the split-screen bar will occupy a certain display area in the layout to realize the separate display of the initial playing window and the to-be-screen-projected window. Therefore, the window layout module needs to take the placeholder area of the split-screen bar into the overall layout calculation in actual operation to ensure that the corresponding separation space is left when the window size and position are adjusted, so as to avoid picture overlapping or display overflow.

[0240] It can be understood that the position of the split-screen bar can also be adaptively adjusted according to the window layout to maintain the coordination of the split-screen ratio.

[0241] As shown in FIG. 13, the display interface of the display 260 also includes a control window. The control window can display control controls for controlling the initial playing window and the to-be-screen-projected window, such as a volume adjustment button, a brightness adjustment button, and a progress adjustment button. The display principle of the control window is not to block the initial playing window and the to-be-screen-projected window. Therefore, the position of the control window needs to be dynamically adjusted according to the layout of the initial playing window and the to-be-screen-projected window. Figure 18 As shown in FIG. 13, the controller 250, when the display plays the media content picture based on the full-screen playing mode, executes the selection instruction for selecting the multi-window template, and after the multi-window control application is pulled up, is further configured to:

[0242] Figure 19 S11: Determine the window position of the control window based on the window positions of the initial playing window and the to-be-screen-projected window through the multi-window control application.

[0243] S12: Control the display to display the control window.

[0244] S12: Control the display to display the control window.

[0245] The third layer is located above the first layer, and the third layer is a transparent layer.

[0246] In some embodiments, the control window is displayed in the OSD layer in the form of a window. The display level of the window is higher than that of the initial playing window and the to-be-screen-projected playing window, so as not to interfere with the normal display of the initial playing window and the to-be-screen-projected playing window.

[0247] ​The split-screen bar position monitoring module can report the screen mirroring media content picture parameters to the drawing module when it is monitored that the position of the to-be-screened window needs to be changed, so that the drawing module recalculates the window layout of the control window according to the updated parameters.

[0248] After the drawing module in the multi-window control application receives the screen mirroring media content picture parameters and the trigger signal reported by the split-screen bar position monitoring module, it further executes the process of drawing the control window, so that the control window adjusts the position in real time with the layout change of the initial playing window and the to-be-screened window.

[0249] After the drawing module draws the control window, the display device 200 can create a transparent layer in the OSD layer in the form of window and superimpose the control window to ensure that it is located above the initial playing window and the to-be-screened window and does not block the picture content.

[0250] It can be understood that after the control window is displayed, the user can independently adjust the playing parameters such as the volume, brightness, and playing progress of the initial playing window and the to-be-screened window through the controls in the control window, realizing independent control and interactive response of each playing picture in the multi-window state.

[0251] In some embodiments, the transparency of the layer can change with the ambient brightness to provide better visual experience for the user, so as to facilitate the user to quickly find the required controls.

[0252] It should be noted that since there are multiple playing windows for independently playing media data in the multi-window playing mode, the user can adjust the volume, progress, brightness, and other parameters of each playing window when watching. For example, when the user watches the media picture transmitted by the game console through the initial playing window, and watches the game guide through the to-be-screened window, the user can adjust the playing progress of the game guide to adapt to the game picture of the initial playing window.

[0253] It can be understood that the user can first select the window whose playing parameters are to be changed, and then adjust the playing parameters of the window through the trigger control button. Therefore, from the perspective of visual interaction, the display device 200 needs to draw a focus box for the window to indicate that the window is selected by the user through the focus box.

[0254] That is, the display device 200 can draw a focus box through the drawing module in the multi-window playing mode, and display the focus box at the specified playing window according to the operation of the user selecting the playing window.

[0255] As shown in Figure 20 The controller 250, after the display plays the media picture based on the full-screen playing mode, executes the multi-window control application in response to the selection instruction for selecting the multi-window template, and is further configured to:

[0256] S13: determining a focus frame drawing parameter based on the window position of the initial playing window and the to-be-screened window by the multi-window control application.

[0257] S14: drawing the focus frame based on the focus frame drawing parameter in response to a selection instruction for selecting the playing window.

[0258] S15: controlling the display to display the focus frame at the playing window pointed by the selection instruction.

[0259] In some embodiments, the drawing module in the multi-window control application can calculate the size and position of the focus frame according to the window position of the initial playing window and the to-be-screened window. Wherein, in the case that the size and position of the initial playing window and the to-be-screened window change, the drawing module can pre-calculate the focus frame drawing parameter adapted to the initial playing window and the to-be-screened window, and then draw and display the corresponding focus frame according to the focus frame drawing parameter in the case that the user selects the playing window, so as to ensure that the focus frame is aligned with the boundary of the playing window and does not block the picture content. The focus frame can be presented in a high-light frame or dynamic light effect to improve visual recognition.

[0260] In some embodiments, the display device 200 can not display the focus frame in the display interface in the case that it initially enters the multi-window playing mode.

[0261] As shown in FIG. 6, in some embodiments, the display device 200 can display the focus frame at the initial playing window by default in the case that it initially enters the multi-window playing mode, so as to prompt the user to the window object currently operated. Figure 21 As shown in FIG. 7, in some embodiments, the display device 200 can also display the focus frame at the to-be-screened window by default, so as to prompt the user to the window object currently operated.

[0262] Figure 22 As shown in FIG. 7, in some embodiments, the display device 200 can also display the focus frame at the to-be-screened window by default, so as to prompt the user to the window object currently operated.

[0263] In some embodiments, the user can select the playing window through the direction keys of the remote controller, at which time the focus frame is switched to the corresponding playing window synchronously with the movement of the direction keys, so as to ensure that the user accurately locates the operation target. After the user selects the playing window and adjusts the playing parameters, the focus frame can return to the default position or be hidden.

[0264] In this way, based on the cooperation between the drawing module and the split-screen bar position listening module, the focus frame and the control window can be adaptively changed based on the change of the initial playing window and the to-be-screened window, so as to ensure the coordination of the interface layout and improve the intuitiveness and interaction efficiency of the user in the multi-window operation.

[0265] Embodiments of the present application provide a multi-playing window control method, comprising:

[0266] ​In a case that the display plays the initial projection media picture in full screen, the projection instance is invoked from the current task stack in response to a selection instruction for selecting the multi-window template. The projection instance is an instance supporting the initial projection media picture playing, and the running mode of the projection instance is a single task mode.

[0267] The initial playing window is adjusted to the first window position of the first layer by the projection instance, and the second window position of the first layer of the to-be-projected window is determined by the projection instance. The to-be-projected window is a playing window newly added to the first layer in the multi-window playing mode. The first initial playing window and the to-be-projected window have no intersection area.

[0268] The display is controlled to continue playing the initial media picture in the first initial playing window, and the display is controlled to display a waiting picture in the to-be-projected window.

[0269] Embodiments of the present application also provide a multi-playing window control method, the method comprising:

[0270] In a case that the display plays the physical signal source media picture in full screen based on the initial playing window, the multi-window control application is pulled up in response to a selection instruction for selecting the multi-window template.

[0271] The fifth window position of the to-be-projected window in the first layer is determined by the multi-window control application, and the fifth window position is sent to the projection instance. In addition, the sixth window position of the initial playing window in the second layer is determined by the multi-window control application, and the sixth window position is sent to the physical signal source application.

[0272] The to-be-projected window is set in the fifth window position of the first layer by the projection application, and the initial playing window is adjusted to the sixth window position of the second layer by the physical signal source application. The first layer is above the second layer, and the first layer is a transparent layer. The second initial playing window and the to-be-projected window have no intersection area.

[0273] The display is controlled to continue playing the physical signal source media picture in the initial playing window, and the display is controlled to display a waiting picture in the to-be-projected window.

[0274] Similar parts among embodiments provided by the present application can be referred to each other, and the specific embodiments provided above are only several examples under the general concept of the present application, and do not limit the protection scope of the present application. Any other embodiments extended according to the present application scheme without creative labor are within the protection scope of the present application.

Claims

1. A display device, characterized by comprising: The application relates to a display device and a control method thereof. The display device comprises a display and a controller configured to: play an initial projection media content picture on the display in full screen in response to a first projection instruction sent by an external device; invoke a projection instance from a current task stack in response to a selection instruction for selecting a multi-window template; the projection instance is an instance corresponding to the initial projection media content picture, and a running mode of the projection instance is a single-task mode; set an initial playing window in a first window position of a first layer through the projection instance, and set a to-be-projected window in a second window position of the first layer through the projection instance; the initial playing window is a window playing the initial projection media content picture; the to-be-projected window is a playing window newly added in the first layer in a multi-window playing mode; the initial playing window and the to-be-projected window have no intersection area; control the display to continue playing the initial projection media content picture in the initial playing window, and control the display to display a waiting picture in the to-be-projected window. The controller, when playing the initial projection media content picture on the display in full screen, is further configured to, before invoking the projection instance from the current task stack in response to the selection instruction for selecting the multi-window template:

2. The display device of claim 1, wherein, control the display to display a multi-window template list in response to a starting instruction for starting the multi-window template list; the multi-window template list comprises at least one multi-window template corresponding to a multi-window playing mode. The controller, after invoking the projection instance from the current task stack in response to the selection instruction for selecting the multi-window template, is further configured to:

3. The display device of claim 1, wherein, create a window container through the projection instance; set a view type corresponding to the window container through the projection instance; the view type of the to-be-projected window is different from that of the initial playing window. The controller, when invoking the projection instance from the current task stack in response to the selection instruction for selecting the multi-window template, is specifically configured to:

4. The display device of claim 1, wherein, set a multi-window template identifier for the projection instance based on the multi-window template pointed by the selection instruction; the multi-window template identifier is used for indicating preset size adjustment parameters; invoke the projection instance from the current task stack, so that the projection instance adjusts window positions of the initial playing window and the to-be-projected window based on the preset size adjustment parameters. The controller, when setting the initial playing window in the first window position of the first layer through the projection instance and setting the to-be-projected window in the second window position of the first layer through the projection instance, is specifically configured to:

5. The display device of claim 4, wherein, obtain preset size adjustment parameters based on the multi-window template identifier through the projection instance; the preset size adjustment parameters comprise a preset coordinate system, a resolution of the display and a size adjustment ratio; calculate a first window size of the initial playing window and a second window size of the to-be-projected window based on the resolution of the display and the size adjustment ratio through the projection instance. ​ determining, by the screen projection instance, first window coordinates of the first play window based on the first window size and the preset coordinate system, and determining second window coordinates of the window to be projected based on the second window size and the preset coordinate system; setting, by the screen projection instance, the initial play window at the first window position based on the first window coordinates, and setting the window to be projected at an initial window to be projected position based on the second window coordinates.

6. The display device of claim 1, wherein, The controller is further configured to control the display to continue playing the initial media content picture in the initial play window, and control the display to display a waiting picture in the window to be projected after the initial play window is further configured to: in response to a second screen projection instruction sent by an external device, obtaining a display attribute of screen projection media content data from the second screen projection instruction; the display attribute at least includes a resolution of the screen projection media content data; calculating, by the screen projection instance, a third window size of the window to be projected according to the resolution of the screen projection media content data; the third window size is the window size when the window to be projected plays the media content picture corresponding to the screen projection media content data; determining, by the screen projection instance, third window coordinates of the window to be projected and fourth window coordinates of the initial play window based on the third window size and a preset coordinate system; the fourth window coordinates correspond to a fourth window size when the first initial play window continues to play the initial media content picture; setting, by the screen projection instance, the window to be projected at a third window position of the first layer based on the third window coordinates, and setting, by the screen projection instance, the initial play window at a fourth window position of the first layer based on the fourth window coordinates; controlling the display to play the media content picture corresponding to the screen projection instruction in the window to be projected, and controlling the display to continue playing the initial media content picture in the first initial play window.

7. A display device, characterized by comprising: a display; a controller configured to: in a case where the display plays the physical signal source media content picture in the initial play window full screen, in response to a selection instruction for selecting a multi-window template, pull up a multi-window control application; determining, by the multi-window control application, a fifth window position of the window to be projected in a first layer, and sending the fifth window position to a screen projection application; and determining, by the multi-window control application, a sixth window position of the initial play window in a second layer, and sending the sixth window position to a physical signal source application; the screen projection application and the physical signal source application are applications running in the background of the display device after the display device is started; setting, by the screen projection application, the window to be projected at the fifth window position of the first layer, and adjusting, by the physical signal source application, the initial play window to the sixth window position of the second layer; wherein the first layer is above the second layer, the first layer is a transparent layer; the initial play window and the window to be projected have no intersection region; The controller controls the display to continue playing the physical signal source media content in the initial playing window, and controls the display to display a waiting screen in the to-be-cast window.

8. The display device of claim 7, wherein, The controller determines, by the multi-window control application, a fifth window position of the to-be-cast window in a first layer, and sends the fifth window position to the casting application; and determines, by the multi-window control application, a sixth window position of the initial playing window in a second layer, and sends the sixth window position to the physical signal source application, and the controller is specifically configured to: obtain preset size adjustment parameters; the preset size adjustment parameters include a preset coordinate system, a resolution of the display, and a size adjustment ratio; calculate a fifth window size of the to-be-cast window and a sixth window size of the initial playing window based on the resolution of the display and the size adjustment ratio; determine a fifth window coordinate of the to-be-cast window based on the fifth window size and the preset coordinate system, and determine a sixth window coordinate of the initial playing window based on the sixth window size and the preset coordinate system; send the fifth window coordinate to the casting instance, and send the sixth window coordinate to the physical signal source application; the controller executes through the casting application to set the to-be-cast window in the fifth window position of the first layer, and through the physical signal source application to adjust the initial playing window to the sixth window position of the second layer, and the controller is specifically configured to: through the casting application, based on the fifth window coordinate, set the to-be-cast window in the fifth window position of the first layer, and through the physical signal source application, based on the sixth window coordinate, set the initial playing window in the sixth window position of the second layer.

9. The display device of any of claims 7-8, wherein, The controller, in a case where the display plays media content based on a full-screen playing mode, after executing the multi-window control application in response to a selection instruction for selecting a multi-window template, is further configured to: determine, by the multi-window control application, a window position of a control window based on the window positions of the initial playing window and the to-be-cast window; and draw the control window on a third layer based on the window position of the control window. control the display to display the control window. The third layer is above the first layer, and the third layer is a transparent layer; and the display area of the control window includes a control control for controlling the initial playing window or the to-be-cast window.

10. The display device of any of claims 7-8, wherein, The controller, in a case where the display plays media content based on a full-screen playing mode, after executing the multi-window control application in response to a selection instruction for selecting a multi-window template, is further configured to: determine, by the multi-window control application, focus box drawing parameters based on the window positions of the initial playing window and the to-be-cast window; draw a focus box based on the focus box drawing parameters in response to a selection instruction for selecting a playing window; control the display to display the focus box at the playing window pointed by the selection instruction.

Images