Display device and multi-media asset display method

Abstract

The invention provides a display device and a multi-media asset display method. When the display device displays the first media resource provided by the first data source, the multi-screen display function can be started to display the second media resource provided by the second data source at the same time. When the multi-screen display function is started, the first data source page of the first data source on the application layer can be quitted in a mode that the video layer is separated from the application layer, the first picture of the first data source on the video layer continues to be displayed, and normal playing of the first media resource is guaranteed. A unified user interface is displayed to a user by displaying a multi-screen interface on an application layer. Moreover, the data source page corresponding to each data source can be converted into a component to be embedded into a corresponding area in the multi-screen interface, so that a user can directly control the corresponding data source based on the component in the multi-screen interface.

Description

Technical Field

[0001] This application relates to the technical field of display devices, and in particular, to a display device and a method for displaying multimedia resources. Background Art

[0002] A display device refers to a terminal device that can output a specific display screen, which can be a smart TV, a mobile terminal, a smart advertising screen, a projector, or other terminal devices. Taking a smart TV as an example, a smart TV is a TV product based on Internet application technology, with an open operating system and a chip, an open application platform, and can realize two-way human-computer interaction functions, integrating various functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] A display device can support different data sources. For example, live signals, High-Definition Multimedia Interface (HDMI) signals, network applications, screen mirroring signals, etc. To improve the convenience of users using different data sources, the display device 200 is configured to support simultaneous display of multimedia resources provided by multiple data sources, that is, to support the multi-screen display function. Based on the multi-screen display function, users can simultaneously view multimedia resources provided by multiple data sources without switching the data source to switch the display of different multimedia resources.

[0004] The pages provided by different data sources may have different styles, sizes, etc. Therefore, when simultaneously displaying multimedia resources provided by different data sources, different data source pages will be displayed, resulting in a lack of unity in the overall interface. Even, there may be problems such as mutual occupation of the display area between the data source pages, which requires manual coordination by the user. And only when the data source obtains the focus, the user can control the data source that obtains the focus, and the operation of controlling the data source is relatively cumbersome. Summary of the Invention

[0005] This application provides a display device and a method for displaying multimedia resources, which can simultaneously display multimedia resources provided by different data sources in a unified multi-screen interface, and can provide components for directly managing and controlling each multimedia resource by the multi-screen interface.

[0006] In a first aspect, this application provides a display device, including: a display, configured to: display a first multimedia resource provided by a first data source, where a first picture of the first multimedia resource is displayed in a video layer, and a first data source page for controlling the first data source is displayed in an application layer, and the application layer is located above the video layer;

[0007] The controller is configured to: in response to a first operation of the user, control the display to display a multi-screen interface at the application layer and display the first media asset within the multi-screen interface, where the multi-screen interface includes a main screen area and a secondary screen area, both the main screen area and the secondary screen area are in a transparent state, display the first picture in the area corresponding to the main screen area in the video layer, and convert the first data source page into a first component and embed it into the main screen area; in response to a second operation of the user instructing to display a second media asset provided by a second data source, control the display to display the second media asset within the multi-screen interface, where display the second picture corresponding to the second media asset in the area corresponding to the secondary screen area in the video layer, and convert the second data source page for controlling the second data source into a second component and embed it into the secondary screen area.

[0008] The above technical solution has the following beneficial effects or advantages:

[0009] When the display device displays the first media asset provided by the first data source, it can start the multi-screen display function to simultaneously display the second media asset provided by the second data source. When starting the multi-screen display function, it can separate the video layer from the application layer, exit the first data source page of the first data source at the application layer, and continue to display the first picture of the first data source in the video layer to ensure the normal playback of the first media asset. By displaying a multi-screen interface at the application layer, a unified user interface is presented to the user. Moreover, the data source pages corresponding to each data source can be converted into components and embedded into the corresponding areas in the multi-screen interface, so that the user can directly control the corresponding data source based on the components in the multi-screen interface.

[0010] In some embodiments of the present application, the controller's execution of controlling the display to display a multi-screen interface at the application layer and display the first media asset within the multi-screen interface in response to a first operation of the user is specifically configured as: in response to a first instruction of the user, control the display to display a first menu above the first data source page, where in the case that the first data source supports multi-screen display, the first menu includes a first setting item for starting the multi-screen display function; in the case that the first menu includes the first setting item, in response to an instruction of the user selecting the first setting item, control the display to display the multi-screen interface at the application layer and display the first media asset within the multi-screen interface.

[0011] The above technical solution has the following beneficial effects or advantages:

[0012] The display device can configure a first setting item for starting the multi-screen display function in the first menu. In this way, when displaying the first media asset, the user can quickly enter the multi-screen interface based on the first menu. Moreover, the display device dynamically displays the first setting item in the first menu according to the actual situation of the first data source supporting multi-screen display, which can ensure the effectiveness of the user's operation based on the first setting item.

[0013] In some embodiments of the present application, the display device further includes a memory configured to store application data, and the application data includes status data indicating whether an application provided by the network supports multi-screen display; the controller is configured to, in response to a first instruction of the user, control the display to display a first menu on the upper layer of the first data source page, and is specifically configured to: in response to the first instruction, obtain the type of the first data source; if the type of the first data source is a specified type, control the display to display the first menu on the upper layer of the first data source page, and the first menu includes the first setting item; if the type of the first data source is a network application, control the display to display the first menu on the upper layer of the first data source page according to the status data corresponding to the first data source in the application data, wherein, if the status data corresponding to the first data source is supporting multi-screen display, the first menu includes the first setting item; if the status data corresponding to the first data source is not supporting multi-screen display, the first menu does not include the first setting item; if the type of the first data source is not the specified type and the network application, control the display to display the first menu on the upper layer of the first data source page, and the first menu does not include the first setting item.

[0014] The above technical solution has the following beneficial effects or advantages:

[0015] When the display device displays the first menu, it can accurately identify whether the first setting item needs to be displayed in the first menu based on the type of the currently displayed first data source, thereby ensuring the effectiveness of the user's operation based on the first setting item.

[0016] In some embodiments of the present application, the controller is configured to, in response to a first operation of the user, control the display to display a multi-screen interface at the application layer and display the first media asset within the multi-screen interface, and is specifically configured to: in response to the first operation, control the display to exit the first data source page and display the multi-screen interface at the application layer; start the first data source page and convert the first data source page into the first component; according to the size of the main screen area, reduce the first picture, and according to the position of the main screen area, obtain the area corresponding to the main screen area in the video layer; display the reduced first picture in the area corresponding to the main screen area in the video layer, and embed the first component into the main screen area.

[0017] The above technical solution has the following beneficial effects or advantages:

[0018] After the display device starts the multi-screen display function, the currently displayed first data source can be used as the main data source. When exiting the first data source, only the first data source page is exited, and the first picture is not exited, that is, the display of the first media asset is not affected. When displaying the multi-screen interface, by adjusting the size and position of the first picture, the first picture is correspondingly displayed with the main screen area, so that the complete first picture can be displayed through the main screen area. Moreover, converting the first data source page into the first component and embedding it into the main screen area allows the user to directly control the first data source based on the first component on the multi-screen interface.

[0019] In some embodiments of the present application, the controller is configured to, in response to a second operation of the user instructing to display a second media asset provided by a second data source, control the display to display the second media asset within the multi-screen interface, and is specifically configured to: in response to the second operation, start the second data source page and convert the second data source page into the second component; according to the size of the secondary screen area, start the second picture, and according to the position of the secondary screen area, obtain the area corresponding to the secondary screen area in the video layer; display the second picture in the area corresponding to the secondary screen area in the video layer, and embed the second component into the secondary screen area.

[0020] The above technical solution has the following beneficial effects or advantages:

[0021] When the display device multi-screen displays the first media asset and the second media asset, after starting the second data source, the second picture can be correspondingly displayed with the secondary screen area by adjusting the size and position of the second picture, so that the complete second picture can be displayed through the secondary screen area. Moreover, converting the second data source page into the second component and embedding it into the secondary screen area allows the user to directly control the second data source based on the second component on the multi-screen interface.

[0022] In some embodiments of the present application, after the controller controls the display to display the second media asset provided by the second data source in response to a second operation of a user instructing to display the second media asset, and is further configured to: in response to a second instruction to exit the second data source, exit the second data source, where the second screen is exited, and the second component is destroyed from the secondary screen area.

[0023] The above technical solution has the following beneficial effects or advantages:

[0024] The display device can respond to the user's second instruction to exit the second data source, and free up the secondary screen area again for displaying other data sources in the freed-up secondary screen area. Moreover, the process of exiting the second data source will not affect the normal display and use of the first data source.

[0025] In some embodiments of the present application, after the controller controls the display to display the second media asset provided by the second data source in response to a second operation of a user instructing to display the second media asset, and is further configured to: in response to a third instruction to exit the first data source, exit the multi-screen interface, where the first component is destroyed from the main screen area, and the second data source is exited; control the display to adjust the first screen to the full-screen state, and start the first data source page, and display the first data source page in the application layer in the full-screen state.

[0026] The above technical solution has the following beneficial effects or advantages:

[0027] The display device can respond to the user's third instruction to directly exit the multi-screen display, and after exiting the multi-screen display, readjust the first data source (i.e., the data source displayed before entering the multi-screen display) to the full-screen state to ensure the display effect of the first data source.

[0028] In some embodiments of the present application, after the controller responds to a user instruction to display a second operation of a second media asset provided by a second data source and controls the display to display the second media asset within the multi-screen interface, it is further configured to: in response to a fourth instruction to exit a first target data source, exit the first target data source, where the screen corresponding to the first target data source and the component corresponding to the first target data source are destroyed from the multi-screen interface, and the first target data source is any one of the first data source and the second data source; in response to a fifth instruction to exit a second target data source, if no data source other than the second target data source is currently displayed on the multi-screen interface, exit the multi-screen interface, where the component corresponding to the second target data source is destroyed from the multi-screen interface, and the second target data source is any one of the first data source and the second data source; control the display to adjust the screen corresponding to the second target data source to a full-screen state, and start the data source page corresponding to the second target data source and display the data source page corresponding to the second target data source in a full-screen state at the application layer.

[0029] The above technical solution has the following beneficial effects or advantages:

[0030] When a display device displays multiple data sources in a multi-screen manner, it can respond to a fourth instruction from a user to exit any data source (such as a first target data source), exit the first target data source, and continue to display the remaining data sources in a multi-screen display state. The display device can continue to respond to a fifth instruction from a user to exit any data source (such as a second target data source). When it is detected that there is no other data source except the second target data source currently, it directly exits the multi-screen display. After exiting the multi-screen display, it adjusts the second target data source to a full-screen state to ensure the display effect of the second target data source.

[0031] In some embodiments of the present application, after the controller responds to a user instruction to display a second operation of a second media asset provided by a second data source and controls the display to display the second media asset within the multi-screen interface, it is further configured to: in response to an instruction input by the user based on a specified key, when a focus is obtained in any one of the main screen area and the secondary screen area, switch to the data source corresponding to the target area that obtains the focus; in response to an instruction input by the user based on the specified key, when a focus is obtained in any one of the first data source and the second data source, switch to the area corresponding to the target data source in the multi-screen interface that obtains the focus.

[0032] The above technical solution has the following beneficial effects or advantages:

[0033] When a display device displays multiple data sources on multiple screens, it can switch the focus between the multi-screen interface and the data sources in response to an instruction input by the user based on a specified key, so as to directly control each data source based on the multi-screen interface, or further control the data source when the data source obtains the focus.

[0034] In a second aspect, the present application also provides a method for displaying multimedia resources, which is applied to a display device. The display device displays first multimedia resources provided by a first data source. Among them, a first picture of the first multimedia resources is displayed in the video layer, and a first data source page for controlling the first data source is displayed in the application layer, and the application layer is located above the video layer;

[0035] The method includes: in response to a first operation of the user, displaying a multi-screen interface in the application layer, and displaying the first multimedia resources in the multi-screen interface, where the multi-screen interface includes a main screen area and a secondary screen area, both the main screen area and the secondary screen area are in a transparent state, a region corresponding to the main screen area in the video layer displays the first picture, and converting the first data source page into a first component and embedding it into the main screen area; in response to a second operation of the user instructing to display second multimedia resources provided by a second data source, displaying the second multimedia resources in the multi-screen interface, where a region corresponding to the secondary screen area in the video layer displays a second picture corresponding to the second multimedia resources, and converting a second data source page for controlling the second data source into a second component and embedding it into the secondary screen area.

[0036] The above technical solutions have the following beneficial effects or advantages:

[0037] When the display device displays the first multimedia resources provided by the first data source, it can start the multi-screen display function to simultaneously display the second multimedia resources provided by the second data source. When starting the multi-screen display function, it can exit the first data source page of the first data source in the application layer in the form of separation between the video layer and the application layer, and continue to display the first picture of the first data source in the video layer to ensure the normal playback of the first multimedia resources. By displaying a multi-screen interface in the application layer, a unified user interface is presented to the user. Moreover, the data source pages corresponding to each data source can be converted into components and embedded into the corresponding areas in the multi-screen interface, so that the user can directly control the corresponding data sources based on the components in the multi-screen interface. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0039] Figure 1Schematic diagram of the operation scenario between the display device 200 and the control device 100 in the embodiments of the present application;

[0040] Figure 2 Hardware configuration block diagram of the display device 200 in the embodiments of the present application;

[0041] Figure 3 Operating system configuration diagram of the display device 200 in the embodiments of the present application;

[0042] Figure 4 Schematic diagram of the media assets of the display data source in the embodiments of the present application;

[0043] Figures 5A - 5D Schematic diagram of the multi-media asset display mode in the embodiments of the present application;

[0044] Figure 6 Flowchart of the display device 200 for multi-screen display of multiple media assets provided in the embodiments of the present application;

[0045] Figure 7 Flowchart of the display device 200 for starting the multi-screen display function provided in the embodiments of the present application;

[0046] Figure 8 Timing diagram of the display device 200 for displaying the first menu provided in the embodiments of the present application;

[0047] Figure 9 Flowchart of the display device 200 for displaying the first media asset in the main screen area provided in the embodiments of the present application;

[0048] Figure 10 Timing diagram of the display device 200 for displaying multiple media assets provided in the embodiments of the present application;

[0049] Figure 11 Flowchart of the display device 200 for displaying the second media asset in the secondary screen area provided in the embodiments of the present application;

[0050] Figures 12A - 12C Another schematic diagram of the multi-media asset display mode in the embodiments of the present application;

[0051] Figure 13 Timing diagram of the display device 200 for exiting the data source provided in the embodiments of the present application;

[0052] Figure 14 Flowchart of the display device 200 for exiting the main data source provided in the embodiments of the present application

[0053] Figure 15 Flowchart of the display device 200 for exiting the data source provided in the embodiments of the present application. Detailed implementation manners

[0054] Embodiments will be described in detail below, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following examples do not represent all embodiments consistent with the present application. They are merely examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0055] It should be noted that the brief description of terms in this application is only for facilitating the understanding of the following described embodiments, rather than intending to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0056] In this application, terms such as "first", "second", "third", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms used can be interchanged under appropriate circumstances.

[0057] The terms "comprising" and "having" and any variations thereof are intended to cover but not exclusively include. For example, a product or device comprising a series of components does not necessarily have to be limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0058] In the embodiments of this application, a display device generally refers to a device with the ability to display a screen and process data. For example, display devices include but are not limited to smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0059] Figure 1 It is a schematic diagram of the operation scenario between a display device and a control device provided for some embodiments of this application. As Figure 1 shown, the user can operate the display device 200 through touch operations, voice, a mobile terminal 300, and a control device 100. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0060] As Figure 1 also shown, the display device 200 also communicates with the server 400 through various communication methods. The display device 200 is allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0061] The display device 200 can provide a broadcast receiving TV function, and can also additionally provide a smart network TV function with computer support functions, including but not limited to, network TV, smart TV, Internet Protocol TV (IPTV), etc.

[0062] Figure 2 Provided for some embodiments of the present application Figure 1 is a block diagram of the hardware configuration of the display device 200 shown therein.

[0063] In some embodiments, the display device 200 may include at least one of a tuner demodulator 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a user input interface 280, a memory, and a power supply.

[0064] In some embodiments, the communication device 220 is a component for communicating with external devices or a server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication methods. The communication device 220 may communicatively connect the display device 200 with external devices or the server 400 in a wireless or wired connection manner.

[0065] In some embodiments, the detector 230 is used to collect signals from the external environment or for external interactions. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures. Or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0066] In some embodiments, the device interface 240 is used to access external devices.

[0067] In some embodiments, the controller 250 is used to control the overall operation of the display device 200. The controller 250 may include at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), and a power processor, and first to n interfaces for input / output. The controller 250 controls the operation of the display device 200 and responds to user operations through various software control programs stored in the memory.

[0068] In some embodiments, the controller 250 and the tuner demodulator 210 may be located in different split devices, that is, the tuner demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0069] In some embodiments, the display 260 is used to receive an image signal output from the controller 250 for display. The display 260 may include a display function component for presenting a picture and a driving component for driving the image display.

[0070] In some embodiments, the user may input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface 280 may receive the user command through the GUI.

[0071] In some embodiments, the audio output device 270 may be the built-in speaker of the display device 200 or an external audio output device connected to the display device 200.

[0072] In some embodiments, the user input interface 280 can be used to receive instructions from the user input.

[0073] To perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface. For example, the operating system can directly control the display device to provide a user interface, or provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0074] The operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system developed specifically for the display device 200.

[0075] As Figure 3 shown, Figure 3 is a schematic diagram of the software configuration of the display device provided by some embodiments of the present application. In some embodiments, the system of the display device 200 can be divided into three layers, from top to bottom are the application layer, the middleware layer, and the hardware layer.

[0076] The application layer mainly includes applications on the TV and an application framework. Among them, the applications are mainly applications developed based on the browser, such as HTML5 APPs, and native applications (Native APPs).

[0077] The application framework is a complete program model with all the basic functions required by standard application software, such as file access, data exchange, etc., and the usage interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0078] Native applications (Native APPs) can support online or offline, message push, or local resource access.

[0079] The middleware layer includes various middleware such as TV protocols, multimedia protocols, and system components. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system on the network or different applications, and can achieve the purpose of resource sharing and function sharing.

[0080] The hardware layer mainly includes the HAL interface, hardware, and drivers. Among them, the HAL interface is the unified interface for all TV chips to dock, and the specific logic is implemented by each chip. The drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor drivers (fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0081] It should be noted that the above examples are only simple divisions of the functions of the operating system, and do not limit the specific form of the operating system of the display device 200 in the embodiments of the present application. According to factors such as the functions of the display device and the type of the operating system, the number of levels and the specific level types included in the operating system can be in other forms.

[0082] The display device 200 can support different data sources, such as live signals, HDMI signals, network applications, screen mirroring signals, etc. Correspondingly, the display device 200 can support playing media resources provided by different data sources. For example, if the display device supports a live signal, the display device 200 can support displaying the live program provided by the live signal. If the display device 200 supports an HDMI signal, the display device 200 can support displaying videos, audios, pictures, documents, etc. provided by the HDMI device. If the display device 200 supports a network application, the display device 200 can support displaying web pages, videos, audios, pictures, documents, etc. provided by the network application. If the display device 200 supports a screen mirroring signal, the display device 200 can support displaying videos, audios, pictures, documents, etc. provided by the screen mirroring device.

[0083] To improve the convenience for users to use media resources provided by different data sources, the display device 200 is configured to support simultaneous display of multiple media resources, that is, to support the multi-screen display function. For example, the display device 200 is simultaneously displaying a live program and a shopping web page of a network application. Based on the multi-screen display function, users can watch media resources provided by multiple data sources simultaneously without having to switch data sources to switch and display different media resources.

[0084] In some embodiments, in combination with Figure 4Schematic diagram of media assets showing data sources. After any data source is started, the display device 200 displays the picture of the media assets provided by the data source on the video layer and displays the data source page of the data source on the application layer. The data source page is used to control the corresponding data source and may include corresponding controls. Among them, the application layer is displayed above the video layer. In some embodiments, the area of the data source page for displaying the picture of the media assets is in a transparent state, and through this area, the picture of the media assets located on the video layer can be displayed. In other embodiments, the entire data source page is in a transparent state, and the effect of full-screen display of the picture of the media assets can be achieved. The user can customize and dynamically adjust the area where the data source page is in a transparent state.

[0085] The data source pages of different data sources may have different styles, sizes, etc. Therefore, when displaying the media assets provided by different data sources simultaneously, different data source pages will be displayed, resulting in a lack of unity in the overall interface. Even, problems such as mutual encroachment of the display areas may occur between the data source pages, and users need to manually coordinate. Moreover, only when the data source obtains the focus can the user control the data source that obtains the focus, and the operation of controlling the data source is relatively cumbersome.

[0086] To solve the above problems, the display device 200 provided in the embodiments of the present application can provide a multi-screen interface for unified management of each data source by a multi-screen display application, and provide components that can directly control each data source to the user based on this multi-screen interface.

[0087] Taking the example of the first data source before starting the multi-screen display function in the embodiments of the present application, the first data source can be any one of a live signal, an HDMI signal, a network application, device mirroring, etc. Under the first data source, the display device 200 displays the media assets provided by the first data source, such as the first media asset. Combining Figures 5A - 5D with the schematic diagram of the multi-media asset display method shown, as Figure 5A shown, the display device 200 displays the first picture of the first media asset on the video layer and displays the first data source page corresponding to the first data source on the application layer. In the embodiments of the present application, taking the example of displaying the first data source in a full-screen state before starting the multi-screen display function, both the first picture and the first data source page are in a full-screen state.

[0088] Figure 6 The flowchart of the multi-screen display of multiple media assets by the display device 200 provided in the embodiments of the present application is as follows:

[0089] Step S601: In response to the user's first operation, display the multi-screen interface on the application layer and display the first media asset within the multi-screen interface.

[0090] The display device 200 is configured with a multi-screen display application, and supports the multi-screen display function through the multi-screen display application.

[0091] The first operation is used to start the multi-screen display application to activate the multi-screen display function.

[0092] In some embodiments, the display device 200 may configure an entry for starting the multi-screen display application in the first menu. In one example, the first menu may be a quick setting menu.

[0093] Figure 7 The flowchart for the display device 200 provided by the embodiments of the present application to activate the multi-screen display function is as follows:

[0094] Step S701: In response to a first instruction from the user, display the first menu on top of the first data source page.

[0095] In some embodiments, the first instruction may be a key instruction, and the key instruction includes a first key value. Herein, the key instruction refers to an instruction input based on a key on the control device 100, and the key instruction includes a key value corresponding to the corresponding key.

[0096] In some embodiments, the first key value corresponds to a first key, and the first key is used to start the first menu when the display data source provides media assets. The first key may be any one of the basic keys on the control device 100. The basic keys may include numeric keys, color keys, direction keys, a menu key, a setting key, etc. Alternatively, the first key may also be a specially set key on the control device 100, and the specially set key is dedicated to starting the first menu.

[0097] In the embodiments of the present application, taking the first key as the menu key on the control device 100 as an example. When the user wants to activate the multi-screen display function, the user may output a first instruction based on the menu key, and the first instruction includes the menu key value. Since the menu key is a basic key and has its own corresponding basic function, such as starting the system settings menu. This basic function conflicts with the function of starting the first menu. To solve this conflict, the display device 200, in response to the first instruction, identifies the current operating environment. If it is identified that the current is under the data source, the task of starting the first menu is executed; if it is identified that the current is not under the data source, the task of starting the system settings menu is executed.

[0098] In some embodiments, the display device 200 displays the first menu on the On-Screen Display (OSD) layer, and the OSD layer is displayed above the application layer. Therefore, the first menu is overlaid and displayed on the first data source page.

[0099] Reference may be made to Figure 5B, in response to the first instruction, after recognizing that it is currently under the first data source, the display device 200 displays the first menu 501 at the OSD layer. The first menu 501 is a quick settings menu.

[0100] The first menu includes setting items for setting corresponding functions. Among them, the setting items may include a first setting item for starting the multi-screen display function.

[0101] Not all data sources support multi-screen display. To ensure the effectiveness of the user's operation based on the first setting item, that is, to ensure that the user can normally start the multi-screen display function after selecting the first setting item. The display device 200 is configured to dynamically display the first setting item in the first menu. Among them, when the first data source supports multi-screen display, the first setting item is displayed in the first menu; when the first data source does not support multi-screen display, the first setting item is not displayed in the first menu.

[0102] In some embodiments, the process for the display device 200 to display the first menu may be: in response to the first instruction, obtain the type of the first data source, and display the first data source according to the type of the first data source.

[0103] Combined with Figure 3 the operating system shown, the application layer is configured with a quick setting module and a Multiview UI module. The middleware layer is configured with an App Service.

[0104] Figure 8 This is the timing diagram for the display device 200 provided by the embodiment of the present application to display the first menu.

[0105] Combined with Figure 8 , the display device 200 transmits the first instruction to the quick setting module, and the quick setting module, in response to the first instruction, sends a status request to the multi-screen interface module. The status request is used to request the display status of the multi-screen display, that is, whether the first data source supports multi-screen display. The multi-screen display module calculates whether the first data source supports multi-screen display in response to the status request. For example, obtain the type of the first data source and calculate whether the first data source supports multi-screen display according to the type.

[0106] Among them, if the type of the first data source is a specified type, the first menu including the first setting item is displayed on the upper layer of the first data source page. In one example, the specified type may include a live signal, an HDMI signal, and a screen mirroring signal.

[0107] Combined with Figure 8, if the multi-screen interface module obtains that the first data source is of a specified type, it returns a response message to the quick settings module, and the response message includes support for multi-screen display. The quick settings module displays a first setting item in the first menu according to the response message, which can be referred to Figure 5B , display the first setting item 502 in the first menu 501.

[0108] In some embodiments, the display device 200 stores application data in the memory, and the application data includes status data of whether the applications provided by the network support multi-screen display. In one example, the application data can be stored in the form of an application list.

[0109] Combined with Figure 8 , the display device 200 can send a data request to the server 400 through the application service, and the data request is used to request to obtain the application data. The server 400 responds to the data request and feeds back the application data to the application service, such as feeding back an application list including the application data. The application service stores and manages the application data.

[0110] If the type of the first data source is a network application, the display device 200 can display the first menu above the page of the first data source according to the status data corresponding to the first data source in the application data. Among them, if the status data corresponding to the first data source supports multi-screen display, the first menu includes a first setting item; if the status data corresponding to the first data source does not support multi-screen display, the first menu does not include a first setting item.

[0111] Combined with Figure 8 , if the multi-screen interface module obtains that the first data source is a network application, it sends an application status request to the application service, and the application status request is used to request to obtain the status data corresponding to the first data source. The multi-screen interface module returns a response message to the quick settings module according to the status data. Among them, if the status data supports multi-screen display, the response message includes support for multi-screen display, and the quick settings module displays the first setting item in the first menu according to the response message; if the status data does not support multi-screen display, the response message includes not supporting multi-screen display, and the quick settings module does not display the first setting item in the first menu according to the response message.

[0112] If the type of the first data source is not the specified type and the network application, that is, the first data source is of other types, display the first menu above the page of the first data source, and the first menu does not include a first setting item.

[0113] Combined with Figure 8 , if the multi-screen interface module obtains that the type of the first data source is neither the specified type nor the network application, it returns a response message to the quick settings module, and the response message includes not supporting multi-screen display. The quick settings module does not display the first setting item in the first menu according to the response message.

[0114] Accordingly, the display device 200 can configure a first setting item for starting the multi-screen display function in the first menu. When displaying the first media asset, the user can quickly start the multi-screen display function based on the first menu and enter the multi-screen interface. Moreover, the display device 200 can dynamically display the first setting item in the first menu according to the actual situation of the multi-screen display supported by the first data source, ensuring the effectiveness of the user's operation based on the first setting item.

[0115] Step S702: When the first menu includes the first setting item, in response to the instruction of the user selecting the first setting item, display the multi-screen interface at the application layer and display the first media asset within the multi-screen interface.

[0116] When the first menu includes the first setting item, the user can control the display device 200 to start the multi-screen display application by selecting the first setting item, thereby starting the multi-screen display function.

[0117] After starting the multi-screen display function, the display device 200 displays a unified user interface provided by the multi-screen display application at the application layer, that is, the multi-screen interface. Media assets provided by different data sources will be displayed within the multi-screen interface in a manner that meets the display requirements of the multi-screen interface. Based on this multi-screen interface, the differences between different data sources in terms of style, size, etc. can be reduced, and the unity of the overall effect can be improved.

[0118] The multi-screen interface includes a main screen area and a secondary screen area. Among them, the main screen area is used to display the first media asset, and the secondary screen area is used to display the media asset provided by the subsequently indicated data source.

[0119] In some embodiments, the secondary screen area can be one or more. If the secondary screen area is one, the multi-screen interface is a dual-screen interface, which can support the simultaneous display of two media assets. If the secondary screen area is multiple, the multi-screen interface can support the simultaneous display of at least two media assets.

[0120] In some embodiments, the user can customize the settings of the main screen area and the secondary screen area. For example, swap the default main screen area and the secondary screen area. Or customize parameters such as the position and size of the main screen area and the secondary screen area.

[0121] Both the main screen area and the secondary screen area are in a transparent state, so that the content located in the video layer can be displayed through the main screen area and the secondary screen area.

[0122] In the embodiments of the present application, first, taking the multi-screen interface including one secondary screen area, that is, supporting dual-screen display, as an example, the process of multi-screen display of multiple media assets will be described.

[0123] Combined with Figure 5C, the display device 200 displays a multi-screen interface at the application layer. The multi-screen interface includes a main screen area 503 and a secondary screen area 504. Both the main screen area 503 and the secondary screen area 504 are in a transparent state.

[0124] After the multi-screen interface is displayed, the media assets provided by each data source will be displayed in the multi-screen interface in a multi-screen display manner.

[0125] For the first media asset provided by the first data source (the data source displayed before the multi-screen display function is started), the first picture is displayed in the area corresponding to the main screen area in the video layer, and the first data source page is converted into a first component and embedded in the main screen area.

[0126] Figure 9 The flowchart of the display device 200 provided by the embodiment of the present application for displaying the first media asset in the main screen area is as follows:

[0127] Step S901: In response to the first operation, exit the first data source page and display the multi-screen interface at the application layer.

[0128] The process of the display device 200 starting the multi-screen display function, that is, the process of the display device 200 starting the multi-screen display application, includes exiting the current data source, starting the multi-screen display application, and restarting the current data source after starting the multi-screen display application. Based on this, the display device 200 responds to the first operation. For example, the display device 200 responds to the instruction of the user selecting the first setting item, exits the first data source, and starts the multi-screen display application.

[0129] In the embodiment of the present application, the operation of the display device 200 to exit the first data source in order to start the multi-screen display application is configured to only exit the first data source page in the application layer in a manner of separating the application layer and the video layer, without exiting the first picture in the video layer. That is, the process of the display device 200 starting the multi-screen display application does not affect the playback of the first media asset.

[0130] Combined with Figure 3 the operating system shown, the application layer is configured with a data source page (Source / App UI) module, and the middleware layer is configured with a page software development kit (UI SDK), a data source (Source / App) middleware, and an application performance management (APM) middleware.

[0131] Figure 10 The timing diagram of the display device 200 provided by the embodiment of the present application for displaying multiple media assets.

[0132] Combined with Figure 10, the display device 200 transmits the instruction to select the first setting item to the UI SDK. In response to this instruction, the UI SDK sends a start instruction to the multi-screen interface module and a page conversion instruction to the data source page module. In response to the start instruction, the multi-screen interface module displays the multi-screen interface at the application layer and sends a first message to the data source page module. The first message may include relevant parameters of the main screen area and the secondary screen area, such as the size and position of the area. In response to the page conversion instruction, the data source page exits the first data source page displayed at the application layer without exiting the playback of the first media asset, that is, the first picture is still displayed at the video layer (the first picture is in full-screen state at this time).

[0133] Step S902: Start the first data source page and convert the first data source page into a first component.

[0134] After the display device 200 starts the multi-screen display application, it restarts the first data source. Among them, the process of restarting the first data source includes reopening the first data source page. When reopening the first data source page in the case of starting the multi-screen display application, the first data source page is no longer regarded as a separate page, but is converted into a component of the multi-screen interface (such as the first component) to display the first data source page in the way of embedding in the multi-screen interface.

[0135] Combined with Figure 10 , the data source page module sends a start instruction to the UI SDK according to the first message. In response to this start instruction, the UI SDK reopens the first data source page and sends a page conversion instruction to the multi-screen interface module. In response to this page conversion instruction, the multi-screen interface module intercepts the first data source page, that is, prevents the first data source page from being displayed in the form of a separate page, and converts the intercepted first data source page into a first component. Among them, the process of the multi-screen interface module converting the first data source page into a first component can be configured to reduce the size of the first data source page according to the size of the main screen area so that the reopened first data source page matches the size of the main screen area, and move the reduced first data source page to the position of the main screen area. Thus, the size of the converted first component can match the size of the main screen area, and its position can correspond to the main screen area.

[0136] Step S903: Reduce the first picture according to the size of the main screen area, and obtain the area corresponding to the main screen area in the video layer according to the position of the main screen area.

[0137] The process of the display device 200 restarting the first data source further includes adjusting the first screen. Among them, the display device 200 reduces the first screen according to the size of the main screen area, and the reduced first screen can match the size of the main screen area. The display device 200 obtains the area corresponding to the main screen area in the video layer according to the position of the main screen area, and this area is the position where the reduced first screen is displayed. The picture located in this area can pass through the main screen area and will not be blocked by the multi-screen interface.

[0138] Step S904: Display the reduced first screen in the area corresponding to the main screen area in the video layer, and embed the first component into the main screen area.

[0139] Combined with Figure 10 , after receiving the first message, the data source page module can send a start instruction to the application performance management middleware, and this start instruction indicates the relevant parameters of the main screen area. The application performance management middleware responds to this start instruction and calls the data source middleware to start the application. Among them, the data source middleware starting the application can include adjusting the first screen according to the relevant parameters of the main screen area. After the data source middleware adjusts the first screen, it sends a feedback message to the application performance management middleware, and this feedback message can include the start status. At this time, this start status indicates that the start is completed, that is, the adjustment of the first screen is completed. The application performance management middleware sends an update instruction to the data source page module according to the start status indicating that the start is completed, and this update instruction can include the relevant parameters of the adjusted first screen, such as the size of the reduced first screen and the area in the video layer used to display the reduced first screen. The data source page module responds to this update instruction and updates the first screen to display the reduced first screen in the area corresponding to the main screen area in the video layer.

[0140] The display device 200 also embeds the first component into the main screen area to convert the first data source page into a part of the multi-screen interface. In this way, the control of the first data source can be directly realized through the first component on the multi-screen interface.

[0141] Combined with Figure 5C , the reduced first screen 505 is displayed in the area corresponding to the main screen area 503 in the video layer, and the first data source page is converted into the first component 506 and embedded into the main screen area 503 in the multi-screen interface, and the secondary screen area 504 remains transparent (schematically shown by a dotted line).

[0142] After the display device 200 activates the multi-screen display function, it can use the first data source currently being displayed as the primary data source. When exiting the first data source, only the first data source page is exited, and the first screen is not exited, that is, the display of the first media asset is not affected. When displaying a multi-screen interface, by adjusting the size and position of the first screen, the first screen is correspondingly displayed in the main screen area, so that the entire first screen can be displayed through the main screen area. Moreover, converting the first data source page into a first component and embedding it in the main screen area allows the user to directly control the first data source based on the first component on the multi-screen interface.

[0143] Step S602: In response to a second operation by the user to display a second media asset provided by a second data source, display the second media asset within the multi-screen interface.

[0144] The second operation is used to activate other data sources that are displayed simultaneously with the first data source, such as the second data source. The media asset provided by the second data source can be referred to as the second media asset.

[0145] In some embodiments, after the display device 200 activates the multi-screen display function, it can directly display a data source list on the multi-screen interface. For example, after the display device 200 activates the multi-screen display function, if it recognizes that only one data source is currently activated or the number of activated data sources is less than the maximum number of data sources that can be displayed simultaneously, it displays a data source list on the multi-screen interface. Among them, the data source list can include items of each data source that supports multi-screen display.

[0146] In other embodiments, after the display device 200 activates the multi-screen display function, in response to a control instruction from the user, it displays a data source list on the multi-screen interface. For example, the user can input a control instruction based on the up arrow key of the control device 100, and the display device 200, in response to this control instruction, can display a data source list on the multi-screen interface.

[0147] The user can select the second data source to be activated from the data source list and perform a second operation based on the item of the second data source, such as selecting the item of the second data source.

[0148] The display device 200, in response to the second operation, activates the second data source. After activating the second data source, a second screen corresponding to the second media asset is displayed in the area corresponding to the secondary screen area in the video layer, and the second data source page for controlling the second data source is converted into a second component and embedded in the secondary screen area.

[0149] Figure 11 The flowchart of the display device 200 provided by the embodiments of the present application for displaying the second media asset in the secondary screen area is as follows:

[0150] Step S1101: In response to the second operation, start the second data source page and convert the second data source page into a second component.

[0151] The process of the display device 200 starting the second data source includes starting the second data source page. Among them, the process of the display device 200 starting the second data source page is similar to the process of restarting the first data source page, which will not be elaborated here.

[0152] Combined with Figure 10 , in response to the second operation, the multi-screen interface module sends a start instruction to the data source page module. In response to this start instruction, the data source page module sends a start instruction to the UI SDK. In response to this start instruction, the UI SDK opens the second data source page and sends a page conversion instruction to the multi-screen interface module. In response to this page conversion instruction, the multi-screen interface module intercepts the second data source page, that is, prevents the second data source page from being displayed in the form of a separate page, and converts the intercepted second data source page into a second component. Among them, the process of the multi-screen interface module converting the second data source page into a second component is similar to the process of converting the first data source page into a first component, which will not be elaborated here.

[0153] Step S1102: Start the second screen according to the size of the secondary screen area, and obtain the area corresponding to the secondary screen area in the video layer according to the position of the secondary screen area.

[0154] The process of the display device 200 starting the second data source also includes starting the second screen. Among them, the display device 200 starts the second screen according to the size of the secondary screen area, and the size of the second screen can match the size of the secondary screen area. The display device 200 obtains the area corresponding to the secondary screen area in the video layer according to the position of the secondary screen area, and this area is the position of the displayed second screen. The picture located in this area can pass through the secondary screen area and will not be blocked by the multi-screen interface.

[0155] Step S1103: Display the second screen in the area corresponding to the secondary screen area in the video layer, and embed the second component into the secondary screen area.

[0156] Combined with Figure 10, in response to the start instruction, the data source page module also sends a start instruction to the application performance management middleware, and the start instruction indicates the relevant parameters of the secondary screen area. In response to this start instruction, the application performance management middleware calls the data source middleware to start the application. Among them, the data source middleware starting the application may include starting the second screen according to the relevant parameters of the secondary screen area, and sending a feedback message to the application performance management middleware after starting the second screen. The feedback message may include a start status, and at this time, the start status indicates that the start is completed. The application performance management middleware sends a display instruction to the data source page module according to the start status indicating that the start is completed. The display instruction may include the relevant parameters of the second screen, such as the size of the second screen and the area in the video layer used to display the second screen. The data source page module displays the second screen in response to this display instruction, so as to display the second screen in the area corresponding to the secondary screen area in the video layer.

[0157] The display device 200 also embeds the second component into the secondary screen area to convert the second data source page into a part of the multi-screen interface. In this way, the control of the second data source can be directly realized through the second component on the multi-screen interface.

[0158] Combined with Figure 5D , the second screen 507 is displayed in the area corresponding to the secondary screen area 504 in the video layer, and the second data source page is converted into a second component 508 and embedded into the secondary screen area 504 in the multi-screen interface.

[0159] When the display device 200 displays the first media asset and the second media asset in a multi-screen manner, after starting the second data source, the size and position of the second screen can be adjusted to display the second screen corresponding to the secondary screen area, so that the complete second screen can be displayed through the secondary screen area. Moreover, converting the second data source page into a second component and embedding it into the secondary screen area can enable the user to directly control the second data source based on the second component on the multi-screen interface.

[0160] The case where the multi-screen interface includes multiple secondary screen areas is described. Taking the multi-screen interface including two secondary screen areas as an example, that is, the display device 200 can display three media assets at the same time. Among them, the process of the display device 200 starting the multi-screen display function when displaying the first media asset can refer to the process of starting the multi-screen display function when the multi-screen interface includes one secondary screen area, such as referring to step S601, step S701 - S702, step S901 - S904, etc., which will not be elaborated here.

[0161] Combined with Figures 12A - 12C The schematic diagram of the multi-media asset display method shown in Figure 12AAs shown, a multi-screen interface is displayed at the application layer. The multi-screen interface includes a main screen area 1201, a first secondary screen area 1202, and a second secondary screen area 1203. In the video layer, a first picture 1204 is displayed in the area corresponding to the main screen area 1201, and both the first secondary screen area 1202 and the second secondary screen area 1203 are in a transparent state (shown by dashed lines).

[0162] The display device 200 can respond to a second operation of the user and display second media assets provided by a second data source in the secondary screen area. In some embodiments, the second operation can indicate a second data source, such as a first sub-data source, and the display device 200 displays the media assets provided by the first sub-data source in a secondary screen area. Among them, the process of the display device 200 displaying the media assets provided by the first sub-data source in a secondary screen area can refer to the process of displaying second media assets when the multi-screen interface includes a secondary screen area, such as referring to step S602, steps S1101 - S1103, etc., which will not be elaborated here.

[0163] Combined Figure 12B As shown, in the video layer, a picture of the media assets provided by the first sub-data source, such as a third picture 1205, is displayed in the area corresponding to the first secondary screen area 1202, and the third data source page corresponding to the first sub-data source is converted into a third component 1206 and embedded in the first secondary screen area 1202. The second secondary screen area 1203 is still in a transparent state (shown by dashed lines).

[0164] Furthermore, the display device 200 can respond to a second operation of the user and display media assets provided by another second data source in another secondary screen area. In some embodiments, the second operation can indicate another second data source, such as a second sub-data source, and the display device 200 displays the media assets provided by the second sub-data source in another secondary screen area. Among them, the process of the display device 200 displaying the media assets provided by the second sub-data source in another secondary screen area can refer to the process of displaying second media assets when the multi-screen interface includes a secondary screen area, such as referring to step S602, steps S1101 - S1103, etc., which will not be elaborated here.

[0165] Combined Figure 12C As shown, in the video layer, a picture of the media assets provided by the second sub-data source, such as a fourth picture 1207, is displayed in the area corresponding to the second secondary screen area 1203, and the fourth data source page corresponding to the second sub-data source is converted into a fourth component 1208 and embedded in the second secondary screen area 1203.

[0166] In some other embodiments, the second operation may indicate two second data sources, such as a first sub-data source and a second sub-data source, that is, the user can indicate two second data sources at one time. The display device 200 displays the media assets provided by the first sub-data source and the second sub-data source in the two secondary screen areas respectively. Among them, the process of the display device 200 displaying media assets in each secondary screen area can refer to the process of displaying the second media assets when the multi-screen interface includes one secondary screen area, such as referring to step S602, steps S1101 - S1103, etc., which will not be elaborated here.

[0167] The process of multi-screen display in the case of more than two secondary screen areas can refer to the process of multi-screen display in the case of two secondary screen areas, which will not be elaborated here.

[0168] In some embodiments, when the display device 200 performs multi-screen display of multiple media assets, based on the component corresponding to the area where each media asset is located, the data source corresponding to the media asset is directly controlled on the multi-screen interface. Specifically, when the multi-screen interface obtains the focus, the user can control the display device 200 to move the focus to the corresponding component, and control the data source through this component.

[0169] In some embodiments, the display device 200 can obtain the focus from the data source when performing multi-screen display of multiple media assets to control the data source.

[0170] In some embodiments, the display device 200 can be configured to flexibly switch the focus between the multi-screen interface and the data source. Among them, the display device 200 can respond to the instruction input by the user based on the specified key and flexibly switch the focus between the multi-screen interface and the data source.

[0171] The specified key can be a basic key on the control device 100 or a specially set key. In one example, the specified key can be the menu key. In this way, when the user uses the multi-screen display function, different operations can be realized based on the menu key in different scenarios, which is more convenient for the user to remember and use.

[0172] In response to the instruction input by the user based on the specified key, if the display device 200 recognizes that the focus is currently obtained by any one of the main screen area and the secondary screen area, it switches to obtaining the focus from the data source corresponding to the target area; if it recognizes that the focus is currently obtained by any one of the first data source and the second data source, it switches to obtaining the focus from the area corresponding to the target data source in the multi-screen interface.

[0173] When the display device 200 performs multi-screen display of multiple media assets, it can flexibly exit the data source.

[0174] In some embodiments, the data sources displayed by the display device 200 are distinguished into a primary data source and a secondary data source. Among them, the primary data source is the data source displayed before the multi-screen display function is started, such as the first data source; the secondary data source is the data source added and displayed after the multi-screen display function is started, such as the second data source. Alternatively, the primary data source is the data source displayed within the main screen area, such as the first data source; the secondary data source is the data source displayed within the secondary screen area, such as the second data source.

[0175] In some embodiments, when the display device 200 exits a data source, different exit operations may be performed according to the primary / secondary situation corresponding to the data source to be exited.

[0176] In some embodiments, the display device 200 may respond to a second instruction to exit the second data source and exit the second data source. That is, the display device 200 may respond to an instruction to exit the secondary data source, exit the secondary data source, but does not exit the multi-screen interface and does not affect the display of the primary data source.

[0177] In some embodiments, the second instruction may be an instruction generated when the second data source actively exits. In some other embodiments, the second instruction may be an instruction input by the user to exit the second data source.

[0178] In one example, the display device 200 may respond to a second instruction input by the user based on the return key of the control device 100 and exit the second data source when it is recognized that the secondary screen area corresponding to the second data source obtains focus.

[0179] In another example, the display device 200 may respond to a second instruction input by the user based on the return key of the control device 100 and exit any secondary data source when it is recognized that multiple data sources are currently being displayed.

[0180] Figure 13 This is a timing diagram for the display device 200 provided by the embodiments of the present application to exit a data source.

[0181] Combined with Figure 13 , if the second instruction is an instruction generated when the second data source actively exits, after the data source middleware detects the active exit of the second data source, it may send a first notification to the application performance management middleware. The first notification includes information about exiting the second data source. The application performance management middleware sends a notification to exit the secondary screen to the multi-screen interface module according to this first notification. The notification includes the secondary screen area corresponding to exiting the second data source. The multi-screen interface module sends a control instruction to the data source page module according to this notification to exit the secondary screen. The data source page module responds to this control instruction and exits the second picture displayed in the video layer. The multi-screen interface module also destroys the second component according to this notification to vacate the secondary screen area in the application layer.

[0182] If the second instruction is an instruction to exit the second data source input by the user, the display device 200 transmits the instruction to the data source page module. In response to the instruction, the data source page module sends a request-to-exit message to the multi-screen interface module. The message may include the secondary screen area corresponding to the second data source. The multi-screen interface module may, according to the message, destroy the second component to free up the secondary screen area at the application layer. In response to the instruction to exit the second data source, the data source page module also sends an instruction to exit the data source to the application performance management middleware. In response to the instruction to exit the data source, the application performance management middleware sends an instruction to exit the data source to the data source middleware. The instruction to exit the data source instructs to exit the second data source. In response to the instruction to exit the data source, the data source middleware exits the second picture displayed in the video layer. After exiting the second picture, the data source middleware returns a data source exit notification to the application performance management middleware. The data source exit notification is used to notify that the second data source has been exited. The application performance management middleware, according to the data source exit notification, sends a data source exit status notification to the multi-screen interface module. The data source exit status notification is used to notify that the second data source has been exited.

[0183] In some embodiments, the display device 200 may exit the multi-screen interface in response to a third instruction to exit the first data source. That is to say, the display device 200 may directly exit the multi-screen display function in response to an instruction to exit the primary data source, rather than only exiting the primary data source.

[0184] In some embodiments, the third instruction may be an instruction generated when the first data source actively exits. In other embodiments, the third instruction may be an instruction to exit the first data source input by the user.

[0185] Figure 14 The following is a flowchart of the display device 200 provided by the embodiments of the present application for exiting the primary data source. The specific steps are as follows:

[0186] Step S1401: Exit the multi-screen interface in response to a third instruction to exit the first data source.

[0187] In some embodiments, the display device 200 exits the multi-screen display application in response to a third instruction to exit the first data source to exit the multi-screen interface. Among them, in the process of exiting the multi-screen display application, the display device 200 exits the multi-screen interface from the application layer. Exiting the multi-screen interface can destroy the first component embedded in the multi-screen interface and the second component embedded in the multi-screen interface. The display device 200 also exits the second picture displayed in the video layer. Thus, the second data source is exited. That is to say, the process of exiting the multi-screen display application does not affect the playback of the first data source (primary data source).

[0188] Step S1402: Adjust the first screen to the full-screen state, start the first data source page, and display the first data source page in the application layer in the full-screen state.

[0189] The display device 200 is configured to redisplay the first data source (main data source) in the full-screen state after exiting the multi-screen display application. The process of the display device 200 exiting the multi-screen display application and redisplaying the first data source includes exiting the multi-screen display application and restarting the first data source. Among them, the display device 200 restarting the first data source may include adjusting the first screen to the full-screen state, restarting the first data source page in the full-screen state, and displaying the restarted first data source page in the application layer.

[0190] Combined with Figure 13 , if the third instruction is an instruction generated when the first data source actively exits, after the data source middleware detects the active exit of the first data source, it can send a second notification to the application performance management middleware, and the second notification includes information about the exit of the first data source. According to this second notification, the application performance management middleware sends a notification to exit the main screen to the multi-screen interface module, and the notification includes the corresponding main screen area of the exited first data source. The multi-screen interface module sends a message to exit the main screen to the data source page module according to this notification to exit the main screen. The data source page module can exit the second screen displayed in the video layer according to this message.

[0191] If the third instruction is an instruction for the user to input to exit the first data source, the display device 200 transfers the instruction to the data source page module. The data source page module sends a message requesting to exit according to this instruction to the multi-screen interface module, and the message may include the main screen area corresponding to the first data source. The multi-screen interface module can destroy the first component according to this message. In response to the instruction to exit the first data source, the data source page module also sends an instruction to exit the data source to the application performance management middleware. The application performance management middleware sends an instruction to exit the secondary data source to the data source middleware in response to this instruction to exit the data source. The data source middleware exits the second screen displayed in the video layer in response to this instruction to exit the secondary data source, and after exiting the second screen, returns a notification of the exit of the secondary data source to the application performance management middleware, and this notification of the exit of the secondary data source is used to notify that the secondary data source has exited.

[0192] The multi-screen interface module exits the multi-screen interface in response to a notification to exit the main screen sent by the application performance management middleware or a message to request exit sent by the data source page module, and sends an instruction to start the main data source to the data source page module. The instruction to start the main data source indicates to start the main data source in full-screen state. In response to the instruction to start the main data source, the data source page module re-opens the first data source page in full-screen state and sends an application start status to the application performance management middleware, where the application start status indicates that the first data source has been started. In response to the instruction to start the main data source, the data source page module also readjusts the first screen to full-screen state to achieve the effect of destroying the small-size display in the multi-screen interface.

[0193] In some embodiments, when the display device 200 exits the data source, it may not distinguish between the main data source and the secondary data source. That is to say, the display device 200 can respond to an instruction from the user to exit any data source, exit the corresponding data source, and when it detects that the data source to be exited is the only currently displayed data source, exit the multi-screen display function and re-display the data source to be exited in full-screen state.

[0194] Figure 15 The flowchart for the display device 200 provided by the embodiments of the present application to exit the data source is as follows:

[0195] Step S1501: Exit the first target data source in response to a fourth instruction to exit the first target data source.

[0196] Wherein, the first target data source is any one of the first data source and the second data source.

[0197] The display device 200 exiting the first target data source is configured to exit the screen corresponding to the first target data source and destroy the components corresponding to the first target data source from the multi-screen interface.

[0198] Step S1501 can refer to the process of exiting the secondary data source and will not be elaborated here.

[0199] Step S1502: In response to a fifth instruction to exit the second target data source, if no data source other than the second target data source is currently displayed in the multi-screen interface, exit the multi-screen interface.

[0200] Wherein, the second target data source is any one of the first data source and the second data source.

[0201] The display device 200 exiting the multi-screen interface is configured to destroy the components corresponding to the second target data source from the multi-screen interface.

[0202] Step S1503: Adjust the screen corresponding to the second target data source to the full-screen state, start the data source page corresponding to the second target data source, and display the data source page corresponding to the second target data source in the application layer in the full-screen state.

[0203] Steps S1502 - S1503 may refer to Steps S1401 - S1402, which will not be elaborated here.

[0204] When the display device 200 displays multiple data sources in a multi-screen manner, it can respond to a fourth instruction to exit any data source (such as the first target data source) indicated by the user, exit the first target data source, and continue to display the remaining data sources in the multi-screen display state. The display device 200 can continue to respond to a fifth instruction to exit any data source (such as the second target data source) indicated by the user. When it is detected that there are no other data sources except the second target data source currently, it directly exits the multi-screen display. After exiting the multi-screen display, the second target data source is adjusted to the full-screen state to ensure the display effect of the second target data source.

[0205] For the sake of convenience in explanation, the above description has been made in combination with specific embodiments. However, the above discussion in some embodiments is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are for better explaining the content of the present disclosure, so that those skilled in the art can better use the embodiments.

Claims

1. A display device, characterized in that: include: A display, configured to: display a first media asset provided by a first data source, wherein a first picture of the first media asset is displayed on a video layer, and a first data source page for controlling the first data source is displayed on an application layer, wherein the application layer is located above the video layer; The controller is configured as: In response to a first operation of the user, the display is controlled to display a multi-screen interface in the application layer, and the first media resource is displayed in the multi-screen interface, wherein the multi-screen interface includes a main screen area and a secondary screen area, and both the main screen area and the secondary screen area are transparent, the first picture is displayed in an area corresponding to the main screen area in the video layer, and the first data source page is converted into a first component and embedded in the main screen area; In response to a user instruction for a second operation to display a second media asset provided by a second data source, the display is controlled to display the second media asset in the multi-screen interface, wherein a second picture corresponding to the second media asset is displayed in an area of ​​the video layer corresponding to the secondary screen area, and a second data source page for controlling the second data source is converted into a second component embedded in the secondary screen area.

2. The display device according to claim 1, characterized in that The controller executes, in response to a first operation of a user, controlling the display to display a multi-screen interface at the application layer, and displaying the first media asset in the multi-screen interface, and is specifically configured as follows: In response to a first instruction from a user, controlling the display to display a first menu on an upper layer of the first data source page, wherein, in the case where the first data source supports multi-screen display, the first menu includes a first setting item for starting a multi-screen display function; In the case where the first menu includes the first setting item, in response to an instruction of a user selecting the first setting item, the display is controlled to display the multi-screen interface at the application layer, and to display the first media asset in the multi-screen interface.

3. The display device according to claim 2, characterized in that The display device further includes a memory configured to store application data, wherein the application data includes status data of whether an application provided by a network supports multi-screen display; The controller executes, in response to a first instruction from a user, controlling the display to display a first menu on an upper layer of the first data source page, and is specifically configured as follows: In response to the first instruction, obtaining a type of the first data source; If the type of the first data source is a specified type, controlling the display to display the first menu on an upper layer of the first data source page, the first menu including the first setting item; If the type of the first data source is a network application, according to the status data corresponding to the first data source in the application data, controlling the display to display the first menu on an upper layer of the first data source page, wherein if the status data corresponding to the first data source is supporting multi-screen display, the first menu includes the first setting item; if the status data corresponding to the first data source is not supporting multi-screen display, the first menu does not include the first setting item; If the type of the first data source is neither the designated type nor the network application, the display is controlled to display the first menu on an upper layer of the first data source page, wherein the first menu does not include the first setting item.

4. The display device according to claim 1, characterized in that The controller executes, in response to a first operation of a user, controlling the display to display a multi-screen interface at the application layer, and displaying the first media asset in the multi-screen interface, and is specifically configured as follows: In response to the first operation, controlling the display to exit the first data source page and displaying the multi-screen interface at the application layer; Starting the first data source page, and converting the first data source page into the first component; According to the size of the main screen area, the first picture is reduced, and according to the position of the main screen area, an area in the video layer corresponding to the main screen area is acquired; The reduced first picture is displayed in an area of ​​the video layer corresponding to the main screen area, and the first component is embedded in the main screen area.

5. The display device according to claim 1, characterized in that The controller performs a second operation of displaying a second media asset provided by a second data source in response to a user instruction, and controls the display to display the second media asset in the multi-screen interface, and is specifically configured as follows: In response to the second operation, starting the second data source page and converting the second data source page into the second component; According to the size of the secondary screen area, start the second picture, and according to the position of the secondary screen area, obtain the area in the video layer corresponding to the secondary screen area; The second picture is displayed in an area of ​​the video layer corresponding to the secondary screen area, and the second component is embedded in the secondary screen area.

6. The display device according to claim 1, characterized in that After the controller controls the display to display the second media asset in the multi-screen interface in response to a second operation instructing the user to display the second media asset provided by the second data source, the controller is further configured to: In response to a second instruction to exit the second data source, the second data source is exited, wherein the second screen is exited and the second component is destroyed from the secondary screen area.

7. The display device according to claim 1, characterized in that After the controller controls the display to display the second media asset in the multi-screen interface in response to a second operation instructing the user to display the second media asset provided by the second data source, the controller is further configured to: In response to a third instruction to exit the first data source, exit the multi-screen interface, wherein the first component is destroyed from the main screen area, and the second data source is exited; The display is controlled to adjust the first picture to a full-screen state, and the first data source page is started, and the first data source page is displayed in a full-screen state at the application layer.

8. The display device according to claim 1, characterized in that After the controller controls the display to display the second media asset in the multi-screen interface in response to a second operation instructing the user to display the second media asset provided by the second data source, the controller is further configured to: In response to a fourth instruction to exit a first target data source, exit the first target data source, wherein the screen corresponding to the first target data source is exited, and the component corresponding to the first target data source is destroyed from the multi-screen interface, and the first target data source is any one of the first data source and the second data source; In response to a fifth instruction to exit the second target data source, if the multi-screen interface currently does not display a data source other than the second target data source, exit the multi-screen interface, wherein a component corresponding to the second target data source is destroyed from the multi-screen interface, and the second target data source is any one of the first data source and the second data source; The display is controlled to adjust the screen corresponding to the second target data source to a full-screen state, and a data source page corresponding to the second target data source is started, and the data source page corresponding to the second target data source is displayed in a full-screen state at the application layer.

9. The display device according to claim 1, characterized in that: After the controller controls the display to display the second media asset in the multi-screen interface in response to a second operation instructing the user to display the second media asset provided by the second data source, the controller is further configured to: In response to an instruction input by a user based on a designated key, when any one of the target areas in the main screen area and the secondary screen area obtains focus, switching to a data source corresponding to the target area to obtain focus; In response to an instruction input by a user based on the designated key, when any one of the first data source and the second data source obtains focus, switching to an area corresponding to the target data source in the multi-screen interface to obtain focus.

10. A method for displaying multimedia resources, characterized in that: applied to a display device, the display device displays a first media asset provided by a first data source, wherein a first picture of the first media asset is displayed on a video layer, and a first data source page for controlling the first data source is displayed on an application layer, the application layer being located above the video layer; The method comprises: In response to a first operation of the user, a multi-screen interface is displayed in the application layer, and the first media resource is displayed in the multi-screen interface, wherein the multi-screen interface includes a main screen area and a secondary screen area, and both the main screen area and the secondary screen area are transparent, the first picture is displayed in an area corresponding to the main screen area in the video layer, and the first data source page is converted into a first component and embedded in the main screen area; In response to a user's instruction to display a second operation of a second media asset provided by a second data source, the second media asset is displayed in the multi-screen interface, wherein a second picture corresponding to the second media asset is displayed in an area of ​​the video layer corresponding to the secondary screen area, and a second data source page for controlling the second data source is converted into a second component embedded in the secondary screen area.

Images