Display device and play control method

Abstract

The invention provides a display device and a play control method, and the method can respond to a play instruction indicating to play target media assets on a second display, create a media play window, and control the second display to display the media play window. Wherein the media playing window is used for playing the target media assets. And creating a broadcast control menu of the media playing window, and controlling the first display to display the broadcast control menu. Wherein the playing control menu comprises a control used for executing playing control on the playing process of the target media asset. According to the method, the playing control menu can be displayed on the first display during double-screen display, playing control is carried out on the media assets played on the second display, and the problem that playing control cannot be carried out on the media assets displayed on the second display due to the fact that the second display is out of focus is solved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a playback control method. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, communication terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] The display device can form a dual-screen system by connecting an external display to its own display to achieve a dual-screen display mode. In some interactive scenarios, the dual-screen display mode includes an extended mode and a clone mode. In the clone mode, the display content of the main screen (the display of the display device itself) is the same as the display content of the secondary screen (the external display). In the extended mode, the display content of the main screen is different from the display content of the secondary screen.

[0004] However, in a dual-screen interactive scenario in which the display device is in extended mode, when media data such as videos, music, and pictures are displayed only on the secondary screen, the secondary screen may be out of focus, resulting in the inability to operate and control media data such as videos, music, and pictures displayed on the secondary screen, reducing the user experience. Summary of the invention

[0005] The present application provides a display device and a playback control method to solve the problem that the playback of media displayed on the secondary screen cannot be controlled due to the secondary screen being out of focus in a dual-screen system.

[0006] In a first aspect, the present application provides a display device, including a first display, a device interface, and a controller. The first display is configured to display a first user interface; the device interface is configured to connect to a second display, the second display is used to display a second user interface; the controller is configured to execute the following program steps:

[0007] In response to a play instruction instructing to play the target media asset on the second display, creating a media play window, and controlling the second display to display the media play window, wherein the media play window is used to play the target media asset;

[0008] A play control menu of the media play window is created, and the first display is controlled to display the play control menu, wherein the play control menu includes controls for performing play control on the play process of the target media asset.

[0009] In a second aspect, the present application further provides a playback control method, which is applied to the above-mentioned display device, and the method includes:

[0010] In response to a play instruction instructing to play the target media asset on the second display, creating a media play window, and controlling the second display to display the media play window, wherein the media play window is used to play the target media asset;

[0011] A play control menu of the media play window is created, and the first display is controlled to display the play control menu, wherein the play control menu includes controls for performing play control on the play process of the target media asset.

[0012] It can be seen from the above technical solutions that the present application provides a display device and a playback control method, which can create a media playback window in response to a playback instruction indicating that the target media asset is to be played on the second display, and control the second display to display the media playback window. The media playback window is used to play the target media asset. Then, a playback control menu for the media playback window is created, and the first display is controlled to display the playback control menu. The playback control menu includes a control for performing playback control on the playback process of the target media asset. The method can control the playback of the media asset played on the second display by displaying the playback control menu on the first display when dual-screen display is used, thereby solving the problem that the second display is out of focus and the media asset displayed on the second display cannot be played back. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 A schematic diagram of a usage scenario of a display device provided in some embodiments of the present application;

[0015] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0016] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0017] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0018] Figure 5 A structural diagram of a dual-screen display system provided for some embodiments of the present application;

[0019] Figure 6 A schematic diagram of a dual-screen display effect provided in some embodiments of the present application;

[0020] Figure 7 A schematic diagram of a sidebar tool provided for some embodiments of the present application;

[0021] Figure 8 A schematic diagram of a dual-screen interaction scenario of a display device in extended mode provided in some embodiments of the present application;

[0022] Fig. 9 A flowchart of a playback control method provided in some embodiments of the present application;

[0023] Fig.10 A schematic diagram of a playback control menu provided in some embodiments of the present application;

[0024] Fig.11 Another schematic diagram of a playback control menu provided in some embodiments of the present application;

[0025] Fig.12 A schematic diagram of the display effect of the playback control menu provided in some embodiments of the present application;

[0026] Fig.13 A schematic diagram of a progress bar display provided in some embodiments of the present application;

[0027] Fig.14 A schematic diagram showing that the play button control in the play control menu provided in some embodiments of the present application is a pause icon;

[0028] Fig.15 A schematic diagram of a play button control in a play control menu provided for some embodiments of the present application is a play icon. DETAILED DESCRIPTION

[0029] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0030] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0031] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0032] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0033] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0034] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through the mobile terminal 300 and the control device 100 .

[0035] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0036] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0037] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0038] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0039] In some embodiments, the display device 200 includes at least one of a tuner 210 , a communicator 220 , a detector 230 , a device interface 240 , a controller 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a communication device interface 280 .

[0040] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0041] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0042] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0043] In some embodiments, the display device 200 may also support touch interaction, that is, a touch interaction module may be provided on the display device 200, and the touch interaction module may be a built-in module or an external module. For example, in order to implement touch interaction operations, a touch panel (TP) may be provided on the display 260 of the display device 200, and the touch panel may receive touch signals input by fingers, contacts, styluses, etc., and execute touch interaction responses according to a pre-set interaction strategy. Of course, the display device 200 may also be a touch-integrated screen.

[0044] In some embodiments, the communicator 220 is a component for communicating with an external device or a server 400 according to various communication protocol types.

[0045] In some embodiments, the device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0046] For example, in order to enrich the display types of the display device 200, the HDMI interface can connect devices with video and audio input / output functions, and implement the corresponding audio and video output / input functions through the HDMI driver in the hardware layer. The USB interface can connect devices with signal and data transmission functions and interactive functions, and access signals, transmit data, and perform specific interactive actions based on the USB driver in the hardware layer.

[0047] In some embodiments, the display device 200 can support multiple interface accesses and perform corresponding functions according to the specific device type accessed. For example, the display device 200 can be connected to another display or terminal device 500 via an HDMI interface to form display functions such as dual screen, split screen, screen transmission, conference, whiteboard, NFC, etc. The display device 200 can also be connected to the terminal device 500 via a USB interface, and activate the Open Pluggable Specification (OPS) channel to connect the terminal device 500. When the terminal device 500 is connected through the OPS channel, since the display device 200 has an independent operating system (system A), and the terminal device 500 can also have an independent operating system (system B), a dual system mode can be formed.

[0048] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0049] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0050] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0051] In some embodiments, the communication device interface 280 is an interface that can be used to receive control input.

[0052] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 includes a controller 110 , a communication interface 130 , a user input / output interface, a memory 190 , and a power supply 180 .

[0053] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0054] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0055] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0056] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0057] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0058] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0059] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0060] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0061] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0062] Figure 4 Some embodiments of the present application provide Figure 1 Schematic diagram of software configuration in the display device. In some embodiments, the system of the display device 200 can be divided into three layers, namely, application layer, middleware layer and hardware layer from top to bottom.

[0063] The application layer mainly includes commonly used applications on TV and application frameworks. Common applications are mainly applications developed based on browsers, such as HTML5 APPs and native APPs.

[0064] The Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0065] Native apps can support online or offline, message push or local resource access.

[0066] The middleware layer includes various TV protocols, multimedia protocols, system components and other middleware. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, and can achieve the purpose of resource sharing and function sharing.

[0067] The hardware layer mainly includes the hardware abstraction layer (HAL) interface, hardware and drivers. The HAL interface is a unified interface for all TV chips to connect, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0068] In some embodiments, the display device 200 can connect an external display via HDMI output to form a dual-screen system with the display 260 itself to achieve a dual-screen display mode. Figure 5 As shown, the dual-screen system may include an image generation (ImageProducer) module, an image rendering (surface flinger) module, a synthesis (Composer) module, a video process (VideoProcessing Unit hardware, VPU hardware) module, a main screen (panel screen) and a secondary screen (HDMI screen).

[0069] During the display process, the media player (media player), camera peripheral (camera per view) and graphics interface (openGL ES) of the image generation module can generate image content and pass the image content to the image rendering module. The image rendering module includes a primary display unit (primary display) and an extended display unit (external display). The primary display unit is used to control the display content of the main screen, that is, the main display unit renders the image content through the user interaction layer (UI layer) or the video layer (video layer) according to the image content. In order to obtain a better display effect, the image content of the UI layer can be rendered by the GPU component, so as to form a picture frame for display in the synthesis module. Among them, the synthesis module can form the picture content by software or hardware. For example, the synthesis module can be a hardware synthesis (Hardware Composer, HWC) module.

[0070] In the synthesis module, the picture frames can be stored through the frame buffer and the video buffer, and the picture content can be formed in the corresponding layers through the video process module. That is, the image frames generated by the video playback process form the picture content through the video plane, and the image frames of other image pictures (such as UI graphics, etc.) form the picture content through the primary OSD plane. Finally, the display device 200 forms the final picture through the blending unit of the video process module and displays it on the main screen.

[0071] The extended display unit is used to control the display content of the secondary screen, that is, after the image generation module passes the image content to the image rendering module, the extended display unit also renders the image content through the UI layer or the video layer. According to the display process of the main screen, the image frame is formed by the image rendering module, the synthesis module and the video process module respectively, and displayed on the secondary screen.

[0072] Based on the above dual-screen display system, the user can control the display device 200 to execute different display strategies as needed. For example, the main screen is connected to the secondary screen through HDMIOUT to achieve dual-screen display. After the connection is completed, the display device 200 can control the display of the main screen and the secondary screen. Among them, the main screen and the secondary screen can support the same display, that is, the display 260 (main screen) of the display device 200 and the external display display the same content. The main screen and the secondary screen can also support different displays, that is, the content displayed on the main screen and the secondary screen is different. Different display is also called extended display. The main scenarios include: the physical signal source screen is displayed on the secondary screen; the player screen is displayed on the secondary screen; the wireless screen transmission screen displays two screen transmission screens; the conference application displays the shared content on the secondary screen, etc.

[0073] like Figure 6 As shown, according to the above dual-screen system connection method, after connecting to the external screen via an HDMI cable, the dual-screen mode can be started. In the dual-screen mode, the display device 200 can set the display mode of the secondary screen, and the display mode defaults to the extended mode, that is, the secondary screen displays the content of the main screen in an extended manner, which is equivalent to increasing the display area of ​​the main screen. Therefore, the wallpaper displayed on the secondary screen can automatically change with the wallpaper displayed on the main screen.

[0074] After starting the dual-screen mode, the main screen and the secondary screen can display the user interface according to the set display mode. In order to facilitate page control, views can be displayed on the main screen and the secondary screen respectively. In the display device 200, a view is an element of the user interface that can be used to display data, layouts, and controls, such as buttons, text boxes, lists, cards, etc. For example, a view can be as follows: Figure 7 The sidebar and its associated extended menu shown in FIG. That is, the interface displayed on the main screen and the secondary screen may include a sidebar, and the sidebar can be used to control the content displayed on another screen under one screen environment. For example, Figure 7 As shown, the sidebar is an arrow icon displayed on both sides of the interface, and the user can click the icon to call out the extended menu. The extended menu can include quick function options such as "Back", "Home", "Settings", "Switch", etc. The user can click any option icon to control the dual-screen display process.

[0075] In some embodiments, the display process of the sidebar can be controlled by an independent application, that is, the display device 200 can run a sidebar application, and after the sidebar application is started, the sidebar views can be laid out in the two screens respectively, so as to display the sidebar on the two screens respectively. In addition, the sidebar application can detect the status of the two screens in real time, and adjust the display form of the sidebar and the content in the extended menu in real time based on the detected status.

[0076] The sidebar application can also support operations such as adjusting, switching, and closing the content displayed on the main screen and the secondary screen. For example, the user can adjust the display mode of the main screen and the secondary screen through the sidebar, that is, in the extended mode, the user can adjust the main screen and the secondary screen to the copy mode with the same display content by clicking the "copy mode" control in the sidebar extended menu. In order to implement the copy mode, the display device 200 can modify the signal source output form of the display device 200 to copy when it detects that the display mode parameter is modified to the parameter value corresponding to the copy mode, such as "display mode = 01 (copy mode)", that is, copy the image frame in the frame buffer, and send it to the main screen and the secondary screen for display synchronously.

[0077] In the extended mode, the display device 200 can control the user interface to be displayed on the main screen or on the secondary screen according to the user settings and the application display requirements. For example, when the user controls to start application A, the screen ID to be displayed can be set for application A. When the screen ID to be displayed is set to panel screen, it means that its corresponding interface is displayed on the main screen; when the screen ID to be displayed is set to HDMI screen, it means that its corresponding interface is displayed on the secondary screen.

[0078] In some embodiments, the display device 200 can also automatically set the screen ID of the application display according to the application parameters. For example, according to the startup order of the applications in the display device 200, the application that is started first is displayed on the main screen, and the application that is started later is displayed on the secondary screen. Then, when the application is started, the display device 200 can detect the number of applications that have been started in addition to the application that is currently being started. When the number of applications started is 0, the screen ID of the display is set to panel screen; when the number of applications started is 1, the screen ID of the display is set to HDMI screen. Since the display device 200 can run multiple applications at the same time, the screen ID of the display can also be set according to the odd and even numbers of the application startup order. That is, the application with an odd startup order is displayed on the main screen, and the screen ID of the display is set to panel screen; the application with an even startup order is displayed on the secondary screen, and the screen ID of the display is set to HDMI screen.

[0079] In some embodiments, the display device 200 can construct an application status table to record relevant information of the application displayed on the main screen or the secondary screen. The recorded information can be recorded and stored through the application status table, and other applications can obtain the specific display parameters of the application by calling and reading the application status table when needed. For example, the sidebar application can obtain which screen the application is displayed on by reading the application status table, and can change the display mode by modifying the application status table. That is, the user can specify the screen to display the user interface through the extended menu of the sidebar. When the user clicks the "main screen" control in the sidebar extended menu of the secondary screen, the sidebar application can modify the display screen ID of the secondary screen application to panelscreen, and update the application status table to adjust the application interface displayed on the secondary screen to the main screen for display.

[0080] The sidebar application can also support the dual-screen swap display function, that is, there is a "swap" control in the extended menu of the sidebar. When the user clicks the "swap" control, the sidebar application can modify the display screen ID of the main screen display application A to hdmiscreen, and modify the screen display channel through the hardware layer to control the display of the interface of application A on the secondary screen; at the same time, modify the display screen ID of the secondary screen display application B to panel screen to control the display of the interface of application B on the main screen.

[0081] The content displayed on the main screen and the secondary screen can also be adjusted according to the signal source input. For example, when the display device 200 is not connected to the signal source and the main screen displays the home page, the secondary screen can display the home page wallpaper according to the home page content. When the display device 200 is connected to the signal source and the display screen ID of the signal source is set to HDMI screen, the content corresponding to the signal source (including the no signal interface) is displayed on the secondary screen.

[0082] The user can also control the exit of the dual-screen mode through the sidebar tool. For example, when the user controls the display device 200 to stop running the application B corresponding to the secondary screen by clicking the "Back" control in the secondary screen sidebar multiple times, the display device 200 can exit the dual-screen mode. For another example, the extended menu of the sidebar includes a "Close" control. When the user clicks the "Close" control, the application displayed on the corresponding screen can be controlled to be closed.

[0083] When the display device 200 exits the dual-screen mode, the remaining user interface can be displayed on one of the main screen or the secondary screen. For example, since the display device 200 has its own display, it is more convenient for data transmission and interface rendering. Therefore, when the user closes the display application A through the sidebar of the main screen, the application B displayed on the secondary screen can be automatically adjusted to the main screen for display.

[0084] It should be noted that in the above example, the display device 200 is used as the signal input source, while in actual applications, the signal input source can be the display device 200 or the terminal device 500 externally connected to the display device 200. In the case where the signal input source is the external terminal device 500, the processes of adjusting, switching, and displaying the screen in the above example can be executed by the terminal device 500, or by the display device 200, or by both the display device 200 and the terminal device 500.

[0085] In some embodiments, based on the dual-screen display system, the user can control the display device 200 to execute different display strategies according to the needs. In order to facilitate the user to interact with the main screen and the secondary screen, the display device 200 can preset the focus switching strategy to set the focus screen, wherein the focus screen is the display screen that obtains the focus, that is, the display screen that responds to the user's operation.

[0086] In the dual-screen display scenario of the display device 200. For the copy mode, since the display content of the main screen is the same as the display content of the secondary screen, the secondary screen synchronously displays the display content on the main screen. Therefore, the display device 200 only needs to set the main screen as the focus screen, and there is no problem of focus switching. For the extended mode, the display content of the main screen is different from the display content of the secondary screen. In order to independently control the display content of the main screen and the secondary screen, the problem of focus switching will be involved.

[0087] For example, Figure 8 As shown in FIG. 1 , a schematic diagram of a dual-screen interactive scene of a display device 200 in an extended mode provided in an embodiment of the present application is shown. The display device 200 has a built-in display 260 as the main screen, and an external display as the secondary screen. The main screen displays Figure 8 The interface of the file manager shown in FIG. 1 includes media data such as videos, music, and pictures stored in the display device 200. The user can click the option icon of the target media to control the display device 200 to start the media player and play the target media on the secondary screen. At this time, the display device 200 sets the secondary screen as the focus screen, that is, the user can control the playback of the media player on the secondary screen by means of a remote control or the like.

[0088] When the secondary screen plays the target media, if the user touches the main screen content, the display device 200 will set the focus to the main screen according to the preset focus switching strategy, which will cause the secondary screen to lose focus, making it impossible for the user to control the playback of the media player on the secondary screen.

[0089] It can be seen that in the above implementation, when the display device 200 is in the dual-screen interactive scenario in the extended mode, when media data such as video, music, and pictures are displayed on the secondary screen, there will be a problem of the secondary screen being out of focus, resulting in the inability to operate and control the media data such as video, music, and pictures displayed on the secondary screen, thereby reducing the user experience.

[0090] In order to solve the above problems, some embodiments of the present application provide a playback control method, which is applied to the display device 200 and is used to control the media played on the secondary screen through the playback control menu on the main screen. Among them, the display device 200 that can apply the playback control method includes: a first display, a device interface 240 and a controller 250. Among them, the first display is the display 260 (main screen) of the display device 200 itself, which is configured to display a first user interface; the device interface 240 is configured to connect to a second display (secondary screen) for displaying a second user interface. The controller 250 is configured to execute the above-mentioned playback control method, such as Fig. 9 As shown, Fig. 9 The flowchart of the playback control method provided in the embodiment of the present application specifically includes the following contents:

[0091] S100: In response to a play instruction instructing to play a target media asset on a second display, create a media play window, and control the second display to display the media play window.

[0092] The display device 200 uses an external display as a second display through the device interface 240, and the display 260 provided by the display device 200 is used as a first display, and dual-screen display is realized through the first display and the second display. When dual-screen display is performed, the display device 200 can set a dual-screen mode. When the dual-screen mode is set to a copy mode, the first display and the second display are regarded as a logical display, and the contents displayed by the two are the same. When the dual-screen mode is set to an extended mode, display contents can be allocated to the first display and the second display, so that the second display can display the contents of the first display in an extended manner.

[0093] In this regard, in order to improve the user experience, in the extended mode, the display device 200 can set the media data such as video, music, and pictures to be displayed only on the second display. When the user inputs a play instruction for playing the target media, the display device 200 can respond to the play instruction and detect the current dual-screen mode of the display device 200. If the dual-screen mode is the copy mode, the first display is controlled to play the target media, and the second display is controlled to synchronously display the display content on the first display. If the dual-screen mode is the extended mode, the second display is controlled to play the target media. At this time, the display device 200 can also display other media screens based on the first display to achieve independent display of the first display and the second display.

[0094] The dual-screen mode can be set manually by the user. The display device 200 can be pre-set with a mode switch control for the dual-screen mode so that the user can set the dual-screen mode of the display device 200. For example, after the display device 200 is turned on, the sidebar application is started to control the first display to display the following Figure 7 The sidebar shown in the figure can be called out by the user by clicking the sidebar icon. The extended menu can include quick function options such as "Back", "Home", "Settings", "Switch", and "Mode". The user can click the "Mode" option icon to enter the mode switching menu. Then the user clicks the "Extended Mode" or "Copy Mode" option icon to set the dual-screen mode of the display device 200 to the extended mode or the copy mode. In addition, the dual-screen mode of the display device 200 can also default to the extended mode so that the second display can display the content of the first display in an extended manner.

[0095] In some embodiments, in the extended mode, before the display device 200 controls the second display to play the target media, the display device 200 also needs to detect whether the second display is currently connected. If the second display is currently connected, the second display can be controlled to play the target media. If the second display is not currently connected, the first display is controlled to play the target media.

[0096] For whether the display device 200 is currently connected to a second display, the display device 200 can obtain a display list of the display device 200. The display list includes the display identification data (DisplayIdentification Data, DisplayId) currently loaded by the display device 200. The display identification data is a data type that identifies the display, which is assigned to the display by the operating system when the driver is initialized. It is an identifier (id) used by the DisplayManagerService in the display device 200 system to manage the screen, and is used to identify and manage different displays.

[0097] By traversing the display list, if there is only a display identification with the first attribute value in the display list, it means that the display device 200 is not currently connected to the second display. If there is a display identification with a value other than the first attribute value in the display list, it means that the display device 200 is currently connected to the second display.

[0098] For example, the display device 200 obtains a DisplayManager instance from the system service. DisplayManager is a class used to manage display screens in the operating system of the display device 200. The DisplayId of all currently connected displays is obtained through the getDisplays() method of DisplayManager, and the obtained DisplayIds are traversed. If there is only a DisplayId of 0, it means that the display device 200 is currently only connected to the first display (the display of the display device 200 itself) and the second display is not connected. If there is a DisplayId that is not 0, it means that the display device 200 is currently connected to the second display.

[0099] In order to improve performance, the display device 200 can also create and maintain a database for storing the display identification identifier of the second display. When the second display is connected to the device interface 240, the display identification identifier of the second display is obtained in response to the plug-in and unplug signal generated when the second display is connected to the device interface 240, and the display identification identifier is stored in the database. When it is necessary to detect whether the display device 200 is connected to the second display, the display device 200 can call the database to detect the display identification identifier stored in the database. If the display identification identifier of the second display exists in the database, it means that the display device 200 is currently connected to the second display. If the display identification identifier of the second display does not exist in the database, it means that the display device 200 is not currently connected to the second display.

[0100] In some embodiments, a sidebar application is installed on the display device 200, and the sidebar application can be used as an independently running application to control the display content of the first display and the second display. Therefore, the database can be associated with the sidebar application for calling when the sidebar application performs corresponding logical processing. And when the display device 200 runs the sidebar application, it can monitor the access of the second display based on the sidebar application, and when the second display is connected to the display device 200 through the device interface 240, the display identification of the second display is stored in the database.

[0101] After determining that the current dual-screen mode is the extended mode and the second display is connected, the display device 200 can create a media playback window and control the second display to display the media playback window, play the target media through the media playback window, and display the media data on the second display.

[0102] For the creation and display of the media playback window, the display device 200 can create a media playback window based on the Activity component, and use ActivityOptions to specify the display on the second display. That is, the display device 200 can first obtain the display identification identifier (DisplayId) of the second display, and obtain the display identification object (Display object) based on the display identification identifier. Then create a media playback window Activity and create an activity custom object (ActivityOptions object), and define the Display object of the second display in the ActivityOptions object, wherein the ActivityOptions object is used to specify the display that displays the media playback window Activity. Finally, start the media playback window Activity, and pass the ActivityOptions object to specify the display on the second display.

[0103] S200: creating a playback control menu of the media playback window, and controlling the first display to display the playback control menu.

[0104] The play control menu includes controls for performing play control on the play process of the target media asset in the media play window, including but not limited to play, pause, fast forward, fast rewind, previous song, next song, menu, return and other functional controls, so as to perform play control on the play process of the target media asset. For the creation of the play control menu, the display device 200 can be implemented through the window management service (WindowManager), wherein the window management service is used to manage the state, properties, view addition, deletion, update, window order, message collection and processing of the window. The display device 200 can use the window management service (WindowManager) to draw the play control menu and control the first display to display the play control menu.

[0105] It should be noted that the present application does not limit the display style of the broadcast control menu. For example, in order to improve the display effect, the display device 200 can simulate remote control operation, set the display shape of the broadcast control menu and the layout of the controls in the broadcast control menu. For example, the display device can create Fig.10 For another example, in order to reduce the display area occupied by the play control menu, the display device 200 can set the display shape of the play control menu to a standard shape, such as the display device can create a Fig.11 The rectangular play control menu shown.

[0106] In some embodiments, for the media data played by the display device 200, the media types of the media data are different, and the corresponding function controls are also different. For example, the media types include video types, music types, picture types, etc. Among them, the video can be fast forward, fast rewind, play, pause, etc.; the music can be fast forward, fast rewind, play, pause, etc.; the picture does not support fast forward, fast rewind, play, pause, etc.

[0107] Therefore, the display device 200 can dynamically adjust the function controls in the play control menu according to the media type of the target media, so that the play control menu displayed on the first display is adapted to the media content played on the second display, so as to facilitate the user to perform interactive operations. That is, the display device 200 can respond to the play instruction indicating to play the target media on the second display, detect the media type of the target media, obtain the preset target control associated with the media type, and then create the play control menu based on the target control, and finally control the first display to display the play control menu.

[0108] For example, Figure 8 Taking the media asset list interface shown in the figure as an example, media asset 1 is a video media asset, and the user clicks the option icon of media asset 1 to input the play instruction of media asset 1 to the display device 200. In response to the play instruction, the display device 200 starts the video player Activity (media play window) and uses ActivityOptions to specify the display on the second display to play the media asset 1 on the second display. Then, according to the media asset type of media asset 1, the preset target control associated with the video type is obtained, and the play control menu including the above target control is created on the first display using WindowManager, as shown in FIG. Fig.10 The playback control menu shown in the figure includes function controls such as play, pause, fast forward, fast rewind, previous, next, menu, and return. The display effects of the first display and the second display can be presented as follows Fig.12 The display effect shown enables the user to perform playback control on the playback process of the media resource 1 in the second display through the playback control menu in the first display.

[0109] In some embodiments, in order to improve the display effect, the display device 200 can also plan the display position of the broadcast control menu. The display position of the broadcast control menu can be preset in the display device 200. When displaying the broadcast control menu, the preset display position can be obtained, and the first display can be controlled to display the broadcast control menu at the display position.

[0110] The present application does not limit the display position of the playback control menu, which can be set according to actual needs. For example, in order to reduce the area covered by the playback control menu, the display device 200 can display the playback control menu at the edge of the first user interface, such as the upper boundary, lower boundary, left boundary, right boundary, etc. of the first user interface. Fig.12 As shown, the play control menu is displayed in the upper right corner of the first user interface so that the user can perform interactive operations on the displayed content in the first user interface except the play control menu.

[0111] In some embodiments, after the display device 200 controls the first display to display the playback control menu, the user can also adjust the playback control menu through specific interactive actions, such as moving the playback control menu, adjusting the display size of the playback control menu, etc. Fig.12 Taking the broadcast control menu shown as an example, a touch panel (TP) is provided on the first display as a touch display. The user can press the area other than the function control button in the broadcast control menu on the first display with a finger, and then keep the finger in contact with the first display to input a sliding action to control the display device 200 to move the broadcast control menu. The user can also press the top corner area of ​​the broadcast control menu on the first display with a finger, and then keep the finger in contact with the first display and move it outward or inward to control the display device 200 to enlarge or reduce the broadcast control menu.

[0112] In order to improve the user experience, after the user adjusts the play control menu, the display device 200 can record the display area of ​​the play control menu. When the play control menu is displayed next time, it can be displayed according to the pre-recorded display area. That is, when the display device 200 controls the first display to display the play control menu, it can obtain the historical display area and control the first display to display the play control menu in the historical display area, wherein the historical display area is the display area of ​​the play control menu recorded when the first display last displayed the play control menu.

[0113] After the display device 200 displays the playback control menu, the user can use the function controls in the playback control menu to perform playback control on the target media asset played on the second display. In order to realize the playback control of the media playback window by the playback control menu, the playback control menu and the media playback window can realize playback control by obtaining a playback service processing instance (PlayerServiceHandler instance), including but not limited to playback control such as play, pause, fast forward, fast rewind, previous song, next song, menu, and return.

[0114] The PlayerServiceHandler instance is used to manage the state and control of media playback, and the instance can handle various playback control operations. When a user inputs a playback control instruction through the playback control menu, in response to the playback control instruction, the display device 200 can call the corresponding method in the PlayerServiceHandler instance to perform the playback control operation associated with the playback control instruction on the target media asset played in the media playback window.

[0115] The PlayerServiceHandler instance can also be used to monitor the status, including but not limited to obtaining the playback progress, video length, playback status, and current video name. Therefore, in order to improve the user experience, in some embodiments, the display device 200 can obtain the playback progress and total length of the target media asset based on the PlayerServiceHandler instance, draw a progress bar based on the playback progress and total length of the media asset, and control the first display to display the progress bar, so that the user can intuitively understand the playback progress of the current target media asset. For example, Fig.13 , Fig.14 As shown, the first display shows Fig.13 , Fig.14 In the rectangular play control menu shown, the display device 200 draws a progress bar in the play control menu according to the acquired play progress and the total length of the media resources. The progress bar indicates that the total length of the currently played media resources is 56 minutes and 10 seconds, and the current playback is up to 46 minutes and 50 seconds.

[0116] In some embodiments, the display device 200 can also obtain the playback status of the target media asset played in the media playback window based on the PlayerServiceHandler instance, including play, pause, etc., and set the display status of the playback control menu based on the playback status of the target media asset played in the media playback window, so as to synchronize the playback status of the media playback window in the second display with the playback control menu in the first display. For example, if the current playback status of the target media asset obtained by the display device 200 is the paused status, the first display is controlled to display the play button control in the playback control menu as follows: Fig.14 If the current playback state of the target media asset is the playback state, the first display is controlled to display the playback button control in the playback control menu as follows Fig.15 The play icon is shown.

[0117] In some embodiments, the display device 200 can also implement the playback control of the media playback window by the playback control menu based on the local broadcast. That is, the display device 200 can register the local broadcast in the playback control menu and the media playback window, such as registering the local broadcast through the LocalBro adcastManager class, so that the playback control menu and the media playback window can respond to specific events and perform corresponding operations. For example, when the user wants to pause the playback of the target media, he can click the play button control of the playback control menu on the first display. At this time, the control menu can monitor the click event and send a local broadcast indicating the pause of the playback of the target media to the media playback window. The media playback window receives the local broadcast and pauses the playback of the target media.

[0118] In some embodiments, when the display device 200 exits the playback of the target media asset, the playback control of the target media asset is not involved. Therefore, when the display device 200 exits the playback of the target media asset, the playback control menu is also canceled synchronously.

[0119] Among them, there are multiple ways to exit the playback of the target media. When the playback of the target media ends, or the user actively controls the display device 200 to exit the playback of the target media, or the display device 200 switches from the extended mode to the copy mode, or the second display is disconnected from the device interface 240, the display device 200 will be triggered to exit the playback of the target media. That is, the display device 200 can remove the media playback window in the second user interface of the second display and remove the playback control menu in the first user interface displayed on the first display when it is detected that the playback progress of the target media reaches the end progress, or when an exit instruction for stopping the playback of the target media is received, or when it is detected that the dual-screen mode is the copy mode, or when the plug-in signal generated when the second display is disconnected from the device interface 240 is detected.

[0120] That is, when the second display plays the media, the playback control menu is displayed on the first display to control the playback of the media played on the second display. When the playback of the target media ends, the user actively exits the playback of the target media, the dual-screen mode is switched to the copy mode, or the second display is disconnected (the user manually unplugs the HDMI cable), the playback control menu is canceled on the first display.

[0121] Based on the above playback control method. In some embodiments of the present application, a display device 200 is also provided, including: a first display, a device interface 240 and a controller 250. The first display is configured to display a first user interface; the device interface 240 is configured to connect to a second display, and the second display is used to display a second user interface; the controller 250 is configured to execute the following program steps:

[0122] S100: In response to a play instruction instructing to play a target media asset on a second display, create a media play window, and control the second display to display the media play window.

[0123] Among them, the media playback window is used to play the target media asset.

[0124] S200: creating a playback control menu of the media playback window, and controlling the first display to display the playback control menu.

[0125] The play control menu includes controls for performing play control on the play process of the target media asset.

[0126] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0127] It can be seen from the above technical solutions that the display device and playback control method provided in the above embodiments can create a media playback window and control the second display to display the media playback window in response to a playback instruction indicating that the target media asset is to be played on the second display. The media playback window is used to play the target media asset. A playback control menu for the media playback window is then created, and the first display is controlled to display the playback control menu. The playback control menu includes a control for performing playback control on the playback process of the target media asset. The method can control the playback of the media asset played on the second display by displaying the playback control menu on the first display during dual-screen display, thereby solving the problem that the second display is out of focus and the media asset displayed on the second display cannot be played back.

[0128] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0130] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, It is characterized in that include: A first display configured to display a first user interface; a device interface configured to connect to a second display, wherein the second display is used to display a second user interface; The controller is configured as: In response to a play instruction instructing to play the target media asset on the second display, creating a media play window, and controlling the second display to display the media play window, wherein the media play window is used to play the target media asset; A play control menu of the media play window is created, and the first display is controlled to display the play control menu, wherein the play control menu includes controls for performing play control on the play process of the target media asset.

2. The display device according to claim 1, It is characterized in that The controller controls the second display to display the media playback window, and is further configured to: Obtaining a display identification mark of the second display; Acquire a display identification object based on the display identification identifier; Creating an activity custom object, and defining the display identification object in the activity custom object, wherein the activity custom object is used to specify a display that displays the media playback window; Based on the active custom object, the second display is controlled to display the media playback window.

3. The display device according to claim 1, It is characterized in that The controller is also configured to: Calling a database, the database being used to store a display identification mark of the second display; If the display identification mark of the second display exists in the database, controlling the second display to display the media playback window; If the display identification mark of the second display does not exist in the database, the first display is controlled to display the media playback window.

4. The display device according to claim 1, It is characterized in that The controller is also configured to: Acquire a display list of the display device, wherein the display list includes display identification identifiers loaded by the display device; If there is a display identification identifier in the display list that is not the first attribute value, controlling the second display to display the media playback window; If only a display identification identifier with a first attribute value exists in the display list, the first display is controlled to display the media playback window.

5. The display device according to claim 1, It is characterized in that The controller is also configured to: Detecting a dual-screen mode of the display device; If the dual-screen mode is the extended mode, controlling the second display to display the media playback window; If the dual-screen mode is a copy mode, the first display is controlled to display the media playback window.

6. The display device according to claim 1, It is characterized in that The controller is also configured to: Receiving a broadcast control instruction input by a user through the broadcast control menu; In response to the play control instruction, calling the play service processing instance; Based on the playback service processing instance, the playback control associated with the playback control instruction is executed on the target media asset played in the media playback window.

7. The display device according to claim 1, It is characterized in that The controller executes the creation of the play control menu and is further configured to: Detecting the media asset type of the target media asset, where the media asset type is a video type, a music type, or a picture type; Obtaining a preset target control associated with the media asset type; Based on the target control, the play control menu is created.

8. The display device according to claim 1, It is characterized in that The controller is also configured to: When it is detected that the playback progress of the target media asset reaches the end progress, or when an exit instruction for stopping the playback of the target media asset is received, or when it is detected that the dual-screen mode is a copy mode, or when a plug-in signal generated when the second display is disconnected from the device interface is detected, the playback control menu is removed from the first user interface displayed on the first display.

9. The display device according to claim 1, It is characterized in that The controller controls the first display to display the playback control menu, and is further configured to: Acquire a historical display area, where the historical display area is a display area of ​​the play control menu when the first display last displayed the play control menu; Control the first display to display the play control menu in the history display area.

10. A playback control method, It is characterized in that Applied to a display device, the display device includes a first display, a device interface and a controller, the device interface is connected to a second display, the first display is configured to display a first user interface, and the second display is used to display a second user interface, the method includes: In response to a play instruction instructing to play the target media asset on the second display, creating a media play window, and controlling the second display to display the media play window, wherein the media play window is used to play the target media asset; A play control menu of the media play window is created, and the first display is controlled to display the play control menu, wherein the play control menu includes controls for performing play control on the play process of the target media asset.

Images