Display device and broadcast control view display method

Abstract

The invention provides a display device and a broadcast control view display method, and the method can respond to a connection event for establishing a screen projection connection relationship between the display device and a terminal device, and obtains a media asset type of screen projection data. And extracting broadcast control data associated with the media asset type from the broadcast control list, and creating a broadcast control view according to the broadcast control data. And setting the display position of the broadcast control view according to the display area of the screen projection window, and finally, controlling the display to display the broadcast control view at the display position. Wherein the broadcast control list comprises broadcast control data of a preset broadcast control view, and the broadcast control view is a control used for executing broadcast control on the projection screen data in the projection screen interface. According to the method, the display of the broadcast control function control can be dynamically adjusted according to the media asset type and the screen projection display area, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a broadcast control view display method. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, communication terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] The display device can also establish a screen projection connection relationship with the terminal device through communication methods such as wireless LAN, and obtain the screen projection data of the terminal device based on a specific screen projection protocol to display the projection screen of the terminal device through the display device. Some display devices can establish communication connections with multiple terminal devices at the same time, and receive screen projection data from multiple terminal devices to form a multi-channel screen projection scenario. However, due to the diversity of screen projection content between display devices and terminal devices, the displayed broadcast control function controls may not be compatible with the screen projection content, reducing the user experience. Summary of the invention

[0004] The present application provides a display device and a broadcast control view display method to solve the problem that the displayed broadcast control function controls are not compatible with the projected content.

[0005] In a first aspect, the present application provides a display device, including a display, a communicator, and a controller. The display is configured to display a screen projection interface, the screen projection interface includes a screen projection window, and the screen projection window is used to display a screen projection screen of a terminal device. The communicator is configured to establish a screen projection connection with the terminal device. The controller is configured to perform the following program steps:

[0006] In response to a connection event of establishing a screen projection connection relationship between the display device and the terminal device, obtaining a media asset type of the screen projection data;

[0007] Extracting broadcast control data associated with the media asset type from the broadcast control list, and creating a broadcast control view according to the broadcast control data, wherein the broadcast control list includes broadcast control data of a preset broadcast control view, and the broadcast control view is a control in the projection interface for performing playback control on the projection data;

[0008] Setting the display position of the playback control view according to the display area of ​​the screen projection window;

[0009] The display is controlled to display the play control view at the display position.

[0010] In a second aspect, the present application further provides a broadcast control view display method, which is applied to the above-mentioned display device, and the method includes:

[0011] In response to a connection event of establishing a screen projection connection relationship between the display device and the terminal device, obtaining a media asset type of the screen projection data;

[0012] Extracting broadcast control data associated with the media asset type from the broadcast control list, and creating a broadcast control view according to the broadcast control data, wherein the broadcast control list includes broadcast control data of a preset broadcast control view, and the broadcast control view is a control in the projection interface for performing playback control on the projection data;

[0013] Setting the display position of the playback control view according to the display area of ​​the screen projection window;

[0014] The display is controlled to display the play control view at the display position.

[0015] It can be seen from the above technical solution that the present application provides a display device and a broadcast control view display method, which can obtain the media type of the projection data in response to a connection event that establishes a projection connection relationship between the display device and the terminal device. Extract the broadcast control data associated with the media type in the broadcast control list, and create a broadcast control view based on the broadcast control data. Then set the display position of the broadcast control view according to the display area of ​​the projection window, and finally control the display to display the broadcast control view at the display position. Among them, the broadcast control list includes the broadcast control data of the preset broadcast control view, and the broadcast control view is a control in the projection interface for performing playback control on the projection data. The method can dynamically adjust the display of the broadcast control function control according to the media type and the projection display area to improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0018] Figure 2 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0019] Figure 3 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0020] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0021] Figure 5 A schematic diagram of mirror screen projection provided in some embodiments of the present application;

[0022] Figure 6 A schematic diagram of push screen projection provided in some embodiments of the present application;

[0023] Figure 7 A schematic diagram of a multi-screen projection interface provided in some embodiments of the present application;

[0024] Figure 8 A schematic diagram of a functional mode of a display device provided in some embodiments of the present application;

[0025] Fig. 9 A schematic diagram of a guide page provided for some embodiments of the present application;

[0026] Fig.10 A schematic diagram of the interface display when casting a screen in some embodiments of the present application;

[0027] Fig.11 A schematic diagram of the effect of dynamically adjusting the display area provided in some embodiments of the present application;

[0028] Fig.12 A flowchart of a method for displaying a broadcast control view provided in some embodiments of the present application;

[0029] Fig.13 A schematic diagram of the effects of six playback control button controls provided in some embodiments of the present application;

[0030] Fig.14 A schematic diagram of the effect of dynamically adjusting the number of play control button controls provided in some embodiments of the present application;

[0031] Fig.15 A schematic diagram of the effects of three playback control button controls provided in some embodiments of the present application;

[0032] Fig.16 A schematic diagram of the effects of two playback control button controls provided in some embodiments of the present application;

[0033] Fig.17 A schematic diagram of the effect of a play control button control provided in some embodiments of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0035] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0036] It should be understood that the terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0037] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0038] The display device provided in the embodiments of the present application may have various implementation forms, for example, it may be a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figure 1 and Figure 3 This is a specific implementation of the display device of the present application.

[0039] Figure 1 FIG. 1 is a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control apparatus 100 .

[0040] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wired. The user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, etc.

[0041] In some embodiments, a smart device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop computer, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the smart device.

[0042] In some embodiments, the display device 200 may not use the above-mentioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0043] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0044] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be one cluster or multiple clusters, and may include one or more types of servers.

[0045] Figure 2 Schematically shows a block diagram of a configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive input operation instructions from the user, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200.

[0046] like Figure 3 As shown, the display device 200 includes at least one of a tuner and demodulator 210 , a communicator 220 , a detector 230 , a device interface 240 , a controller 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a communication device interface 280 .

[0047] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output.

[0048] The display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from the controller, and display video content, image content, and menu control interface components and user control UI interface.

[0049] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0050] The audio output interface 270 is used to output the audio signal to other devices to play the audio. In some embodiments, the display device 200 has a built-in speaker, and the audio output interface 270 can input the audio signal to the speaker to play the audio through the speaker. In other embodiments, the display device 200 can be connected to an external audio device such as an amplifier, a speaker, or a loudspeaker through the audio output interface 270, and then the audio signal can be input to the external audio device to play the audio through the external audio device.

[0051] The audio output interface 270 may include, but is not limited to, any one or more of the following interfaces: an RCA port, a butterfly clip port, an SPDIF port, and a headphone port, etc. It may also be a composite output interface formed by the above multiple interfaces.

[0052] The communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0053] The communication device interface 280 may be used to receive a control signal from the control device 100 (eg, an infrared remote controller, etc.).

[0054] The detector 230 is used to collect signals from the external environment or the external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0055] The device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0056] The tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0057] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0058] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0059] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM (Random Access Memory, RAM), a ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0060] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

[0061] "User interface" is the medium interface for interaction and information exchange between applications or operating systems and users. It realizes the conversion between the internal form of information and the form acceptable to users. The commonly used form of user interface is the Graphical User Interface (GUI), which refers to the user interface related to computer operation displayed in a graphical way. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.

[0062] like Figure 4As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0063] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0064] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0065] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (NotificationManager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0066] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0067] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0068] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0069] After establishing a communication connection relationship with the terminal device 500, the display device 200 can establish a screen projection connection channel, and transmit the projection data based on the screen projection connection, so that the display device 200 can display the projection screen of the terminal device 500. In some embodiments, in order to establish a screen projection connection relationship, the user can send a screen projection connection request through the terminal device 500, and send the screen projection connection request to the display device 200 through the WiFi network. The display device 200 then completes the transmission protocol configuration according to the screen projection connection request, thereby establishing a transmission channel for the projection data with the terminal device 500.

[0070] The display device 200 can also achieve a screen projection connection with the terminal device 500 through different connection methods. For example, when the display device 200 and the terminal device 500 are connected to the same wireless local area network, a screen projection connection can be established based on a WiFi network. For another example, when both the display device 200 and the terminal device 500 are provided with NFC (Near Field Communication) components, a screen projection connection relationship can be established through the NFC component. Obviously, other wired or wireless connection methods can also be used between the display device 200 and the terminal device 500 to establish a screen projection connection relationship, such as RF radio frequency connection, infrared connection, cellular network, and other connection methods that can be associated with those skilled in the art based on the connection methods provided in the above embodiments.

[0071] After the screen projection connection is established, the display device 200 can receive the screen projection related data of the terminal device 500 through the screen projection data channel, so as to play the screen projection related content in the display device 200. Under different screen projection data transmission protocols, the screen projection data obtained by the display device 200 is in different forms. For example, the display device 200 and the terminal device 500 can establish a screen projection connection through the Digital Living Network Alliance (DLNA) protocol, Miracast protocol, AirPlay push protocol, WiDi screen projection, NFC mirror screen projection, LeTV screen projection and other protocols.

[0072] Among them, the DLNA protocol is a protocol for sharing media resources between devices. The protocol uses network technology (wired or wireless) to interconnect various devices and exchange data through standard protocols, so that media resources can be shared between devices. The DLNA protocol is based on the UPnP protocol, which enables devices to discover and control each other and achieve seamless transmission and sharing of media resources. The DLNA protocol supports a variety of devices, including TVs, speakers, mobile phones, computers, etc., allowing users to seamlessly switch and share media content between different devices.

[0073] Miracast protocol is a wireless display standard based on Wi-Fi direct connection and developed by the Wi-Fi Alliance. Miracast screen projection protocol is applicable to Android devices, that is, Android devices can share video images through Miracast screen projection. Smart devices with Android 5.0 or above have built-in Miracast protocol, so you can directly wirelessly project the screen through the WiFi P2P function between the projection device and the WiFi network card of the receiving device without installing any software. In addition, Miracast screen projection also supports synchronous transmission of audio and video.

[0074] Airplay protocol is a wireless playback technology launched by Apple, which is applicable to the wireless screen projection protocol of iOS and Mac systems. Apple devices have built-in Airplay screen projection function. When used with a screen projection device that supports AirPlay server service, the effect is clear and stable. The protocol comes with a screen projection code function. However, it is necessary to ensure that the Apple device and the screen projection device are used in the same LAN (same network segment), and cannot be used across network segments or VLANs.

[0075] WiDi screen projection is a wireless playback technology led by Intel, and is also based on WiFi Direct technology. Windows 8 / 10 / 11 laptops natively support WiDi screen projection, which can be directly projected without installing any software, and supports USB reverse control function.

[0076] In some embodiments, after the display device 200 establishes a screen projection connection with the terminal device 500, the screen projection data can be transmitted through the mirror mode and the push mode. Among them, the mirror mode means that the display device 200 synchronously displays the screen content displayed by the terminal device 500. For example, Figure 5As shown, the display device 200 displays the main interface of the terminal device 500, and when the user performs an interactive operation on the terminal device 500, resulting in a change in the content of the main interface, the content displayed by the display device 200 also changes. To this end, the terminal device 500 can generate a screen projection data stream in the form of a video based on the displayed content, and send the screen projection data stream to the display device 200 for display.

[0077] The push mode means that the display device 200 accesses the network link according to the network link pushed by the terminal device 500 to obtain the corresponding projection screen. Figure 6 As shown, when the terminal device 500 plays the video media, a screen projection control is provided in the upper right corner of the play interface. After the user clicks the screen projection control and selects the display device 200 as the device for displaying the screen projection image, the terminal device 500 can send the URL address of the currently played video media to the display device 200. After receiving the URL address, the display device 200 accesses the URL address to obtain the video media data and plays the video media.

[0078] Among the screen projection protocols supported by the display device 200 and the terminal device 500, some protocols are based on the mirror mode, and some protocols are based on the push mode. For example, under the DLNA protocol, the display device 200 can obtain the link address of a resource in the terminal device 500 through the screen projection data channel, and obtain the media content data by accessing the link address, thereby playing the media data. At the same time, the display device 200 can also obtain control instructions sent by the terminal device 500 through the screen projection data channel when playing media data, such as fast forward, pause, stop screen projection, etc. For example, after establishing a connection through the communication protocol, the terminal device 500 such as a mobile phone and the display device 200 will negotiate the device functions and network conditions to select a suitable audio and video transmission format. After establishing the online protocol (Session) of the audio and video streaming, a series of real-time streaming protocol (RTSP) control commands are used to control the playback and termination of the audio and video streaming.

[0079] Under other screen projection connection protocols such as Miracast, Airplay, NFC mirroring, and LeTV, the display device 200 can directly obtain the video data stream from the terminal device 500. The video data stream can be the video data stream corresponding to the final display screen of the display unit of the terminal device 500, that is, the video data stream includes the media content screen, UI interface, and other video screen content that can be displayed in the display unit of the terminal device 500. The video data stream can also be only the video data that can be obtained in the terminal device 500, such as the media video data being played. At this time, the terminal device 500 can be used as a signal source for the display device 200, and the display device 200 can continuously obtain video data through the screen projection data channel, and display it in real time on the display 260, so as to achieve the effect of synchronous display with the terminal device 500.

[0080] It should be noted that the above-mentioned screen projection protocol can not only obtain the video data stream in the terminal device 500, but also obtain the audio data stream in the terminal device 500, so that the video data and audio data can be sent to the display device 200 synchronously, so that the display device 200 can play the video and audio synchronously.

[0081] The terminal device 500 can obtain different projection data according to different projection protocols. For example, for system applications or direct projection at the system level, the terminal device 500 can use Miracast projection, Airplay projection or WiDi projection protocol to establish a projection data channel. Since the above-mentioned Miracast projection protocol and Airplay projection protocol require system-level application permissions, the Miracast projection protocol and Airplay projection protocol are only applicable to system applications, or the display device 200 and the terminal device 500 of the same manufacturer directly establish a projection connection through the built-in rules of the operating system.

[0082] The display device 200 can establish a screen projection connection with the terminal device 500 based on any of the screen projection methods provided in the above embodiments. In some embodiments, the display device 200 can also establish a screen projection connection relationship with multiple communication terminals through one or more of the above screen projection methods. For example, the display device 200 can simultaneously establish a screen projection connection with the first terminal device 510 and the second terminal device 520, and divide the display area in the display screen to display the projection screen shared by the first terminal device 510 and the second terminal device 520 respectively.

[0083] In order to display the projection screen shared by multiple terminal devices 500 on the display device 200, in some embodiments, when the display device 200 establishes a projection connection with multiple terminal devices 500, the display device 200 can display a multi-channel projection interface, which can include multiple projection windows. The projection window is used to display the projection screen of the corresponding terminal device 500 under a projection connection channel. For example, Figure 7 As shown, the multi-channel screen projection interface may include two screen projection windows, namely a first screen projection window and a second screen projection window. The first screen projection window is used to display the projection screen of the first terminal device 510; the second screen projection window is used to display the projection screen of the second terminal device 520.

[0084] It should be noted that the projection window in the multi-channel projection interface may not be strongly bound to the terminal device 500, that is, in some embodiments, the display content of the projection window may be determined according to the order in which the terminal device 500 establishes a projection connection with the display device 200. For example, when the first terminal device 510 establishes a projection connection with the display device 200 before the second terminal device 520, the first projection window on the left side of the multi-channel projection interface displays the projection screen of the first terminal device 510. When the first terminal device 510 establishes a projection connection with the display device 200 later than the second terminal device 520, the first projection window on the left side of the multi-channel projection interface displays the projection screen of the second terminal device 520.

[0085] In order to obtain a better picture display effect and improve the coordination of the multi-channel projection interface, in some embodiments, the display device 200 can also set the layout of the projection window. Figure 8 As shown, the display device 200 can have multiple functional modules built in, including: a data management module, a message processing module, a service management module, an algorithm processing module, a View management module, an application interaction processing module, a location management module, a replacement management module, a broadcast control management module, a time management module, a focus management module, and a screen projection management module, etc.

[0086] Among them, the data management module mainly manages all message data involved in the current template and updates the data information synchronously. The message processing module mainly transmits related messages between different modules, such as transmitting data changes to the View layer for refresh display, or synchronizing View changes to the data management module for change processing of broadcast control data, etc. The service management module is responsible for opening and closing the services of the entire application and some customized processing, such as starting specific services in specific scenarios. The algorithm processing module is responsible for all layout position calculations of the entire application. The View management module is mainly responsible for all UI displays of the entire template application, and receives data to refresh the View display.

[0087] The application interaction processing module is responsible for the processing of interactions with external applications. For example, any entry to start the current template application needs to interact with this module, and different customized management is performed according to the differences in the startup entries. The position management module is responsible for the rules for displaying the projection position in the left and right windows, such as on the left or on the right. The replacement management module is responsible for the replacement rules when the number of projection paths exceeds the maximum display number, such as how to replace when there is focus or no focus. The broadcast control management module is responsible for the broadcast control processing of different media types in the projection, such as fast forward, rewind, play pause, etc. The time management module is responsible for the focus display timing processing in the projection, whether it is the large frame focus or the broadcast control position focus. The focus management module is responsible for the focus display and button processing in the entire application. The projection management module is responsible for the projection protocol interaction and the client list management and window display content status information processing.

[0088] It should be noted that the multi-channel screen projection interface can realize screen display based on the multi-channel screen projection viewing application. The multi-channel screen projection viewing application is an application that can display multi-protocol multi-channel mirror projection and push content, and is an upper-level application inside the display device 200. The framework layer of the display device 200 can provide a Java interface for docking with hardware, etc. The multi-channel screen projection viewing application realizes docking with the lower layer and obtains information through the Java interface of the Framework layer. Other National, Runtime, HAL, and Linux kernel layers are all standard layers of the Android architecture, and each layer interacts according to the business process.

[0089] The display device 200 first starts the application through the service management module and performs some initial preparations, such as starting the monitoring mirror or pushing the corresponding protocol service; the View management module initializes the View displayed by the application layer; the data management module creates some data for the data processing module; the algorithm processing module performs algorithmic layout calculations on the projection content, and the position management module is responsible for positioning the left and right positions of the projection content.

[0090] The service management module can start the screen projection application and register the Session during the process of establishing the screen projection connection; Usession Connection Listener, Usession Prepare Listener, Usession DisplayListener and Usession Event Listener. Among them, Session represents a conversation between the client and the server, which can save the client's information. Usession Connection Listener is used to monitor the connection and disconnection events of the Session. Usession Prepare Listener interacts with the screen projection preparation of the Client. Usession DisplayListener monitors the changes in playback information and status and the status changes of the Client. Usession Event Listener processes information such as Client control.

[0091] After the initialization of View is completed, the display device 200 will display the default guide page when entering the application. The guide page will display some device information, such as the device name of the current display device 200 and the corresponding network information. Then, the guide page will be displayed in equal proportion to the left and right by controlling the standard control TextView and ImageView of the system and the corresponding Layout, such as Fig. 9 shown.

[0092] After displaying the guide page, the display device 200 can monitor and receive the screen projection connection instruction sent by the terminal device 500, and establish a screen projection connection channel according to the screen projection connection instruction, so as to display it in the screen projection window. For example, the screen projection management module is always ready for the access of a new Session. When Client1 is ready to start the connection, the screen projection management module monitors the onConnect connection of the new Session through UsessionConnectionListener, and then creates the playback control SurfaceView / TextureView. After the creation is successful, the position management module is started to set and manage the display position of the current Session. The position management module processes and obtains the position. The first screen projection is displayed on the left by default. The mViewPosition of Client1 is marked as 1, where 1 represents the left side and 2 represents the right side. It can also be represented by other parameters.

[0093] To facilitate the management of View and Session, in some embodiments, a management object MultiCastPlayView can be set, wherein the management object MulfiCastPlayView can be associated with information such as Session, SurfaceView / TextureView, and mViewPosition. For example, for the first terminal device 510, the data corresponding to its Client1 can be represented as: Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1.

[0094] When the display device 200 prepares the View as the server, it can notify the terminal device 500 as the mobile terminal to record or send the video stream through UsesessionPrepareListener. When the terminal device 500 is ready to send video information, the display device 200 knows that it will be cast through onFirstFrame() of UsesessionDisplayListener, so it can get the current width and height of Client1's video from the Session, determine the direction of the current video stream through the width and height, and then calculate the current position and size to be displayed through the algorithm.

[0095] That is, the first projection window and the second projection window can also present corresponding shapes according to the size of the displayed projection screen. For example, when the first projection screen and the second projection screen are both 1920×1080 horizontal screens, the display device 200 can present the first projection window and the second projection window with a size of 960×540 in the multi-channel projection interface.

[0096] In some embodiments, in order to distinguish the display range of the projection area, a certain blank area may be left in the first projection window and the second projection window, the projection window and the guide area. For example, the blank area is 12 pixels, and the resolution of the display device 200 is 1920×1080. Fig.10 As shown, when the first terminal device 510 (Client 1) is displayed in horizontal format, the size of the projection screen in the corresponding projection video stream is 1920×1080, and the guide page display size is 954×538. At this time, the ratio of the video stream and the ratio of the guide video stream on the right are close to 16:9 and both are horizontal. It can be calculated through the algorithm that the left and right sides can be displayed in equal proportion, that is, they can be displayed with equal width and height, that is, the width of the video stream and the guide page are both 954, and the height is about 538.

[0097] Since the terminal device 500 may be placed in different states according to different usage modes, for example, a terminal device 500 such as a smart phone may be in a portrait state and a landscape state during use. The portrait state refers to a state in which the width of the displayed image is less than the height, and the landscape state refers to a state in which the width of the displayed image is greater than the height. In different placement states, the terminal device 500 may present different screen layouts.

[0098] Based on this, in some embodiments, when the display device 200 displays the projection screens of multiple terminal devices 500 by area division, each display area will present different screen ratios due to the different placement states of the terminal devices 500. For example, the first terminal device 510 is in a horizontal screen state, the first projection screen size is 1920×1080, and the second terminal device 520 is in a vertical screen state, and the second projection screen size is 512×1088. Then the first projection screen window can be 954×538, and the second projection screen window can be 508×1080.

[0099] In order to make full use of the display space of the multi-way projection interface and reduce large black screen areas, in some embodiments, when the sizes of the first projection screen and the second projection screen are inconsistent, the display device 200 can set the width or height of the first projection window and the second projection window to be consistent, and scale the projection windows according to the width or height.

[0100] For example, when the first terminal device 510 is in portrait mode, the size of the first projection screen is 512×1088; the second terminal device 520 is in landscape mode, the size of the second projection screen is 1920×1080, and the display resolution of the display device 200 is 1920×1080, the default size of the first projection window and the second projection window (i.e., the guide page display size) is 954×538. At this time, the video stream ratio is different from the right guide video stream ratio and the direction is different. It can be calculated through the algorithm that the left video stream display height is about 848 and the width is about 400. Excluding the blank area, the remaining width is the guide page width, and the height is consistent with the video stream.

[0101] To this end, the display device 200 can set the parameters of the first projection window, namely Client 1: width1 (width), height1 (height); Guide (guide page): width_guide, height_guide. The height of the projection screen video stream from the upper edge of the screen is set to top, because height1 and height_guide are ultimately the same height, and can be replaced by the same variable height. The video stream ratio ratio1 and the guide page ratio ratio_guide are both known constants. Through the following combined calculations, width1, top, height, and width_guide can be calculated; that is, width1+width_guide=1920; top*2+height=1080; width1 / height=ratio1; width_guide / height=ratio_guide.

[0102] Based on the above algorithm, the algorithm processing module of the display device 200 can obtain the corresponding width and height position of the content displayed on the left and right sides. That is, the width and height of Client1 can be calculated to be 848 and 400, and the corresponding position coordinates are (0, 116) from the upper left corner, and the guide page coordinates are (412, 116).

[0103] After the above position calculation is completed, the data management module of the display device 200 can sort and store, and then transmit the data, including: Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1, Position: (0, 116), Width: 400, Height: 848; Session: View: layout, MViewPosition: Position: (412, 116), Width: 1508, Height: 848. After receiving the data information, the View management module of the display device 200 sets the display parameters through LayoutParams, including the position from the upper boundary and the left boundary (i.e., coordinates), as well as the corresponding display width and height, and then displays the position through the setLayoutParams method. That is:

[0104] FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width, height, 0); lp1.width=400; lp1.height=848; lp1.leftMargin=0; lp1.topMargin=116; mPlayerView, setLayoutParams(lp1);

[0105] FrameLayout.LayoutParamslp1=newFrameLayout.LayoutParams(width,height,0);lp1.width=1508;lp1.height=848;lp1.leftMargin=412;lp1.topMargin=116;mPlayerView.setLayoutParams(lp1).

[0106] By setting the display position of the projection SurfaceView / TextureView and the display position of the guide Layout respectively, you can get the following Fig.11 The display effect of the multi-channel screen projection interface is shown.

[0107] In some embodiments, when the display device 200 displays the multi-channel screen projection interface, it can also display the playback control function controls in the multi-channel screen projection interface, such as full screen, fast forward, fast rewind, play / pause, mute / unmute, close, etc. The user can perform interactive operations based on the playback control function controls to input control instructions and control the display device 200 to perform interactive responses.

[0108] In some embodiments, the display device 200 can display a broadcast control function control adapted to the screen projection content of the terminal device 500 by executing a broadcast control view display method. In order to meet the implementation of the broadcast control view display method, the display device 200 should at least include a display 260, a communicator 220 and a controller 250.

[0109] The display 260 is configured to display a projection interface, wherein the projection interface includes a projection window, and the projection window is used to display the projection screen of the terminal device 500. The communicator 220 is configured to establish a projection connection with the terminal device 500. Fig.12 As shown, the controller 250 is configured to execute program steps corresponding to the broadcast control view display method, which specifically includes the following contents:

[0110] When the display device 200 establishes a projection connection with the terminal device 500, the display device 200 and the terminal device 500 will communicate and negotiate to complete the transmission protocol configuration and determine the content and format to be transmitted. Therefore, the display device 200 can obtain the configuration information when the display device and the terminal device establish a projection connection, and read the media type of the projection data of the terminal device 500 in the configuration information, such as VIDEO_URL (video URL)\MIRROR_STREAM (mirror stream)\MUSIC (music)\PIC (photo) and other media types.

[0111] After obtaining the media type of the projection data, a broadcast control view associated with the media type can be created, that is, a broadcast control function control in the projection interface of the display device 200 for performing playback control on the projection data, so that the displayed broadcast control function control is adapted to the projection content of the terminal device 500 to facilitate the user to perform interactive operations.

[0112] In order to create a broadcast control view associated with a media asset type, the display device 200 may pre-create and maintain a broadcast control list, in which broadcast control data of various types of broadcast control views may be preset, such as broadcast control data of broadcast control function controls such as full screen, fast forward, fast rewind, play / pause, mute / unmute, and close. Among them, the broadcast control data in the broadcast control list includes a status identifier, a broadcast control type, and a resource identifier. The status identifier is used to indicate the functional state of the broadcast control view, the broadcast control type is used to indicate the processing operation associated with the broadcast control view, and the resource identifier is used to indicate the functional module associated with the broadcast control view. For example, the broadcast control list may be a List list of DoublePushControlBean, and the corresponding data may be expressed as: DoublePushControlListBean[List<DoublePushControlBean>=××,Status=××]. The broadcast control data of each broadcast control function control may be expressed as: DoublePushControlBean[mStatus(status identifier)=××,mType(broadcast control type)=××,mResId(resource identifier)=××].

[0113] In this regard, the display device 200 can respond to the connection event of establishing the screen projection connection relationship between the display device 200 and the terminal device 500, and obtain the media asset type of the screen projection data in the configuration information when the display device 200 and the terminal device 500 establish the screen projection connection. Then call the preset broadcast control list, extract the broadcast control data associated with the media asset type in the broadcast control list, and create a broadcast control view according to the broadcast control data, and finally control the display 260 to display the broadcast control view.

[0114] In some embodiments, in order to facilitate the subsequent processing of the broadcast control data, the display device 200 can generate a broadcast control sub-list based on the broadcast control data associated with the media asset type in the broadcast control list, and create a broadcast control view based on the broadcast control sub-list. When the broadcast control view is subsequently subjected to relevant logical processing, only the broadcast control data in the broadcast control sub-list needs to be processed. In other words, the broadcast control list is a total list of broadcast control data containing all types of broadcast control views, and the broadcast control sub-list is a sub-list of broadcast control data containing the broadcast control view related to the current screen projection connection of the terminal device 500.

[0115] The generation of the broadcast control sublist may include multiple methods. After the display device 200 obtains the media asset type of the projection data, it calls the broadcast control list, copies the broadcast control list, and deletes the broadcast control data not associated with the media asset type in the broadcast control list to generate a broadcast control sublist. The display device 200 may also create a broadcast control sublist, and add the broadcast control data associated with the media asset type in the broadcast control list to the broadcast control sublist.

[0116] For example, when the projection management module listens to the onConnect connection of the new Session through UsessionConnectionListener, the media type of the current Client can be obtained from the Session, such as VIDEO_URL\MIRROR_STREAM\MUSIC\PIC, etc. For example, the display device 200 currently displays the media data obtained based on the VIDEO_URL address pushed by the terminal device 500, then full screen, fast forward, fast rewind, play / pause, mute / unmute, close and other operations can be performed on the display device 200. At this time, broadcast control data can be added to this Session. For example, the broadcast control sublist includes 6 broadcast control function controls associated with the media type, and the corresponding data format is:

[0117] Full screen button control: DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132];

[0118] Rewind button control: DoublePushControlBean[mStatus=true, mType=rewind, mResId=2131231110];

[0119] Play button control: DoublePushControlBean[mStatus=true, mType=playstatus, mResId=2131231149];

[0120] Fast forward button control: DoublePushControlBean[mStatus=true, mType=forward, mResId=2131231111];

[0121] Mute button control: DoublePushControlBean[mStatus=true, mType=mutestatus, mResId=2131231152];

[0122] Close button control: DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121].

[0123] In this way, the display device 200 can create and display the play control function controls related to the media asset type of the current Client according to the play control sub-list, so that the user can perform interactive operations.

[0124] In some embodiments, when multiple screens are projected, in order to facilitate users to distinguish the broadcast control function controls corresponding to each screen projection, the display device 200 can dynamically adjust the broadcast control function controls according to the display area of ​​the projection window in the multi-channel projection interface, so that the display position of the broadcast control function controls is adapted to the display area of ​​the projection window. That is, the display device 200 can set the display position of the broadcast control view according to the display area of ​​the projection window, and control the display 260 to display the broadcast control view at the display position.

[0125] The view size of the broadcast control view can be preset according to actual needs. For example, the broadcast control list includes full screen, fast forward, fast rewind, play / pause, mute / unmute, close and other broadcast control function controls. The size of each broadcast control function control can be set to 60×60, and the horizontal spacing of the broadcast control function controls is 24. Combined with the left and right margins of the broadcast control list, the minimum display width of six broadcast control function controls is 528, the minimum display width of three broadcast control function controls is 276, the minimum display width of two broadcast control function controls is 192, and the minimum display width of one broadcast control function control is 108. The same applies to other numbers.

[0126] In order to improve the display effect, the broadcast control view can be displayed based on the horizontal / vertical arrangement. The display device 200 can plan the display position of the broadcast control view according to the display area of ​​the projection window, and can also calculate the maximum number of broadcast control views that can be displayed when arranged horizontally / vertically according to the display area size of the projection window and the preset view size of the broadcast control view. Therefore, the display device 200 can obtain the area size of the display area of ​​the projection window, and obtain the preset view size of the broadcast control view. The display number of the playback view is calculated based on the area size and the view size, and the broadcast control data associated with the media asset type is extracted from the broadcast control list according to the display number.

[0127] When the broadcast control function control is displayed in horizontal version, the display device 200 can obtain the area width of the display area of ​​the projection window and the total width of the broadcast control bar, wherein the broadcast control bar is arranged according to the preset spacing according to the broadcast control view associated with the media asset type. The area width of the display area of ​​the projection window is compared with the total width of the broadcast control bar. If the area width is greater than or equal to the total width, it means that the broadcast control view associated with the media asset type can be fully displayed when the horizontal version is displayed, then the broadcast control data associated with the media asset type is extracted from the broadcast control list, and the broadcast control view is created and displayed according to the broadcast control data.

[0128] If the area width is smaller than the total width, it means that the broadcast control views associated with the media asset type cannot be fully displayed in the horizontal version. In this case, the number of broadcast control views that can be displayed is calculated based on the area width and the total width. According to the number of displays, the broadcast control data associated with the media asset type is extracted from the broadcast control list, and the broadcast control views of that number are created and displayed based on the broadcast control data.

[0129] It is understandable that when the broadcast control function control is displayed in horizontal version, the maximum number of broadcast control views that can be displayed when arranged horizontally can be calculated based on the area width of the projection window display area. When the broadcast control function control is displayed in vertical version, the maximum number of broadcast control views that can be displayed when arranged vertically can also be calculated based on the area height of the projection window display area.

[0130] It should be noted that the display method of the broadcast control function controls provided in the above embodiment is only an example. The broadcast control function controls of the display device 200 are not limited to the above-mentioned horizontal / vertical arrangement, but can also be displayed in other forms. For example, they can be arranged and displayed in a rectangular array or a circular array, or by edge sliding. However, the basic display principles are universal.

[0131] In some embodiments, in order to meet the actual needs of users, a weight value can be set for the broadcast control view in the broadcast control list. When the display device 200 extracts the broadcast control data associated with the media asset type in the broadcast control list according to the display quantity, the weight value of the broadcast control data in the broadcast control list can be traversed. The broadcast control data in the broadcast control list are sorted in descending order of the weight value. The broadcast control data associated with the media asset type of the first display quantity is extracted from the sorted broadcast control list.

[0132] That is, the priority of the control views in the control list is set based on the weight value. When all control views cannot be displayed, the control view with a higher weight value is selected for display according to the weight value. For example, in the order of weight value from high to low, the control views in the control list are sorted as follows: full screen, off, mute / unmute, play / pause, fast forward, and fast rewind. When only three control views can be displayed, the full screen, off, mute / unmute control views can be displayed.

[0133] For example, taking the VIDEO_URL address pushed as an example, in order to display the above-mentioned broadcast control function control, after the data management module obtains the display size of the projection window, the View management module performs typesetting and display as follows: Fig.13 The display effect shown is that the left projection window displays the media resource screen obtained based on the pushed URL address, which can be controlled by full screen, fast forward, fast rewind, play / pause, mute / unmute, close, etc. The algorithm processing module can calculate that the size of the left projection window is 1508×848, and the size of the right projection window is 400×848. The following data results are obtained:

[0134] Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1, Position: (0, 116), Width: 1508, Height: 848, mDoublePushControlListBean:…

[0135] Session: SessionClient2, View: SurfaceView / TextureView, MViewPosition: 2, Position: (1520, 116), Width: 400, Height: 848, mDoublePushControlListBean:…

[0136] After calculating the window display positions of the left and right projection windows, the View management module will layout and display the left and right projection windows. At the same time, the broadcast control view will be initialized and displayed according to the broadcast control sublist data of the Session. For example, the display device 200 can use the Recycleview control to display the broadcast control view, and set the display parameters of the Recycleview control through LayoutParams, including the position (i.e., coordinates) from the upper and left boundaries, and the corresponding display width and height, and then display the position through the setLayoutParams method.

[0137] As for the number of playback control views displayed when displayed horizontally, after calculation by the algorithm processing module, the display area width 1508 of the left projection window is greater than the total width 528 of the playback control bar when six playback control function controls are displayed, indicating that all playback control function controls can be displayed. Therefore, the playback control view created and displayed based on the playback control list data of the Session is presented as follows: Fig.13 The display effect is that six playback control function buttons are displayed in the center below the left projection window. Of course, they can also be displayed in any other position.

[0138] After the display device 200 displays the broadcast control view, the user can input a control instruction based on the broadcast control view to control the display device 200 to perform an interactive response. The display device 200 can respond to a click instruction of the target broadcast control view, obtain the broadcast control type of the target broadcast control view in the broadcast control list, call the processing interface associated with the broadcast control type, and perform the processing operation associated with the target broadcast control view on the projection data based on the processing interface.

[0139] For example, Fig.13 The user can use the arrow keys on the remote control that is compatible with the display device 200 to control the focus cursor to move to the "play" button, and press the "confirm" key on the remote control to enter the pause command. At this time, Recycleview will listen to the user's onClick operation, and for the index type of the play control list, that is, the mType type is playstatus, then call the Session's setPlayerPause() interface to pause the playback. Similarly, fast forward and fast rewind can also call Session's setPlayerSeekBackward() and setPlayerSeekForward() to perform corresponding play control processing.

[0140] In some embodiments, when the display area of ​​the projection window changes, the play control function control also needs to be dynamically adjusted according to the display area of ​​the projection window. That is, the display device 200 can respond to the change notification of the display area of ​​the projection window, obtain the changed display area, reset the display position of the play control view according to the changed display area, and recalculate the display quantity of the play view according to the area size of the changed display area.

[0141] In this regard, the screen projection management module can monitor the screen projection Session status. When the Video changes, the algorithm processing module is notified through the message processing module. The algorithm processing module calculates the screen projection display area, and then informs the data management module through the message processing module. After the data management module organizes the data, it notifies the View management module through the message processing module to redraw and display the View.

[0142] For example, when the media content of the VIDEO_URL type changes or the direction of the video stream changes, the display device 200 can rearrange the display of the projection content to present the following: Fig.14 The display effect shown is that the left projection window displays the media type VIDEO_URL type of the media screen. Through the algorithm processing module, it can be calculated that the size of the left projection window is 459×816, and the size of the right projection window is 1449×816. The following data results are obtained:

[0143] Session: SessionClient1, View: SurfaceView / TextureView, MViewPosition: 1, Position: (0, 132), Width: 459, Height: 816, mDoublePushControlListBean:…

[0144] Session: SessionClient2, View:, MViewPosition:, Position: (1520, 132), Width: 1449, Height: 816, mDoublePushControlListBean:…

[0145] For the display quantity when the broadcast control view is displayed horizontally, after calculation by the algorithm processing module, the display area width 459 of the left projection window is less than the total width 528 of the broadcast control bar when six broadcast control function controls are displayed, but greater than the total width 276 of the broadcast control bar when three broadcast control function controls are displayed, indicating that three broadcast control function controls can be displayed. Therefore, the three broadcast control function controls of full screen, mute / unmute, and off can be displayed, and their data format is as follows:

[0146] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132];

[0147] DoublePushControlBean[mStatus=true, mType=mutestatus, mResId=2131231152];

[0148] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121].

[0149] The View management module can notify Recycleview to refresh the View according to the above broadcast control data through the notifyDataSetChanged of the Adapter, and change from 6 broadcast control function controls to 3 broadcast control function controls, showing the following: Fig.14 The display effect is shown, and three broadcast control function buttons are displayed in the center below the left projection window. It can be seen that in the above embodiment, when the width of the projection window cannot meet the arrangement of six broadcast control function controls, the display device 200 can adjust the number of broadcast control function controls from six to three for display.

[0150] Similarly, when the projection management module listens to the onConnect connection of the new Session through UsessionConnectionListener, it can obtain the current media type of Client2 from the Session, which is MIRROR_STREAM, and the View management module can be used to layout and display it as follows Fig.15 As shown in the display effect, the right projection window displays the mirrored media screen, which can be full screen, muted / unmuted, or turned off on the display device 200. At this time, broadcast control data can be added to this Session. The data format is as follows:

[0151] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132];

[0152] DoublePushControlBean[mStatus=true, mType=mutestatus, mResId=2131231152];

[0153] DoublePushControlBean[mStatus=true, mType=close, mResId=2131231121].

[0154] The algorithm processing module can calculate that the size of the right projection window is 400×848, that is, the following data results are obtained:

[0155] Session: SessionClient2, View: SurfaceView / TextureView, MViewPosition: 2, Position: (1520, 116), Width: 400, Height: 848, mDoublePushControlListBean:…

[0156] The algorithm processing module calculates the projection display area, and then informs the data management module through the message processing module. According to the data of the data management module, the size of the projection window on the right is 400×848, and the size of the three broadcast control function controls is 276. Since the display area width of the projection window on the right, 400, is greater than the total width of 276 occupied by the three broadcast control function controls, the three broadcast control function buttons can be fully displayed.

[0157] The View management module can initialize and display Recycleview, set display parameters through LayoutParams, including the position (i.e. coordinates) from the upper and left boundaries, and the corresponding display width and height, and then display the position through the setLayoutParams method to present the following: Fig.15 The display effect is as follows: three playback control function buttons are displayed in the center below the right projection window.

[0158] When the width of the projection window cannot accommodate three playback control buttons, you can continue to adjust the number of playback control buttons according to the width of the display area of ​​the projection window. Fig.16 Taking the display effect shown as an example, if the size of the right projection window is 258×864, the display area width 258 of the right projection window is less than the total width 276 when three playback control function controls are displayed, and greater than the total width 192 when two playback control function controls are displayed. Therefore, the display device 200 can display two playback control function controls below the right projection window. In addition, since the volume can be mutually exclusive controlled by the playback control function controls of the left projection window, only the full screen and closed playback control function controls can be retained, that is, the corresponding playback control list data is:

[0159] DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132];

[0160] DoublePushControlBean[mStatus=true, mType=close, mResId==2131231121].

[0161] Similarly, Fig.17 Taking the display effect shown as an example, if the size of the right screen projection window is 156×864, the display area width 156 of the right screen projection window is less than the total width 192 when two broadcast control function controls are displayed, and greater than the total width 108 when one broadcast control function control is displayed. Therefore, the display device 200 can display a broadcast control function control below the right screen projection window. Since closing the screen projection window can be controlled by stopping the screen projection operation on the terminal device 500, only the full screen button can be retained, that is, the corresponding broadcast control list data is: DoublePushControlBean[mStatus=true, mType=fullscreen, mResId=2131231132].

[0162] By analogy, if the size of the right window is 88×864, the display area width 88 of the right projection window is smaller than the total width 108 when a broadcast control function control is displayed, and the broadcast control button control cannot be displayed. Therefore, the display device 200 can cancel the display of the broadcast control function control below the right projection window.

[0163] In this embodiment, in view of the diversity of the screen projection content between the display device 200 and the terminal device 500, the broadcast control function controls adapted to the screen projection content of the terminal device 500 are displayed. Different broadcast control function controls are displayed for different media types such as mirrored video streams, pushed URL addresses, pictures, music, etc., so as to reduce the phenomenon that there are errors in the broadcast control operations input by users, and when multiple screens are projected, users can intuitively distinguish different types of screen projection. For example, mirrored video projection does not support fast forward and rewind operations, nor does it support play and pause operations, so the broadcast control function controls for fast forward, fast rewind, play, and pause are not displayed; the pushed URL address can perform operations such as fast forward, fast rewind, play, and pause, so the broadcast control function controls for fast forward, fast rewind, play, and pause are displayed; the music can perform operations such as fast forward, fast rewind, play, and pause, so the broadcast control function controls for fast forward, fast rewind, play, and pause are displayed; the picture does not support operations such as fast forward, fast rewind, play, and pause, so the broadcast control function controls for fast forward, fast rewind, play, and pause are not displayed.

[0164] Based on the broadcast control view display method provided in the above embodiment, some embodiments of the present application further provide a display device 200, the display device 200 includes a display 260, a communicator 220 and a controller 250, the display 260 is configured to display a screen projection interface, the screen projection interface includes a screen projection window, and the screen projection window is used to display the screen projection picture of the terminal device 500; the communicator 220 is configured to establish a communication connection with the terminal device 500; the controller 250 is configured to execute the following program steps:

[0165] In response to a connection event for establishing a screen projection connection relationship between the display device 200 and the terminal device 500, a media asset type of the screen projection data is obtained.

[0166] Extract the broadcast control data associated with the media asset type from the broadcast control list, and create a broadcast control view based on the broadcast control data. The broadcast control list includes the broadcast control data of the preset broadcast control view, and the broadcast control view is a control in the projection interface for performing playback control on the projection data.

[0167] Set the display position of the playback control view according to the display area of ​​the projection window.

[0168] The control display 260 displays the play control view at the display position.

[0169] It can be seen from the above technical solution that the display device provided by the above embodiment can obtain the media type of the projection data in response to a connection event that establishes a projection connection relationship between the display device and the terminal device. Extract the broadcast control data associated with the media type in the broadcast control list, and create a broadcast control view based on the broadcast control data. Then set the display position of the broadcast control view according to the display area of ​​the projection window, and finally control the display to display the broadcast control view at the display position. Among them, the broadcast control list includes the broadcast control data of the preset broadcast control view, and the broadcast control view is a control in the projection interface for performing playback control on the projection data. The method can dynamically adjust the display of the broadcast control function control according to the media type and the projection display area to improve the user experience.

[0170] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0171] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0173] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: A display, configured to display a screen projection interface, wherein the screen projection interface includes a screen projection window, and the screen projection window is used to display a screen projection image of a terminal device; A communicator is configured to establish a screen projection connection with a terminal device; The controller is configured as: In response to a connection event of establishing a screen projection connection relationship between the display device and the terminal device, obtaining a media asset type of the screen projection data; Extracting broadcast control data associated with the media asset type from the broadcast control list, and creating a broadcast control view according to the broadcast control data, wherein the broadcast control list includes broadcast control data of a preset broadcast control view, and the broadcast control view is a control in the projection interface for performing playback control on the projection data; Setting the display position of the playback control view according to the display area of ​​the screen projection window; The display is controlled to display the play control view at the display position.

2. The display device according to claim 1, characterized in that The controller is also configured to: Obtaining configuration information when the display device establishes a screen projection connection with the terminal device; The media asset type of the screen projection data of the terminal device is read in the configuration information.

3. The display device according to claim 2, characterized in that The controller extracts the broadcast control data associated with the media asset type from the broadcast control list, and is further configured to: Calling a broadcast control list, wherein the broadcast control data in the broadcast control list includes a state identifier, a broadcast control type, and a resource identifier, wherein the state identifier is used to indicate a functional state of the broadcast control view, the broadcast control type is used to indicate a processing operation associated with the broadcast control view, and the resource identifier is used to indicate a functional module associated with the broadcast control view; generating a broadcast control sub-list according to the broadcast control data associated with the media asset type in the broadcast control list; A play control view is created according to the play control sub-list.

4. The display device according to claim 3, characterized in that The controller generates a broadcast control sub-list according to the broadcast control data associated with the media asset type in the broadcast control list, and is further configured to: Copying the broadcast control list, and deleting the broadcast control data not associated with the media asset type in the broadcast control list, so as to generate a broadcast control sub-list; Alternatively, a broadcast control sub-list is created, and the broadcast control data associated with the media asset type in the broadcast control list is added to the broadcast control sub-list.

5. The display device according to claim 1, characterized in that The controller is also configured to: Obtaining the area size of the display area of ​​the projection window, and obtaining the preset view size of the broadcast control view; Calculating the display quantity of the playback view according to the area size and the view size; According to the display quantity, the broadcast control data associated with the media asset type is extracted from the broadcast control list.

6. The display device according to claim 5, characterized in that The controller is also configured to: In response to the change notification of the display area, acquiring the changed display area; The display position of the play control view is reset according to the changed display area, and the display quantity of the play view is recalculated according to the area size of the changed display area.

7. The display device according to claim 5, characterized in that The controller extracts the broadcast control data associated with the media asset type in the broadcast control list according to the display quantity, and is further configured to: Traversing the weight values ​​of the broadcast control data in the broadcast control list; Sorting the broadcast control data in the broadcast control list in descending order of weight values; The aforementioned displayed number of broadcast control data associated with the media asset type are extracted from the sorted broadcast control list.

8. The display device according to claim 5, characterized in that The controller is also configured to: Obtaining the area width of the display area; Acquire the total width of the play control bar, where the play control bar is arranged according to the play control views associated with the media asset type at a preset spacing; If the area width is greater than or equal to the total width, extracting the broadcast control data associated with the media asset type in the broadcast control list; If the area width is smaller than the total width, the display quantity is calculated according to the area width and the total width, and the broadcast control data associated with the media asset type is extracted from the broadcast control list according to the display quantity.

9. The display device according to claim 1, characterized in that: The controller is also configured to: In response to a click instruction of a target broadcast control view, acquiring a broadcast control type of the target broadcast control view in the broadcast control list; Calling the processing interface associated with the broadcast control type; Based on the processing interface, a processing operation associated with the target broadcast control view is performed on the projection data.

10. A method for displaying a broadcast control view, characterized in that: The display device according to any one of claims 1 to 9, the display device comprising a display, a communicator and a controller, the display being configured to display a screen projection interface, the screen projection interface being used to display a screen projection image of a terminal device; The communicator is configured to establish a screen projection connection with a terminal device; the method comprises: In response to a connection event of establishing a screen projection connection relationship between the display device and the terminal device, obtaining a media asset type of the screen projection data; Extracting broadcast control data associated with the media asset type from the broadcast control list, and creating a broadcast control view according to the broadcast control data, wherein the broadcast control list includes broadcast control data of a preset broadcast control view, and the broadcast control view is a control in the projection interface for performing playback control on the projection data; Setting the display position of the playback control view according to the display area of ​​the screen projection window; The display is controlled to display the play control view at the display position.

Images