Display device and content display method

Abstract

The invention relates to a display device and a content display method. Comprising the following steps: acquiring an interactive application data packet from broadcast information of a live program, and starting an interactive application based on the interactive application data packet; in the process of starting the interactive application, when a first data acquisition instruction is detected and the first data acquisition instruction carries a media asset data type parameter, triggering a data service process through an application service process to acquire a filter identifier; when the data service process obtains the filter identifier, triggering the data service process to call a response callback function of the application service process so as to transmit the filter identifier to the application service process; under the condition that the application service process receives the filter identifier, triggering the application service process to obtain media asset data corresponding to the media asset data type parameter from a target filter corresponding to the filter identifier; and controlling a display to display media asset data corresponding to the media asset data type parameter according to a media asset data display mode preset by the interactive application. The communication efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a content display method. Background Art

[0002] In some countries or regions, such as Brazil in South America, some digital televisions support the Ginga service. The Ginga service is based on the ISDB-T digital terrestrial television middleware standard, which enhances the interactivity of digital television and supports users to enjoy various online services for free while watching digital television (DTV).

[0003] Currently, the Ginga service can support interactive applications, which may include but are not limited to voting or promotional advertising applications. The Ginga service can also support interactive applications that may need to obtain live broadcast data during operation.

[0004] However, currently, the process of the Ginga service in the display device communicating with other services or processes in the display device is relatively cumbersome, resulting in low efficiency of interactive applications in acquiring data, and thus low communication efficiency. Summary of the invention

[0005] The present application provides a display device and a content display method to solve the problem of low communication efficiency.

[0006] In a first aspect, some embodiments provide a display device, comprising: a display and a controller; the controller is configured to: upon receiving a playback operation of a live program, obtain an interactive application data packet from the broadcast information of the live program, and start an interactive application based on the interactive application data packet; during the process of starting the interactive application, when a first data acquisition instruction is detected and the first data acquisition instruction carries a media data type parameter, trigger a data service process through an application service process to obtain a filter identifier, the filter identifier is used to find a target filter, and the target filter is used to obtain media data corresponding to the media data type parameter from the broadcast information; when the data service process obtains the filter identifier, trigger the data service process to call a response callback function of the application service process to transmit the filter identifier to the application service process; when the application service process receives the filter identifier, trigger the application service process to obtain media data corresponding to the media data type parameter from the target filter corresponding to the filter identifier; and control the display to display the media data corresponding to the media data type parameter according to the media data display method preset by the interactive application.

[0007] In this embodiment, since the filter identifier is transmitted first and then the application service process actively obtains the media data, for media data with a large amount of data, the amount of transmitted data is reduced compared to indirectly transmitting the media data to the application service process through the interactive service process, thereby improving data transmission efficiency and communication efficiency.

[0008] In a third aspect, some embodiments further provide a content display method, applied to the display device provided in the first aspect, the display device comprising: a display, a communication device and a controller, the method comprising: upon receiving a playback operation of a live program, obtaining an interactive application data packet from the broadcast information of the live program, and starting an interactive application based on the interactive application data packet;

[0009] During the process of starting the interactive application, when a first data acquisition instruction is detected and the first data acquisition instruction carries a media asset data type parameter, a data service process is triggered by an application service process to acquire a filter identifier, the filter identifier is used to find a target filter, and the target filter is used to acquire media asset data corresponding to the media asset data type parameter from broadcast information;

[0010] When the data service process obtains the filter identifier, the data service process is triggered to call a response callback function of the application service process to transmit the filter identifier to the application service process;

[0011] When the application service process receives the filter identifier, triggering the application service process to obtain the media asset data corresponding to the media asset data type parameter from the target filter corresponding to the filter identifier;

[0012] According to the media data display mode preset by the interactive application, the display is controlled to display the media data corresponding to the media data type parameter.

[0013] In this embodiment, since the filter identifier is transmitted first and then the application service process actively obtains the media data, for media data with a large amount of data, the amount of transmitted data is reduced compared to indirectly transmitting the media data to the application service process through the interactive service process, thereby improving data transmission efficiency and communication efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0016] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0017] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0018] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0019] Figure 5 A system architecture diagram of a display device provided for some embodiments of the present application;

[0020] Figure 6 A standard framework diagram provided for some embodiments of the present application;

[0021] Figure 7 A flowchart of a content display method provided in some embodiments of the present application;

[0022] Figure 8 A timing diagram of transmitting interactive application data packets and filter identifiers provided in some embodiments of the present application;

[0023] Fig. 9 A timing diagram from starting an application service process to closing an application service process provided in some embodiments of the present application;

[0024] Fig.10 A schematic diagram of an interactive interface and a media playback window provided in some embodiments of the present application;

[0025] Fig.11 A schematic diagram of an interactive interface and a media playback window provided in some other embodiments of the present application;

[0026] Fig.12 A timing diagram of the communication between the application service process and the live broadcast application process provided in some other embodiments of the present application;

[0027] Fig.13 A schematic diagram of an interactive application menu interface provided by some other embodiments of the present application;

[0028] Fig.14 A schematic diagram of a WebSocket client list provided for other embodiments of the present application;

[0029] Fig.15 A timing diagram of communication between an application service process and a live broadcast application process provided in some other embodiments of the present application. DETAILED DESCRIPTION

[0030] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0031] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and common meanings.

[0032] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0033] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0034] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0035] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0036] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0037] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0038] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks.

[0039] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function of a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0040] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0041] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0042] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0043] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0044] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0045] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0046] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0047] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0048] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0049] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0050] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0051] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0052] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0053] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0054] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0055] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0056] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0057] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0058] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0059] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0060] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0061] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may provide a user interface (control the display device), allow the user to interact with the display device 200, and support the running of various application programs.

[0062] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0063] The operating system can be divided into different modules or layers according to the functions implemented.

[0064] For example, Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications layer) (referred to as "application layer"), the application framework layer (Application Framework layer) (referred to as "framework layer"), the system library layer and the kernel layer.

[0065] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0066] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions of applications in the application layer. Applications can access system resources and obtain system services during execution through the API interface.

[0067] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager is used to interact with all activities running in the system; a location manager is used to provide system services or applications with access to system location services; a package manager is used to retrieve various information related to the application package currently installed on the device; a notification manager is used to control the display and clearing of notification messages; and a window manager is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0068] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0069] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0070] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0071] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0072] In some embodiments, the display device is a digital television (Digital Television, DTV), for example, a digital television based on Ginga middleware. The Ginga middleware standard is a digital television middleware standard jointly formulated by multiple countries in South America. It is used to provide multimedia content that can interact with users in digital television broadcasts and enhance the user's experience of watching digital television. The Ginga middleware standard has gradually evolved to better meet user needs. At present, the Ginga middleware standard has evolved from the Ginga-A middleware standard, the Ginga-B middleware standard, the Ginga-C middleware standard to the Ginga-D middleware standard. The Ginga middleware supports a variety of functions, including but not limited to program search, program recording, program reservation or interactive services, etc., providing digital TV users with a rich and convenient viewing experience. The display device can display the Ginga menu page, and the user can make relevant settings through the Ginga menu page.

[0073] In some embodiments, the display device provided by the present application may be a display device based on the TIAF (TV Interactive Application Framework) architecture, such as Figure 5 As shown, an overall architecture diagram of a display device based on the TIAF architecture is provided. The overall architecture diagram includes an application layer, in which a live broadcast application is deployed, such as LiveTV. The live broadcast application is mainly used to play live TV, and can be used to interact with different DTV (Digitaltelevision) protocols to complete the display, stop display, setting and switching of display types of subtitles, etc. Hidden subtitles can be displayed in the live broadcast page provided by the live broadcast application. Among them, the live broadcast application includes multiple modules such as the Ginga menu module, the Ginga button distribution module, the Ginga life cycle management module, the interactive application view (TvInteractiveAppView) module, the Ginga aidl client module, etc. The interactive application view (TvInteractiveAppView) module is used to provide an interactive application view on the TV. aidl is a tool in the Android SDK that allows developers to define and implement interfaces between clients and servers, which can communicate across processes.

[0074] The overall architecture diagram also includes the framework layer, which includes TIF (resource management framework), TIAF (interactive application management framework), and TIS (TvInputService, TV input service). Among them, TIF (resource management framework) and TIAF (interactive application management framework) are frameworks launched by Google. Figure 6As shown, a standard framework diagram including a TIF framework and a TIAF framework is shown, wherein a TV App refers to an application used on a large-screen output device such as a smart TV or a TV box. The TIAF framework includes TvInteractiveAppView (TIAV for short, a view component responsible for displaying the user interface of the application on the smart TV screen, which can contain a variety of UI elements such as buttons, text boxes or pictures, etc., for interacting with users), TvInteractiveAppManager (TIAM for short, a management component responsible for managing and coordinating applications on smart TVs, which can include functions such as starting, stopping, switching applications, as well as communication and data sharing between applications, and can also monitor the status of applications to ensure that applications can run normally), TvInteractiveAppManagerService (a service component that can provide functions such as installation, uninstallation, and update of applications, as well as responsible for application permission management and security control), TvInteractiveAppServiceInterface (an interface definition used to specify the communication protocol between smart TV applications and external systems or services, defining the methods and events that applications can call, and the data formats that external systems or services can receive and process. By implementing this interface, applications can interact with the operating system of smart TVs, other applications or cloud services) and other components. TvInteractiveAppService Impl is the specific implementation class of TvInteractiveAppService Interface, which is responsible for providing services for interacting with smart TV applications in the smart TV application framework. The TIF framework includes TvView (a view class used to display TV content. Using TvView, you can browse the channel list, play video content after selecting a channel, etc.), TvInputManager (a system service class that allows TV applications to access TV Input applications for communication to obtain various video input sources), TvInputManagerService (TV app needs to create a session through TvInputManagerService to access each input port. TvInputManagerService maintains the status information of all ports in TV. If Tv App registers TvInputCallback, then the status change of TV Input or the addition and removal of TV Input can be notified), TvInputService Interface (an Android TV server component interface used to manage video input sources), etc.TvInputServiceImpl is the implementation class of TvInputService Interface. It provides the ability to interact with the TV input source by implementing the methods in the TvInputService interface, and realizes communication and data sharing with TV applications. Regular Android App refers to commonly used Android applications, and Interactive App RTE refers to real-time interactive applications. TvInputServiceImpl can be understood as TIS. TIS includes SessionrequestBroadcastInfo (a method for requesting broadcast information, used to obtain the session information of the current TV input service such as channel list, program information, signal status, etc. This method can be called by the TV application to display available channels and programs to the user).

[0075] The overall architecture diagram also includes the middleware layer, which includes AV middleware, system middleware, and live broadcast middleware. Among them, AV middleware (Audio Video Middleware) can process Audio (audio data) attributes for processing, system middleware can be responsible for processing attributes related to the operating system platform platform, and live broadcast middleware includes SI (Service Information, responsible for processing information in live broadcast services) module, Tuner (responsible for tuning and receiving live broadcast signals) module, and Resource (responsible for managing resources in live broadcast services, such as network bandwidth, memory, CPU, etc.) module. The middleware layer also includes Android JAVA SDK API, which includes Tuner Framework (a framework for processing TV signal tuning), Media Codec (an API for video and audio encoding and decoding), and AudioTrack (an audio playback API that allows applications to directly play audio data in PCM (Pulse Code Modulation) format). Tuner Framework includes MediaTvTuner (responsible for tuning and managing TV channels) and Tuner RM (Resource Manager), which is responsible for managing and allocating TV tuner resources. The middleware calls the tuner framework api through the HUI (isolation layer). The tuner framework api is a set of interfaces for managing TV tuner functions.

[0076] The overall architecture diagram also includes MopaGinga service, which includes TIAS (i.e. TvInteractiveAppService Impl), Integration Aidl service, data processing module, network service module (CC WebServices module), html5 application and ncl application.

[0077] The overall architecture diagram also includes HAL (Hardware Abstraction Layer). There are many HAL interfaces for different hardware components in the HAL layer, such as TvInputHAL (used to handle TV input-related functions, such as receiving and decoding TV signals and switching TV channels), TunerHAL (HAL interface specifically used to control and manage TV tuner hardware), PQHAL (Hardware Abstraction Layer interface related to image processing and color correction), PQ is the abbreviation of Picture Quality, Codec2 (multimedia-related software framework), HWC (Hardware Composer HAL, responsible for synthesizing and displaying content on the screen), DisplayHAL (HAL interface for controlling and managing display devices), AudioHAL (used to handle audio-related functions, such as audio recording, playback, volume adjustment, etc.), Audio EXT HAL (Extended HardwareAbstraction Layer, a hardware abstraction layer for processing audio extension functions), TPCHAL (Touch PanelController HAL, a HAL interface for controlling and managing touch screen hardware).

[0078] The overall architecture diagram also includes the kernel layer, which includes the Linux kernel interface and STI DDdriver. DD stands for "Device Driver". STI DD driver includes tuner driver and demodulator driver. The tuner driver sends control instructions to the tuner to implement functions such as channel switching, signal amplification and filtering. These instructions may include frequency setting, gain control, etc. to ensure that the tuner can accurately receive and convert TV signals. Its main function is to select and lock TV signals of a specific frequency, and then convert them into intermediate frequency signals (IF) for subsequent processing. The demodulator driver is responsible for processing the intermediate frequency signal output by the tuner. The main task of the demodulator is to restore the intermediate frequency signal to the original baseband signal, that is, the standard data stream containing TV image and sound information. The demodulator driver ensures accurate data extraction and restoration by executing complex signal processing algorithms such as carrier recovery, clock recovery, symbol synchronization, etc.

[0079] In some embodiments, the present application provides an interactive content display method, which is applied to a display device, the display device is based on the TIAF architecture, such as Figure 7 As shown, the interactive content display method includes:

[0080] Step 702: upon receiving a play operation of a live program, obtaining an interactive application data packet from the broadcast information of the live program, and starting an interactive application based on the interactive application data packet.

[0081] Specifically, when receiving a play operation of a live program, the display device may start an application service process. When the application service process is started, the interactive application data packet is obtained from the broadcast information of the live program through the application service process, and the interactive application is started based on the interactive application data packet through the application service process. The play operation may be an operation triggered in the live application, or may be an operation input by voice or remote control. The live application may be, but is not limited to, LiveTv.

[0082] Among them, the application service process is the process corresponding to the application service, such as the Ginga service. Application services (Ginga services) include but are not limited to GStreamer, GingaCCWS, and IntegrationService. Among them, GStreamer is an open source multimedia framework for building audio and video processing pipelines. On the smart TV platform, GStreamer can be used to play, record and edit audio and video content. GingaCCWS refers to the Ginga interactive content service. Ginga is an interactive content platform for smart TVs that provides a rich multimedia experience. Interactive applications are applications for interaction supported by application services, which can be but are not limited to voting applications, advertising promotion applications, or applications for controlling the live interface of live programs. Interactive application data packets refer to data packets of interactive applications, which are used to start interactive applications.

[0083] A live program may be a television program of a broadcasting system such as a radio station or a television station. The broadcast information is the information of the live program broadcast by the broadcasting system. The broadcasting system may be a system that supports any DTV (Digital Television) protocol, for example, it may be the system of any radio and television station. The DTV protocol may be, but is not limited to, ATSC1.0, ATSC3.0, DVB or ISDB, etc. ATSC is the abbreviation of Advanced Television Systems Committee, which means Advanced Television System Committee in Chinese. DVB is the abbreviation of Digital Video Broadcasting, which means Digital Video Broadcasting in Chinese. ISDB is the abbreviation of Integrated Service Digital Broadcasting, which means Integrated Service Digital Broadcasting in Chinese.

[0084] Step 704, during the process of starting the interactive application, when a first data acquisition instruction is detected and the first data acquisition instruction carries a media data type parameter, the data service process is triggered by the application service process to obtain a filter identifier, and the filter identifier is used to find a target filter, and the target filter is used to obtain media data corresponding to the media data type parameter from the broadcast information.

[0085] The first data acquisition instruction is an instruction issued by an interactive application for acquiring media data. The media data type parameter is used to characterize that the data to be acquired is media data, and the media data type parameter can be, but is not limited to, PES or TS. PES (Packet Elementary Stream) and TS (Transport Stream). PES and TS store audio and video data. The data service process can be, for example, a TIS process.

[0086] Step 706: When the data service process obtains the filter identifier, the data service process is triggered to call the response callback function of the application service process to transmit the filter identifier to the application service process.

[0087] Specifically, after obtaining the filter identifier, the data service process can send a first response message to the application management component TIAM, and the first response message carries the filter identifier. After receiving the first response message, the application management component TIAM sends a second response message carrying the filter identifier to the interactive service process. After receiving the second response message, the interactive service process calls the response callback function in the application service process to transmit the filter identifier to the application service process.

[0088] Step 708: When the application service process receives the filter identifier, the application service process is triggered to obtain media asset data corresponding to the media asset data type parameter from the target filter corresponding to the filter identifier.

[0089] The target filter is used to extract PES and / or TS from the broadcast information.

[0090] Step 710: Display the media data corresponding to the media data type parameter according to the media data display method preset by the interactive application.

[0091] The interactive application may pre-configure a media data display method, which may include but is not limited to: the position and size of the media playback window, the format of subtitles, etc. The media playback window is a video live broadcast area, which is used to play live content.

[0092] In this embodiment, when a play operation of a live program is received, an interactive application data packet is obtained from the broadcast information of the live program, and an interactive application is started based on the interactive application data packet. During the process of starting the interactive application, when a first data acquisition instruction is detected and the first data acquisition instruction carries a media data type parameter, the data service process is triggered by the application service process to obtain a filter identifier, the filter identifier is used to find a target filter, and the target filter is used to obtain media data corresponding to the media data type parameter from the broadcast information. When the data service process obtains the filter identifier, the data service process is triggered to call a response callback function of the application service process to transmit the filter identifier to the application service process. When the application service process receives the filter identifier, the application service process is triggered to obtain media data corresponding to the media data type parameter from the target filter corresponding to the filter identifier. According to the media data display mode preset by the interactive application, the display is controlled to display the media data corresponding to the media data type parameter. Since the filter identifier is transmitted first and then the application service process actively obtains the media data, for media data with a large amount of data, compared with indirectly transmitting the media data to the application service process through the interactive service process, the amount of data transmitted is reduced, thereby improving data transmission efficiency and communication efficiency.

[0093] In some embodiments, the present application provides a display device, including: a display and a controller; the controller is configured to: when receiving a playback operation of a live program, obtain an interactive application data packet from the broadcast information of the live program, and start the interactive application based on the interactive application data packet; during the process of starting the interactive application, when a first data acquisition instruction is detected and the first data acquisition instruction carries a media data type parameter, trigger a data service process through an application service process to obtain a filter identifier, the filter identifier is used to find a target filter, and the target filter is used to obtain media data corresponding to the media data type parameter from the broadcast information; when the data service process obtains the filter identifier, trigger a response callback function of the data service process to transmit the filter identifier to the interactive service process; when the interactive service process receives the filter identifier, trigger the interactive service process to transmit the filter identifier to the application service process; when the application service process receives the filter identifier, trigger the application service process to obtain the media data corresponding to the media data type parameter from the target filter corresponding to the filter identifier; according to the media data display method preset by the interactive application, control the display to display the media data corresponding to the media data type parameter.

[0094] In this embodiment, since the filter identifier is transmitted first and then the application service process actively obtains the media data, for media data with a large amount of data, the amount of transmitted data is reduced compared to indirectly transmitting the media data to the application service process through the interactive service process, thereby improving data transmission efficiency and communication efficiency.

[0095] In some embodiments, when the controller receives a playback operation of a live broadcast program and obtains an interactive application data packet from the broadcast information of the live broadcast program, it is configured to: start the application service process when receiving the playback operation of the live broadcast program, and send an application data acquisition request to the data service process through the application service process, wherein the application data acquisition request carries a business data type parameter; when the data service process receives the data acquisition request, trigger the data service process to obtain the business data in the broadcast information of the live broadcast program from the filter corresponding to the business data type parameter, and transmit the business data to the application service process; when the application service process receives the business data, trigger the application service process to obtain the interactive application data packet from the business data.

[0096] The service data in the broadcast information may include multiple tables, including but not limited to: TOT (Time and Date Table, used to provide current time and date information), PAT (Program Association Table, program association table), PMT (Program Map Table, program mapping table), NIT (Network Information Table, network information table), SDT (Service Description Table, service description table, describes various services provided to viewers, it is divided into different sub-tables, which respectively contain the detailed content of the current TS stream and other TS streams. Without SDT, the decoder cannot provide the audience with relevant service information), EIT (Event Information Table, event information table, describes the various service information at the current and next moments), AIT (Application Information Table, application information table, which may include the name or logo of the interactive application, a brief description or function introduction of the interactive application, an icon or picture of the interactive application, whether the interactive application has been installed or is available, etc., the version number or developer information of the interactive application, etc. According to the information in the interactive application service list, it can be used to build a user interface and display the available interactive applications in the user interface), DSMCC (Digital Storage Media Command and Control, a digital storage media command and control extension protocol, is used to manage MPEG-1 and MPEG-2 data streams, so that the data streams can run on a single machine and in a heterogeneous network environment (i.e., a network constructed with similar devices but running different protocols). Each table may contain multiple sections. These sections are organized according to specific syntax and format. Each section contains specific information, such as program information, network information, event information, etc., and has a unique identifier for easy distinction. The filter corresponding to the service data type parameter is a filter used to filter out or extract service data from the broadcast information.

[0097] Specifically, the application service process can send an application data acquisition request to the interactive service process, and the interactive service process forwards the application data acquisition request to the data service process. The interactive service process can establish a communication connection with the application service component, and the interactive service process can forward the data acquisition request to the application service component, and the application service component forwards the data acquisition request to the data service process. The application service component is a component in the live broadcast application. The live broadcast application process and the application service process communicate through the interactive service process and the application service component.

[0098] In some embodiments, after the data service process receives the data acquisition request, the data service process applies for a filter corresponding to the Section, and obtains the interactive application data packet from the filter corresponding to the Section. The data service process can apply for the filter by calling TunerApi through the middleware (TvMw).

[0099] like Figure 8 As shown, a timing diagram for transmitting interactive application data packets and filter identifiers is shown, wherein TIAM represents the application service component, TIAS represents the interactive service process, Ginga service represents the application service process, and TIS represents the data service process. The timing diagram includes two parts, namely: the timing of obtaining the interactive application data packet and the timing of obtaining the filter identifier. In the timing of the interactive application data packet, requesting broadcast information refers to sending a data acquisition request. "Filter" is a filter used to filter out business data from the broadcast information, and registering a filter callback refers to registering a callback function related to the filter. "Data" includes interactive application data packets. Data callback refers to the middleware transmitting "data" to TIS. "Notify broadcast information reply" is used to transmit the acquired business data. In the timing of obtaining the filter identifier, requesting broadcast information refers to the first data acquisition instruction, "filter" and refers to the filter used to filter out the media data corresponding to the media asset type parameter from the broadcast information, the shared filter is the applied filter, the shared filter is the target filter, and obtaining the shared filter identifier is used to obtain the filter identifier of the target filter, and the filter identifier is used to uniquely identify a filter. "Notify broadcast information reply" is used to transmit the acquired filter identifier.

[0100] In this embodiment, since the amount of business data is usually not very large, the business data is directly transmitted instead of the filter identifier, which can shorten the communication process of the application service process obtaining the business data, thereby improving the efficiency of obtaining interactive application data packets.

[0101] In some embodiments, the business data includes an application information table and an application data table. When the application service process receives the business data, the controller triggers the application service process to obtain an interactive application data packet from the business data. The controller is configured to: when the application service process receives the business data, trigger the application service process to obtain the application information table from the business data, and obtain the interactive application identifier from the application information table; when the application service process obtains the interactive application identifier, trigger the application service process to query the application data table to obtain the interactive application data packet corresponding to the interactive application identifier.

[0102] The application information table is, for example, an AIT table, and the application data table is, for example, a DSMCC table. The interactive application identifier may be an index of an interactive application data packet in the application data table, so that the corresponding interactive application data packet may be queried from the application data table according to the interactive application identifier.

[0103] In this embodiment, the interactive application identifier is obtained from the application information table, and the interactive application data packet corresponding to the interactive application identifier is obtained from the application data table, so that the interactive application data packet can be quickly obtained.

[0104] In some embodiments, when the controller executes the acquisition of interactive application data packets from the broadcast information of the live program and starts the interactive application based on the interactive application data packets, it is configured to: when the live application process is started, the application description information is acquired from the broadcast information of the live program through the live application process; when the application description information indicates that there is an interactive application in the broadcast information of the live program, the live application sends a service start request to the application management component; the application management component responds to the service start request and instructs the interactive service process to start the application service process; when the application service process is started, the interactive application data packet is acquired from the broadcast information of the live program through the application service process, and the interactive application is started based on the interactive application data packet. Among them, the live application process is the process corresponding to the live application.

[0105] Among them, the application description information can be an AIT table, i.e., an application information table. When the AIT table can include an application type corresponding to an application data packet in the broadcast information, and when the application type corresponding to the application data packet is an interactive application, i.e., a Ginga type application, it is determined that the application description information represents the presence of an interactive application in the broadcast information of the live program.

[0106] Specifically, when the application service process is started, the interactive service process may send a service ready notification to the live broadcast application process. When the live broadcast application process receives the service ready notification, the live broadcast application process sends an interactive application start request to the interactive service process. For example, the live broadcast application process may send an interactive application start request to the interactive service process by calling the application management component. When the interactive service process receives the interactive application start request, the interactive service process calls the application service process to start the interactive application.

[0107] like Fig. 9As shown, a timing diagram from starting the application service process to closing the application service process is shown. Among them, LiveTv represents the live application process, Ginga TIAS represents the interactive service process, and Ginga service represents the application service process. Among them, LiveTV (i.e. LiveTv) is started through the standard method of TvInteractiveView, and Ginga TIAS is called through the TIAF framework (TIAM in it) to implement the startup process. LiveTV can control the timing of starting and stopping the Ginga service process. When the input source source is triggered to switch or the application is triggered to switch in the live application, the live application process stops the Ginga service process, and when the live program is triggered to play, the live application process starts the Ginga service process, i.e. the application service process. For example, when switching to the DTV source, turn on the AIT reporting switch, and start the Ginga service process after the interactive service process receives the AIT message from the middleware. The Ginga service process can request resources and release resources. Users can start or stop the Ginga service process through the switch provided on the Ginga menu page. When the display device is in recording, time-shifting, switching channels, EPG (Electronic Program Guide), etc., exit the Ginga service process, return to LiveTv live broadcast and start the Ginga service process again. In the figure, Surface is an abstract concept used for graphics and video rendering in the Android framework, representing a screen surface, which can be an independent window of the entire screen or application UI, and the media view is a view used to play multimedia data. Url stands for Uniform Resource Locator, that is, uniform resource locator.

[0108] In the figure, a TIAS service link is established to establish a communication connection between TIAM and Ginga TIAS. After the communication connection is established, TIAM and Ginga TIAS can communicate to transmit data or messages. Creating a session is used to trigger the interactive service process to create a session to start the application service process in the session. "Notify service status change ready" is a service ready notification, which is used to notify the application service process that it is ready. When the interactive service process receives a request to start an interactive application, the interactive service process sends a "start data processing" notification to the application service process so that the application service process obtains the interactive application data packet from the broadcast information and starts the interactive application. The process of "requesting broadcast information and notifying broadcast information reply" in the figure is consistent with the timing of obtaining the interactive application data packet in the previous step. "Download completed" is used to notify the interactive service process that the application service has obtained the interactive application data packet from the business data, so as to trigger the interactive service process to send an instruction to start the application to the application service process. When the display device switches from the scene of playing the live program to other scenes, the live application process sends an instruction to stop the interactive application to the interactive service process. For example, the live application process sends an instruction to stop the interactive application to the interactive service process by calling the TIAM component to trigger the interactive service process to stop the application service process.

[0109] When TIAM starts, it scans all application service implementations (such as TIAS) in the system and registers them in the service list. The registration information of the service usually includes the name of the service, version number, provided interface, etc.

[0110] When an application needs a service, it will search for available service implementations through TIAM's service list. The application can filter suitable services by conditions such as the service name and version number. Once a suitable service implementation is found, the application will bind the service through the interface provided by TIAM. The binding process usually includes steps such as establishing a communication channel and negotiating a communication protocol. After the binding is successful, the application can communicate with TIAS through TIAM and call the required service interface. The communication process can be synchronous or asynchronous, depending on how the service is implemented and the needs of the application. When an application no longer needs a service, it will unbind it from the service through TIAM. After unbinding, TIAM will remove the service from the application's service list and release related resources. For example, when watching DTV, users can call up the Ginga application at any time to play Ginga resources.

[0111] In this embodiment, the service start request is sent only when the application description information indicates that there is an interactive application in the broadcast information of the live program, thereby avoiding sending the service start request when there is no interactive application and reducing resources in the display device.

[0112] In some embodiments, when the controller executes the media data display method preset by the interactive application and controls the display to display the media data corresponding to the media data type parameters, the controller is configured to: determine the media data display area according to the media data display method preset by the interactive application; control the display to display the media playback window and the interactive interface of the interactive application in layers, and display the media data corresponding to the media data type parameters in the media playback window; wherein the media data is displayed in the media data display area, and the interactive interface is displayed on the upper layer of the media playback window, and the interactive interface does not block the media playback window.

[0113] Among them, the interactive interface can display interactive content such as voting content. Fig.10 As shown, the display device displays an interactive interface 1002 and a media playback window 1004. There is a window in the interactive interface that is a transparent area, which is the media data display area. The media playback window is displayed at the position of the window. Therefore, although the interactive interface is displayed on the upper layer of the media playback window, the interactive interface will not block the media playback window.

[0114] In some embodiments, the interactive interface may display interactive controls related to the media asset playback window. Fig.11 As shown, an interactive interface 1102 and a media playback window 1104 are displayed on the display of the display device. The interactive interface is located above the media playback window. Interactive controls related to the media playback window are displayed in the interactive interface. The interactive controls related to the media playback window include, for example, the START control, STOP control, PAUSE control, and RESUME control in the figure. Window adjustment controls include, for example, the UP control, Left control, Right control, Down control, and ENTER (full screen display) control in the figure.

[0115] In some embodiments, when a trigger operation for an interactive control is detected, the application service process may send a corresponding instruction to the live application process through the interactive service process, so that the live application process performs the processing corresponding to the trigger operation of the interactive control. Fig.12As shown, the application service process can send instructions such as TUNE (for tuning to a specified channel), TUNE_NEXT (for tuning to the next channel, TUNE_PREV (for tuning to the previous channel), STOP (for stopping the currently playing content), SET_STREAM_VOLUME (for setting the volume level of the streaming media), SELECT_TRACK (user selects a specific audio track or subtitle track), SET_AUDIO_PROPERTY (for setting audio properties such as audio output device, audio mode or audio effect, etc.) to the live application process.

[0116] In this embodiment, the media resource playback window and the interactive interface can be displayed simultaneously, so that interaction can be triggered on the interactive interface while watching the media resource, thereby improving the flexibility of interaction.

[0117] In some embodiments, the media asset playback window is provided by a live broadcast application, and the interactive interface includes interactive controls for the media asset playback window. The controller is also configured to: when a trigger operation for the interactive control is detected, call the target interface function corresponding to the trigger operation in the interactive service process through the application service process; when the interactive service process receives the call to the target interface function, initiate a callback request for the target callback function to the live broadcast application process through the interactive service process, wherein the target callback function is a callback function corresponding to the target interface function; when the live broadcast application process receives the callback request, perform processing corresponding to the trigger operation on the media asset playback window through the live broadcast application process.

[0118] The live broadcast application process refers to the process of the live broadcast application. When a trigger operation on the interactive control is detected, the application service process can call an interface related to the view to instruct LiveTV to perform a corresponding operation on the media resource playback interface.

[0119] Specifically, taking the interactive control as a control for changing the size or position of the media asset playback window as an example, the target interface function that the application service process can call can be SetVideoBounds (used to set the boundaries of the video playback area, which refers to the media asset playback window), and the target callback function can be onSetVideoBounds. When the live application process receives the callback request, it performs the operation of setting the boundaries of the video playback area, such as updating the position of the playback window.

[0120] In some embodiments, the application service process can call the interface related to the view, and finally call back to LiveTV for implementation. LiveTV obtains relevant information (such as channel information, channel information, volume, live video window size, track) from the standard TvView and the AudioManager in the controller, and then returns it to the Ginga service process, i.e. the application service process. AudioManager allows the application to control the volume of the device, including media volume, call volume, ringtone volume, etc. For example Fig.12 In the application service process, the application service process can call requestCurrentTvInputId (requesting the identifier of the input source of the current TV), RequestCurrentChannelUri (called when requesting the URL of the current channel), RequestCurrentChannelLcn (called when requesting the logical channel number of the current channel), RequestCurrentVideoBounds (called when requesting the boundaries of the current video playback area), RequestStreamVolume (called when requesting the volume of the current streaming media), RequestTrackInfoList (called when requesting the currently available volume or subtitle track list to allow the user to select different audio tracks or subtitle tracks), sendPlayBackCommandRequest (called when requesting playback control commands such as play, pause, fast forward or rewind, etc.), and obtain information through corresponding callback functions such as onRequestCurrentChannelUri, onRequestCurrentChannelLcn, onRequestCurrentVideoBounds, onRequestStreamVolume, onRequestTrackInfoList, onPlayBackCommandRequest, etc. Correspondingly, after the live broadcast application process obtains relevant information according to the callback function, it can call the sending interface function to transmit the relevant data to the interactive service process. The interactive service process transmits the relevant information to the application service process by calling the callback function corresponding to the sending interface function.

[0121] Among them, the sending interface function is, for example, Fig.12sendCurrentTvInputId (sends the unique identifier of the current TV input source. In the Live TV application, users may switch between different input sources, such as HDMI input, cable TV, satellite TV, or streaming services. Sending the ID of the current input source ensures that the application knows and displays the correct input source information), sendCurrentChannelUri (sends the URI (Uniform Resource Identifier) ​​of the currently playing channel. The URI can be a link to the channel stream address, which can be a live network stream, IPTV service, or other form of streaming media), sendCurrentChannelLcn (sends the logical channel number of the currently playing channel. The logical channel number usually corresponds to the channel number that the user sees in the TV guide or channel list), sendCurrentVideoBounds (sends the boundary information of the current video playback area), sendStreamVolume (used to send the volume level of the current streaming media playback), sendTrackInfoList (used to send the audio track and video track information list of the currently playing content). The callback function corresponding to the sending interface function is, for example, Fig.12 onCurrentTvInputId, onCurrentChannelUri, onCurrentChannelLcn, onCurrentVideoBounds, onStreamVolume, and onTrackInfoList.

[0122] In some embodiments, the live broadcast application can call the interface related to TvView to perform operations related to TvView. The interfaces related to TvView may be, but are not limited to: getLeft, getTop, getRight, getBottom, layout, getTracks, getSelectedTrack, tune, selectTrack, where getLeft, getTop, getRight, and getBottom are used to obtain the position of TvView on the screen. Layout is used to set the position and size of TvView, and can be used to initialize the size of TvView or change the size of TvView. GetTracks is used to obtain the available tracks in the video stream (such as audio tracks, subtitle tracks, etc.), getSelectedTrack is used to obtain the currently selected track, and selectTrack is used to select a specific track. In the live broadcast application, when the user wants to change the audio language or enable / disable subtitles, selectTrack is triggered to change the audio language or enable / disable subtitles. Tune is used to tune to a specific channel or program to change the currently playing TV program. The live broadcast application can call the interface related to TvInteractiveAppView to perform operations related to TvInteractiveAppView.

[0123] Of course, the application service process can also modify the volume, modify the live video window size, switch audiotrack, switch videotrack. The application service process can also support stop av, that is, stop audio and video (implemented by trackid = null), switch channels, such as switching the previous channel and the next channel, and support freeze and unfreeze (freeze is achieved by adding the freeze parameter to stop, and unfreeze is achieved by adding the parameters of the current channel to tune). The application service process can also support the sound effect adjustment function to achieve high and low bass and sound balance adjustment, for example, by adding the SET_AUDIO_PROPERTY type and carrying specific adjustment parameters.

[0124] In this embodiment, the communication between the live broadcast application process and the application service process is realized through the target interface function and the target callback function. The live broadcast application process and the application service process are isolated through the interactive service process, which decouples the live broadcast application and the Ginga service to a certain extent, thereby improving the reusability of the Ginga service.

[0125] In some embodiments, when the controller executes triggering the data service process to obtain a filter identifier through the application service process, it is configured to: send a second data acquisition request carrying a media asset data type to the interactive service process through the application service process, and the second data acquisition request carries a media asset data type parameter; when the interactive service process receives the second data acquisition request, trigger the interactive service process to call the data acquisition callback function registered to the interactive service process in the application management component to send a third data acquisition request to the data service process, and the third data acquisition request carries a media asset data type parameter; when the data service process receives the third data acquisition request, trigger the data service process to obtain the filter identifier.

[0126] The application management component is, for example, the TIAM mentioned above. In response to the calling of the data acquisition callback function, the interactive management component causes the data service process to send a third data acquisition request, wherein the third data acquisition request carries a media asset data type parameter.

[0127] In this embodiment, the data service process and the application service process are isolated through the interactive service process, which decouples the live broadcast application and the Ginga service to a certain extent, thereby improving the reusability of the Ginga service.

[0128] In some embodiments, the playback operation is triggered through an interface provided by the live broadcast application, and the controller is further configured to: establish a communication connection between the live broadcast application process and the application service process when the live broadcast application process and the application service process have been started; in response to an input interactive application menu display instruction, control the display to display the interactive application menu interface, and display an interactive application list entry in the interactive application menu interface; in response to a trigger operation on the interactive application list entry, control the display to display the interactive application list, wherein the interactive application list is obtained by the live broadcast application process from the application service process through the communication connection.

[0129] Among them, the interactive application menu display instruction is used to instruct the display of the interactive application menu interface, such as the Ginga menu interface. The interactive application menu display instruction is, for example, a Ginga menu display instruction, and the Ginga menu display instruction can be input through a remote control or a mobile device, and can be a voice instruction or an instruction input through a remote control. The interactive application list may include icons and names of interactive applications. The interactive application may also be called a Ginga application. The communication connection between the live application process and the application service process may be an aidl service link.

[0130] Specifically, the controller obtains the interactive application list in response to the trigger operation on the interactive application list entry, displays the Ginga application list interface, and displays the interactive application list in the Ginga application list interface. When the icon of the interactive application is selected, the application details of the interactive application can also be displayed in the Ginga application list interface.

[0131] In some embodiments, the live broadcast application process sends an application list acquisition request to the application service process through the aidl communication connection, and receives an interactive application list returned by the application service process in response to the application list acquisition request, where the interactive application list is returned through the aidl communication connection.

[0132] like Fig.13 As shown, an interactive application menu interface, namely, Ginga menu interface 1300, is provided. The Ginga menu interface includes a Ginga setting item 1302, an automatic run item 1304, an interactive application list entry 1306, and a WebSocket client list entry 1308. The WebSocket client is a technology that allows different applications to communicate and interact over a network. Through the Webservices Registry, users can register, search, and call available WebSocket clients, thereby enabling data exchange and function calls between different applications. In response to a trigger operation on the interactive application list entry 1306, the controller displays a Ginga application list interface, in which an interactive application list is displayed, such as Fig.13 As shown, a Ginga application list interface 1400 is provided, and icons corresponding to multiple Ginga applications are displayed in the Ginga application list interface 1400. The multiple Ginga applications are application 1 to application 7, and application 1 to application 7 correspond to icons 1 to icon 7 in sequence. Application 1 is selected in the figure, and the application details of application 1 are displayed in the figure. The application details include description information, version and size, and the unit of size is MB (Megabyte).

[0133] In this embodiment, a communication connection is established between the live broadcast application process and the application service process. Through the communication connection, the live broadcast application process and the application service process can communicate directly, thereby improving communication efficiency.

[0134] In some embodiments, the interactive application list includes icons corresponding to the interactive applications, and the controller is further configured to: in response to an application stop operation triggered by the icon of the interactive application in the interactive application list, trigger the live broadcast application process to send an application stop request for the interactive application to the application service process through a communication connection; when the application service process receives the application stop request, trigger the application service process to stop running the interactive application, and return a status update notification corresponding to the interactive application to the live broadcast application process through the communication connection, wherein the status update notification is used to indicate that the interactive application has stopped running; in response to the status update notification, update the display status of the icon of the interactive application in the interactive application list, wherein, after the update, the display status of the icon indicates that the interactive application has stopped running.

[0135] In some embodiments, the Ginga menu interface also displays a WebSocket client list entry, and the controller is further configured to: in response to a trigger operation on the WebSocket client list entry, control the display to display the WebSocket client list, wherein the WebSocket client list is obtained from the application service process through the aidl communication connection. For example, the live broadcast application process can send a WebSocket client list acquisition request to the application service process; and receive the WebSocket client list returned by the application service process in response to the WebSocket client list acquisition request, wherein the WebSocket client list is returned through the aidl communication connection.

[0136] In some embodiments, the live broadcast application process is further configured to: in response to a setup operation triggered by an option of a target WebSocket client in the WebSocket client list, send a setup request for the target WebSocket client to the application service process through an aidl communication connection, and the setup request matches the setup operation; receive a setup success result returned by the application service process in response to the setup request, and the setup success result is returned through the aidl communication connection; based on the setup success result, update and display the WebSocket client list. The setup operation includes but is not limited to operations such as modifying the authorization status or deleting the WebSocket client, and the authorization status is divided into authorized and unauthorized. Modifying trust refers to modifying the authorization status. Authorized means trusting the WebSocket client, and unauthorized means not trusting the WebSocket client.

[0137] like Fig.14As shown, a WebSocket client list 1500 is displayed, and the WebSocket client list 1500 includes the names of multiple registered WebSocket clients, such as Client name1 to Client name3. The live broadcast application process responds to the trigger operation for the name of the WebSocket client and displays the service setting interface corresponding to the WebSocket client, such as Fig.14 As shown, when Client name1 is clicked, a service setting interface 1600 is displayed, and the service setting interface 1600 includes an Authorization switch and a Remove Entry item. When the Authorization switch is turned on, it means that the WebSocket client represented by Client name1 is trusted. In response to the trigger operation for the Remove Entry item, the live application process displays a removal confirmation window 1700. In response to the trigger operation for the confirmation control in the removal confirmation window 1700, the live application process removes Client name1 from the WebSocket client list.

[0138] like Fig.15As shown, a timing diagram of communication between the application service process and the live application process through an aidl communication connection is shown, including: when the Ginga service is ready, the application service process notifies the live application process of the service state change through the TIAS module and the TIAM module, wherein "service ready" means that the Ginga service has been started and is ready to process requests, and SERVICE_STATE_READY is a specific state transition used to characterize that the Ginga service has changed from a state to a ready state. When the live application process determines that the Ginga service is ready, it establishes an aidl communication connection with the application service process. The aidl communication connection can also be called an aidl service link. A successful link indicates that the aidl communication connection has been successfully established. The live application process can set application state monitoring and platform attribute monitoring for the application service process through the aidl communication connection. Setting application state monitoring is used to set a method for monitoring the state of interactive applications in the application service process, so that the application service process monitors and feeds back the state of interactive applications to the live application process. Setting platform attribute monitoring is used to set an attribute monitoring method for monitoring platform attributes in the application service process. The application service process can monitor the platform attributes from the live application process through the attribute monitoring method. Among them, the platform attribute refers to the attribute of the display device, and the platform attribute can be, but is not limited to, resolution or encoding and decoding mode. The live application process can obtain the application list, i.e., the interactive application list, from the application service process through the aidl service link. When the application service process downloads a certain interactive application, it can return the application status indicating that the application download is complete to the live application process. At this time, the live application process can obtain a new application list, i.e., the interactive application list, to update the interactive application list. When the user triggers the operation of starting / stopping the application (interactive application), the live application process notifies the application service process to start / stop the application (interactive application) through the aidl service link. After starting / stopping the application (interactive application), the application service process can send a notification of application status update STARTED / STOPPED to the live application process through the aidl service link. The STARTED status indicates that the application has been started and is running, and the STOPPED status indicates that the application has stopped running. The live application process can also obtain the WebSocket client list from the application service process through the aidl service link. When the user triggers the setting operation of the WebSocket client, the live application process sends the corresponding setting request to the application service process through the aidl service link to set the WebSocket client. In the figure, wscClient refers to WebSocket Client.

[0139] In this embodiment, the application stop request is directly transmitted through the communication connection between the live broadcast application process and the application service process, thereby improving communication efficiency.

[0140] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0141] Based on the same inventive concept, the embodiment of the present application also provides a content display device for implementing the above-mentioned content display method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method. The specific limitations can be found in the above-mentioned limitations on the content display method, which will not be repeated here.

[0142] In some embodiments, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned content display method are implemented.

[0143] In some embodiments, the present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned content display method when executed by a processor.

[0144] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0146] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A display device, characterized in that: include: Display and controller; The controller is configured to: When receiving a play operation of a live program, obtaining an interactive application data packet from the broadcast information of the live program, and starting an interactive application based on the interactive application data packet; During the process of starting the interactive application, when a first data acquisition instruction is detected and the first data acquisition instruction carries a media asset data type parameter, a data service process is triggered by an application service process to acquire a filter identifier, the filter identifier is used to find a target filter, and the target filter is used to acquire media asset data corresponding to the media asset data type parameter from broadcast information; When the data service process obtains the filter identifier, the data service process is triggered to call a response callback function of the application service process to transmit the filter identifier to the application service process; When the application service process receives the filter identifier, triggering the application service process to obtain the media asset data corresponding to the media asset data type parameter from the target filter corresponding to the filter identifier; According to the media data display mode preset by the interactive application, the display is controlled to display the media data corresponding to the media data type parameter.

2. The display device according to claim 1, characterized in that When the controller performs the playback operation of the live program upon receiving the playback operation of the live program, and acquires the interactive application data packet from the broadcast information of the live program, the controller is configured to: When receiving a play operation of a live program, starting an application service process, and sending an application data acquisition request to the data service process through the application service process, wherein the application data acquisition request carries a service data type parameter; When the data service process receives the data acquisition request, the data service process is triggered to acquire the service data in the broadcast information of the live program from the filter corresponding to the service data type parameter, and transmit the service data to the application service process; When the application service process receives the service data, the application service process is triggered to obtain the interactive application data packet from the service data.

3. The display device according to claim 2, characterized in that The business data includes an application information table and an application data table. When the controller executes the step of triggering the application service process to obtain the interactive application data packet from the business data when the application service process receives the business data, the controller is configured to: When the application service process receives the service data, the application service process is triggered to obtain an application information table from the service data, and obtain an interactive application identifier from the application information table; When the application service process obtains the interactive application identifier, the application service process is triggered to query the application data table to obtain an interactive application data packet corresponding to the interactive application identifier.

4. The display device according to claim 1, characterized in that When the controller executes the step of acquiring the interactive application data packet from the broadcast information of the live program and starting the interactive application based on the interactive application data packet, the controller is configured as follows: In the case of starting a live broadcast application process, obtaining application description information from broadcast information of the live broadcast program through the live broadcast application process; When the application description information indicates that there is an interactive application in the broadcast information of the live program, sending a service start request to the application management component through the live application; Responding to the service start request through the application management component, and instructing the interactive service process to start the application service process; When the application service process is started, the interactive application data packet is obtained from the broadcast information of the live program through the application service process, and the interactive application is started based on the interactive application data packet.

5. The display device according to claim 1, characterized in that When the controller executes the media data display mode preset according to the interactive application and controls the display to display the media data corresponding to the media data type parameter, the controller is configured as follows: Determining a media resource data display area according to a media resource data display method preset by the interactive application; Controlling the display to display the media resource playback window and the interactive interface of the interactive application in layers, and displaying the media resource data corresponding to the media resource data type parameter in the media resource playback window; The media asset data is displayed in the media asset data display area, and the interactive interface is displayed on an upper layer of the media asset playback window, and the interactive interface does not block the media asset playback window.

6. The display device according to claim 5, characterized in that The media resource playback window is provided by a live broadcast application, the interactive interface includes interactive controls for the media resource playback window, and the controller is further configured to: In the case where a trigger operation for the interactive control is detected, calling a target interface function corresponding to the trigger operation in the interactive service process through the application service process; When the interactive service process receives a call to the target interface function, a callback request for a target callback function is initiated to the live broadcast application process through the interactive service process, wherein the target callback function is a callback function corresponding to the target interface function; When the live broadcast application process receives the callback request, the live broadcast application process performs processing corresponding to the trigger operation on the media asset playback window.

7. The display device according to claim 1, characterized in that When the controller executes the step of triggering the data service process to obtain the filter identifier through the application service process, the controller is configured to: Sending a second data acquisition request carrying a media asset data type to the interactive service process through the application service process, wherein the second data acquisition request carries a parameter of the media asset data type; When the interactive service process receives the second data acquisition request, the interactive service process is triggered to call the data acquisition callback function registered to the interactive service process in the application management component, so as to send a third data acquisition request to the data service process, wherein the third data acquisition request carries the media asset data type parameter; When the data service process receives the third data acquisition request, the data service process is triggered to acquire a filter identifier.

8. The display device according to claim 1, characterized in that The playback operation is triggered by an interface provided by the live broadcast application, and the controller is further configured to: When the live broadcast application process and the application service process have been started, establishing a communication connection between the live broadcast application process and the application service process; In response to the input interactive application menu display instruction, controlling the display to display an interactive application menu interface, and displaying an interactive application list entry in the interactive application menu interface; In response to a trigger operation on the interactive application list entry, the display is controlled to display the interactive application list, wherein the interactive application list is obtained by the live application process from the application service process through the communication connection.

9. The display device according to claim 8, characterized in that The interactive application list includes icons corresponding to the interactive applications, and the controller is further configured to: In response to an application stop operation triggered by the icon of the interactive application in the interactive application list, triggering the live broadcast application process to send an application stop request for the interactive application to the application service process through the communication connection; When the application service process receives the application stop request, the application service process is triggered to stop running the interactive application, and a status update notification corresponding to the interactive application is returned to the live broadcast application process through the communication connection, where the status update notification is used to indicate that the interactive application has stopped running; In response to the status update notification, the display status of the icon of the interactive application is updated in the interactive application list, wherein after the update, the display status of the icon indicates that the interactive application has stopped running.

10. A content display method, characterized in that: Applied to a display device, the method comprises: When receiving a play operation of a live program, obtaining an interactive application data packet from the broadcast information of the live program, and starting an interactive application based on the interactive application data packet; During the process of starting the interactive application, when a first data acquisition instruction is detected and the first data acquisition instruction carries a media asset data type parameter, a data service process is triggered by an application service process to acquire a filter identifier, the filter identifier is used to find a target filter, and the target filter is used to acquire media asset data corresponding to the media asset data type parameter from broadcast information; When the data service process obtains the filter identifier, the data service process is triggered to call a response callback function of the application service process to transmit the filter identifier to the application service process; When the application service process receives the filter identifier, triggering the application service process to obtain the media asset data corresponding to the media asset data type parameter from the target filter corresponding to the filter identifier; The media data corresponding to the media data type parameter is displayed according to the media data display mode preset by the interactive application.

Images