Display devices, media playback devices, and media playback methods

Abstract

This application discloses a display device, a media asset playback apparatus, and a media asset playback method, comprising: acquiring a media display description document for a media asset file; acquiring a first description tag and a first channel tag based on the media display description document, wherein the first description tag is used to characterize that the media asset file includes extended layer data, and the extended layer data is used to characterize that the video encoding format can decode the media asset file to obtain ultra-high definition resolution; inserting the first description tag into the first channel tag to convert the first description tag into a second channel tag; establishing a first media asset track based on the first channel tag, and establishing a second media asset track based on the second channel tag; and synchronously decoding the first media asset track and the second media asset track to play the media asset file. This application enables the display device to support the operation of SHVC format media asset files and can simultaneously demultiplex the base layer data and extended layer data, so that the display device can play ultra-high definition resolution media asset files, thereby improving the user experience.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device, a media asset playback device, and a media asset playback method. Background Technology

[0002] With the diversification of display devices, they need to adapt to the needs of different users to cope with more scenarios. In scenarios where multiple people are conducting video conferences simultaneously, different users have different requirements for video bitrate and resolution when sending video files from the same video conferencing terminal to multiple user terminals.

[0003] Scalable High-Efficiency Video Coding (SVC for HEVC, SHVC) is a high-efficiency video coding format applicable to the H.265 / HEVC coding standard that enables temporal, spatial, and quality scalability. It represents the mainstream trend in future video coding. In the aforementioned scenarios, SHVC format video files can meet the different requirements of various users for video bitrate and resolution, and SHVC format video files can be decoded to obtain ultra-high-definition resolution video files.

[0004] However, current display devices do not support ultra-high-definition resolution parsing of SHVC format video files, resulting in the inability to provide ultra-high-definition resolution video files even when network conditions are met. Summary of the Invention

[0005] This application provides a display device, a media asset playback apparatus, and a media asset playback method, which enables the display device to support the operation of SHVC format media asset files and can simultaneously demultiplex base layer data and extended layer data, so that the display device can play ultra-high definition resolution media asset files and improve the user experience.

[0006] In a first aspect, this application discloses a display device, comprising: a display; and a controller configured to: receive a control command sent by a user for acquiring a media asset file; in response to the control command, acquire a media display description file for the media asset file, the media display description file being used to determine the video encoding format contained in the media asset file; acquire a first description tag based on the media display description file, the first description tag being used to characterize that the media asset file includes extended layer data, the extended layer data being used to characterize that the video encoding format is decodeable to obtain an ultra-high definition resolution media asset file; insert the first description tag into a first channel tag, so that the first description tag is converted into a second channel tag; establish a first media asset track based on a first media stream corresponding to the first channel tag, and establish a second media asset track based on a second media stream corresponding to the second channel tag; wherein the first media stream and the second media stream are respectively used to encode different versions of the media asset file; and synchronously activate the first media asset track and the second media track, so that the first media stream and the second media stream are synchronously decoded to play the media asset file. Using this implementation method, the display device can modify the media display specification file, establish a first media asset track and a second media asset track based on the modified media display specification file, demultiplex the first media asset track and the second media asset track respectively, and finally merge the first media stream and the second media stream in the first media asset track and the second media asset track, and decode them at the same time, so that the display device can play media asset files with ultra-high resolution and improve the user experience.

[0007] Secondly, this application discloses a media asset playback device applied to a display device as described in the first aspect and its embodiments, comprising: a data source element, the data source element being used to acquire a media display description file of a media asset file, the media display description file being used to determine the video encoding format contained in the media asset file; a first demultiplexing element, the first demultiplexing element being used to acquire a first description tag according to the media display description file, the first description tag being used to characterize that the media asset file includes extended layer data, the extended layer data being used to characterize a media asset file whose video encoding format can be decoded to obtain ultra-high definition resolution; inserting the first description tag into a first channel tag, so that the first description tag is converted into a second channel tag; a second demultiplexing element; and a second demultiplexing element. The media asset playback device employs several components: a second demultiplexing component to establish a first media asset track based on the first media stream corresponding to the first channel tag, and a second media asset track based on the second media stream corresponding to the second channel tag; wherein the first and second media streams are used to encode different versions of media asset files; an input selection component to synchronously activate the first and second media asset tracks; a receiver component to merge the first and second media streams according to a predetermined time standard setting principle and send the first and second media streams to a decoding component; and a decoding component to receive the first and second media streams to synchronously decode them. Using this embodiment, the media asset playback device can modify the media display specification file, establish the first and second media asset tracks based on the modified file, demultiplex the first and second media asset tracks respectively, and finally merge the first and second media streams in the first and second media asset tracks while simultaneously decoding them, enabling the display device to play ultra-high-definition media asset files and improving the user experience.

[0008] Thirdly, this application also discloses a media asset playback method, comprising: obtaining a media display description file of a media asset file, the media display description file being used to determine the video encoding format contained in the media asset file; obtaining a first description tag according to the media display description file, the first description tag being used to characterize that the media asset file includes extended layer data, the extended layer data being used to characterize that the video encoding format can be decoded to obtain an ultra-high definition resolution media asset file; inserting the first description tag into a first channel tag, so that the first description tag is converted into a second channel tag; establishing a first media asset track according to a first media stream corresponding to the first channel tag, and establishing a second media asset track according to a second media stream corresponding to the second channel tag; wherein the first media stream and the second media stream are respectively used to encode different versions of the media asset file; and synchronously activating the first media asset track and the second media asset track, so that the first media stream and the second media stream are synchronously decoded to play the media asset file. Using this implementation method, the media asset playback method can modify the media display specification file, establish a first media asset track and a second media asset track based on the modified media display specification file, then demultiplex the first media asset track and the second media asset track respectively, and finally merge the first media stream and the second media stream in the first media asset track and the second media asset track, and simultaneously decode them, so that the display device can play media asset files with ultra-high resolution, thereby improving the user experience.

[0009] The technical solutions shown above enable display devices to support the operation of SHVC format media asset files. They can simultaneously demultiplex the base layer data and the extended layer data, allowing the display devices to play ultra-high-definition resolution media asset files and improve the user experience. Attached Figure Description

[0010] Figure 1 An exemplary schematic diagram illustrates an operational scenario between a display device and a control device according to an embodiment of this application;

[0011] Figure 2 An exemplary configuration block diagram of a control device 100 according to an embodiment of this application is shown;

[0012] Figure 3 An exemplary hardware configuration block diagram of a display device 200 according to an embodiment of this application is shown;

[0013] Figure 4 An exemplary software configuration block diagram of a display device 200 according to an embodiment of this application is shown;

[0014] Figure 5 An exemplary diagram illustrating a usage scenario of an embodiment of this application is shown;

[0015] Figure 6 An exemplary diagram of the GStreamer playback pipeline is shown;

[0016] Figure 7 An exemplary schematic diagram illustrates the process of a display device playing an SHVC format media asset file;

[0017] Figure 8 An exemplary flowchart illustrating the display device configuration of an embodiment of this application is shown;

[0018] Figure 9 An exemplary diagram of the structure of an MPD file is shown.

[0019] Figure 10 An exemplary flowchart illustrating the display device configuration of an embodiment of this application is shown;

[0020] Figure 11 An exemplary flowchart illustrating the display device configuration of an embodiment of this application is shown;

[0021] Figure 12 An exemplary schematic diagram of a media asset playback device according to an embodiment of this application is shown. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings.

[0023] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0024] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0025] To facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art, the technical terms involved in the embodiments of this application will be explained below.

[0026] 1. Scalable Video Coding (SVC) can solve the problem that the encoder needs to encode multiple times or the server needs to encode and decode twice when facing different network conditions, different terminal processing capabilities, and different user quality requirements. It can achieve scalability in the temporal domain, spatial domain, and quality domain. That is, a single encoding can produce video compressed bitstreams with different frame rates, resolutions, and image qualities. The decoder can adaptively adjust such video compressed bitstreams, reducing the computational burden on the encoder and server.

[0027] 2. High Efficiency Video Coding (HEVC) is a new video compression standard that extends the H.264 / AVC encoding standard. It enables the actual encoded image to be larger, such as from 2k to 4k, or from 4k to 8k resolution, so that the playback speed of full HD video is faster.

[0028] 3. Scalable High-Efficiency Video Coding (SVC for HEVC, SHVC) is a high-efficiency video coding format applicable to the H.265 / HEVC coding standard, capable of achieving temporal, spatial, and quality scalability. For ease of description, H.265 / HEVC will be referred to as HEVC below.

[0029] The display device provided in this application can have various implementation forms, such as a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figure 1 and Figure 2 This is one specific embodiment of the display device of this application.

[0030] Figure 1 This is a schematic diagram illustrating the operational scenario between the display device and the control unit according to the embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0031] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0032] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, an application running on the smart device may be used to control the display device 200.

[0033] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0034] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0035] Figure 2 An exemplary block diagram of the configuration of the control device 100 according to an exemplary embodiment is shown. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0036] Figure 3 A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown.

[0037] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0038] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first to an nth interface for input / output.

[0039] In some embodiments, the display 260 includes a display screen component for presenting an image, a driving component for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0040] In some embodiments, the display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0041] In some embodiments, the communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0042] In some embodiments, the user interface can be used to receive control signals from the control device 100 (e.g., an infrared remote control).

[0043] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0044] In some embodiments, the external device interface 240 may include, but is not limited to, one or more interfaces such as: High Definition Multimedia Interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0045] In some embodiments, the tuner 210 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0046] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0047] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations via various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0048] In some embodiments, the object can be any of the optional objects, such as a hyperlink, an icon, or other operable controls. Operations related to the selected object include: displaying links to hyperlinked pages, documents, images, etc., or performing operations corresponding to the program associated with the icon.

[0049] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0050] A CPU (CPU) processor is used to execute operating system and application instructions stored in memory, as well as various interactive instructions received from external input, to execute various applications, data, and content, ultimately for the display and playback of various audio and video content. A CPU processor can include multiple processors, such as a main processor and one or more sub-processors.

[0051] In some embodiments, a graphics processor is used to generate various graphical objects, such as icons, operation menus, and graphics displayed based on user input commands. The graphics processor includes an arithmetic logic unit (ALU) that performs calculations based on various user-input interactive commands and displays various objects according to display attributes; it also includes a renderer that renders the various objects obtained from the ALU, and the rendered objects are used to display on a monitor.

[0052] In some embodiments, the video processor is configured to receive external video signals and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard encoding and decoding protocol of the input signals, so as to obtain a signal that can be directly displayed or played on the display device 200.

[0053] In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image compositing module, a frame rate conversion module, and a display formatting module. The demultiplexing module demultiplexes the input audio and video data streams. The video decoding module processes the demultiplexed video signal, including decoding and scaling. The image compositing module, such as an image synthesizer, overlays and blends a GUI signal generated by a graphics generator based on user input or its own generation with the scaled video image to generate a displayable image signal. The frame rate conversion module converts the input video frame rate. The display formatting module modifies the received frame rate-converted video output signal to conform to a display format, such as outputting RGB data signals.

[0054] In some embodiments, the audio processor is configured to receive external audio signals, and according to the standard codec protocol of the input signals, perform decompression and decoding, as well as noise reduction, digital-to-analog conversion, and amplification processing, to obtain a sound signal that can be played in a speaker.

[0055] In some embodiments, the user can input user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0056] In some embodiments, a "user interface" is the medium through which an application or operating system interacts and exchanges information with a user, converting information between its internal form and a form acceptable to the user. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0057] In some embodiments, the display device's system may include a kernel, a command interpreter (shell), a file system, and applications. The kernel, shell, and file system together form the basic operating system structure, allowing users to manage files, run programs, and use the system. Upon power-up, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, and runs and maintains virtual memory, the scheduler, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. Applications are compiled into machine code after startup, forming a process.

[0058] like Figure 4 As shown, a display device's system can include a kernel, a command interpreter (shell), a file system, and applications. The kernel, shell, and file system together form the basic operating system structure, allowing users to manage files, run programs, and use the system. Upon power-up, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, and runs and maintains virtual memory, the scheduler, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. Applications are compiled into machine code after startup, forming a process.

[0059] like Figure 4 As shown, the display device system is divided into three layers, from top to bottom: the application layer, the middleware layer, and the hardware layer.

[0060] The application layer mainly includes commonly used applications on TVs, as well as the application framework. The commonly used applications are mainly browser-based applications, such as HTML5 apps, and native apps.

[0061] An application framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interface for these functions (toolbar, status bar, menu, dialog box).

[0062] Native apps can support online or offline access, push notifications, or access to local resources.

[0063] The middleware layer includes various television protocols, multimedia protocols, and system components. Middleware can use the basic services (functions) provided by system software to connect different parts of application systems or different applications on the network, achieving resource sharing and function sharing.

[0064] The hardware layer mainly includes the HAL interface, hardware, and drivers. The HAL interface is a unified interface for all TV chips, with the specific logic implemented by each chip. The drivers mainly include: audio drivers, display drivers, Bluetooth drivers, camera drivers, Wi-Fi drivers, USB drivers, HDMI drivers, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.), and power drivers.

[0065] SVC has a wide range of applications, such as Figure 5 As shown, taking a multi-user video conference as an example, the video conferencing terminal uses SVC (Simultaneous Visual C++) for encoding, generating compressed video streams with different frame rates, resolutions, and image qualities. These compressed video streams are then distributed by a Multi-Control Unit (MCU) to achieve signal aggregation and switching between multiple user terminals. Each user terminal decodes the SVC-encoded compressed video stream, resulting in videos at different resolutions. For example, if the first user terminal has a poor network environment, 360p resolution can be selected for decoding to ensure smooth video conferencing; if the second user terminal has a better network environment, 1080p resolution can be selected for decoding to improve the video conferencing experience.

[0066] In related technologies, 360p, 720p, and 1080p resolution videos can be encoded and decoded using H.264 / AVC video encoding technology. However, H.264 / AVC video encoding technology suffers from low compression efficiency and high bandwidth consumption, making it difficult to support ultra-high-definition resolution videos such as 4K and 8K, thus preventing users from watching ultra-high-definition videos. HEVC video encoding technology can solve this problem. HEVC video encoding technology can improve compression efficiency, reduce bandwidth consumption, and support resolution changes from 2K to 4K, or from 4K to 8K. Therefore, it is essential for display devices to be configured to support HEVC format encoding and decoding. Furthermore, since different users have different requirements for video quality, in addition to HEVC, other technologies may also be used for video encoding and decoding. Figure 5 In the scenario shown, the display device should also be configured to support SHVC format media files. This way, when different users are decoding the same video compression stream, they can choose the resolution they need according to their actual situation. They can choose ultra-high-definition videos with larger resolutions such as 4K and 8K on top of 1080p.

[0067] Figure 6An exemplary diagram of a GStreamer playback pipeline is shown. GStreamer is an open-source multimedia framework for building streaming media applications, simplifying the development of audio / video applications. It can handle multimedia data in various formats such as MP3, Ogg, MPEG1, MPEG2, and AVI. GStreamer operates on a plugin-based model, where some plugins provide diverse multimedia digital signal codecs, while others offer additional functionalities. Any plugin can be linked into a defined data stream pipeline. Figure 6 As shown, the GStreamer playback pipeline includes a Source element for reading data from the pipeline; a Dash Demux element for initial data processing and splitting; a Buffer element for buffering the split data processed by the Dash Demux element; a Media Segment Demux element for further demultiplexing; an Input Select element for processing the data input from the Media Segment Demux element, selecting the input data, and discarding useless data; and a Sink element for accepting input video and audio data and sending the video and audio data to the Decode element respectively to complete the decoding function.

[0068] Figure 7 An exemplary diagram illustrates the process of a display device playing an SHVC format media asset file through a GStreamer playback pipeline. Figure 7As shown, when the display device receives a control command from the user to obtain a media asset file in SHVC format, in response to the control command, the data source element retrieves the SHVC format media asset file from local storage or a server. This media asset file includes multiple layers of data to meet the user's different quality requirements for the media asset file. For example, this format media asset file includes a base layer data and one or more enhancement layer data. The base layer data carries video data with a basic quality level, such as video that can be played at resolutions such as 360p, 720p, and 1080p. One or more enhancement layer data can carry additional video data to support higher spatial, temporal, or signal-to-noise ratio levels, such as video that can be played at ultra-high-definition resolutions such as 4K and 8K. It should be noted that the enhancement layer data cannot be decoded independently, but needs to be decoded based on the base layer data. When the data source element acquires the SHVC format media asset file, the demultiplexing element demultiplexes the data in that format. However, the existing demultiplexing element can only demultiplex the data at one level. Therefore, it can only demultiplex the basic layer data and cannot simultaneously demultiplex the data at two levels, such as the extended layer data that relies on the basic layer data. This results in the display device only being able to provide the resolution of the basic layer data during the playback of the media asset file. Even when network conditions are met, it still cannot provide the ultra-high-definition resolution supported by the extended layer data.

[0069] To address the aforementioned issues, the technical solution presented in this application enables display devices to support the execution of SHVC format media asset files. It can simultaneously demultiplex base layer data and extended layer data, allowing display devices to play ultra-high-definition resolution media asset files and improve user experience.

[0070] This application discloses a display device, including: a display, a controller; configured to perform such... Figure 8 The steps are shown.

[0071] The display device receives control commands from the user to retrieve media asset files. In response to these commands, it retrieves the Media Presentation Description (MPD) file from the media asset files. Specifically, the display device can retrieve the media asset files from local storage or a server via a control device. The media asset files are typically multiple segments of the same content with different bitrates and resolutions. Each segment corresponds to a Media Presentation Description (MPD) file, which is an XML file used to describe the bitrate of the corresponding segment. For example, the display device can retrieve multiple segments of an SHVC format media asset file. The first segment might correspond to an MPD file with a 1080p resolution and an Advanced Video Coding (AVC) bitrate, while the second segment might correspond to an MPD file with a 4K resolution and an HEVC bitrate.

[0072] Bitrate is used to characterize the amount of data used by a media asset file per unit of time. Different data formats of media assets have different bitrates; therefore, display devices can determine the data format of a media asset file based on the bitrate. After acquiring a media asset file, the display device obtains multiple MPD files. It iterates through these MPD files, determining whether each MPD file contains an HEVC format bitrate. If an MPD file does not contain an HEVC format bitrate, the media asset file is not capable of providing ultra-high definition resolution. The display device is backward compatible with such media asset files and directly decodes their data format for playback. If an MPD file contains an HEVC format bitrate, it indicates that the media asset file is capable of providing ultra-high definition resolution.

[0073] In practice, display devices can determine whether an MPD file contains an HEVC format stream based on the stream's structure. The typical HEVC stream structure is as follows: start code, Video Parameter Set (VPS), start code, Sequence Parameter Set (SPS), start code, Picture Parameter Set (PPS), Supplemental Enhancement Information (SEI), start code, difference frame, start code, reference frame, and so on. AVC format streams typically do not contain VPS. Therefore, the order of the stream structure can be used to determine if an MPD file contains an HEVC format stream. It's important to note that in the HEVC stream structure, VPS, SPS, and PPS are fixed and may not necessarily contain SEI. Then, difference frames, reference frames, and other video frames are arranged sequentially. When a display device reads this type of HEVC format stream, it determines that the media asset file can provide ultra-high-definition resolution.

[0074] Figure 9 An exemplary diagram illustrating the structure of an MPD file is shown. Figure 9 As shown, an MPD file consists of one or more periods, each period comprising one or more channel (Adaptationset) tags. Each channel tag includes one or more media content components, and each media content component can include multiple encoded versions. Each encoded version is called a media stream, and each media stream corresponds to encoding parameter attributes, including bitrate, resolution, and encoder type. For example, the media content components in a channel tag include at least one video component and at least one audio component. Taking one video component as an example, this video component can include a 360p SHVC format media stream, a 720p SHVC format media stream, a 1080p SHVC format media stream, a 4k SHVC format media stream, and an 8k SHVC format media stream. Each media stream corresponds to a Representation tag, and the display device can switch media streams based on the Representation tag.

[0075] It should be noted that the description tag includes a Uniform Resource Locator (URL). The display device can obtain the media stream from the server based on the URL, or it can obtain the media stream from the broadcast based on the Service List Table (SLT) tag. In related technologies, the GStreamer playback pipeline cannot demultiplex media stream data corresponding to multiple description tags simultaneously, but it can demultiplex media stream data corresponding to multiple channel tags simultaneously. This is because different bitrates and resolutions of media streams can be supported within the same channel tag, while the same description tag can only support media streams with one bitrate and resolution. Therefore, for SHVC format media asset files, both the base layer data and the extension layer data have description tags, and the display device cannot demultiplex the base layer data and the extension layer data simultaneously. This application improves the GStreamer playback pipeline based on this limitation.

[0076] Once the display device determines that the MPD file contains an HEVC format bitstream, it acquires at least one candidate description tag from the MPD file. The candidate description tag describes the bitstream, resolution, and encoder type of the MPD file. For example, the display device can acquire candidate description tags for describing a 360p SHVC format media stream, a 720p SHVC format media stream, a 1080p SHVC format media stream, a 4k SHVC format media stream, and an 8k SHVC format media stream.

[0077] After obtaining at least one candidate description tag in the MPD file, the display device iterates through all candidate description tags and determines whether any of them include a first identifier. The first identifier identifies that the candidate description tag includes extended layer data. If a candidate description tag includes the first identifier, it is identified as the first description tag. For example, the first identifier could be "lhe1". For example, the first identifier can be included in the candidate description tags of a 4k resolution SHVC format media stream and an 8k resolution SHVC format media stream, and the display device can identify such candidate description tags with the first identifier as the first identifier.

[0078] After the display device determines the candidate description tag as the first description tag, it stores the first description tag in the storage space (Buffer); the storage space is used to cache the first description tag; when the display device stores the first description tag in the storage space, it triggers a modification instruction for modifying the MPD file; in response to the modification instruction, the first description tag is inserted into the first channel tag, so that the description tag is transformed into the second channel tag, thereby generating the modified MPD file.

[0079] It should be noted that the display device stores other candidate description tags in the storage space at the same time as storing the first description tag. This is because the extended layer data corresponding to the first description tag cannot be decoded independently, but needs to rely on the basic layer data corresponding to other candidate description tags for decoding. This application embodiment only describes the operation process of the first description tag.

[0080] When the display device defaults to playing 4K resolution media, it executes the method described in this application embodiment without needing to obtain other instructions. When the display device defaults to playing media with a resolution obtained from the parsing of the base layer data (such as 360p, 720p, 1080p), the display device needs to receive a user's switching instruction for switching resolution before executing the method described in this application embodiment. In specific implementation, the configuration process of the display device can be as follows: Figure 10As shown, after the display device determines the candidate description tag as the first description tag, it performs the operation of storing the first description tag in the storage space. For example, after the display device obtains the first description tag of the SHVC format media stream with a resolution of 4k and the first description tag of the SHVC format media stream with a resolution of 8k, it stores the two first description tags in the storage space at the same time and inserts the corresponding first channel tag at the same time, so as to generate modified MPD files based on the MPD file used to describe the first description tag with a resolution of 4k and the modified MPD file used to describe the first description tag with a resolution of 8k, respectively. In response to the switching command, the modified MPD file corresponding to the switching command is selected. For example, if the user switches the resolution to 4k, the modified MPD file generated based on the MPD file used to describe the first description tag with a resolution of 4k is selected for subsequent operations.

[0081] In practice, the configuration process for the display device can also be as follows: Figure 11 As shown, after the display device determines the candidate description tag as the first description tag, it responds to the switching command, selects the first description tag corresponding to the switching command, and stores the first description tag corresponding to the switching command in the storage space. For example, if the user switches the resolution to 4k, the first description tag used to describe the resolution of 4k is selected and stored in the storage space, the first description tag used to describe the resolution of 4k is inserted into the first channel tag, and then subsequent operations are performed.

[0082] The display device parses the modified MPD file to obtain a first channel label and a second channel label; it establishes a first media asset track based on the first media stream corresponding to the first channel label, and establishes a second media asset track based on the second media stream corresponding to the second channel label; wherein, the first media stream can be obtained from local storage or a server, and the second media stream can be obtained from local storage or a server.

[0083] It should be noted that the base layer data and extended layer data in SHVC format are distinguished by description tags. Therefore, the GStreamer playback pipeline in related technologies cannot process the base layer data and extended layer data simultaneously. Furthermore, the first media stream in the base layer data and the second media stream in the extended layer data may have different sources, and media streams from different sources need to be processed in the same GStreamer playback pipeline. Based on this, the embodiments of this application insert the first description tag into the first channel tag, so that the first description tag is transformed into the second channel tag, and the first channel tag and the second channel tag are in the same GStreamer playback pipeline.

[0084] The display device establishes a first media asset track based on a first media stream corresponding to a first channel tag, and establishes a second media asset track based on a second media stream corresponding to a second channel tag; wherein the first media stream and the second media stream are used to encode different versions of media asset files. In this embodiment, the first media stream is used to encode base layer data, and the second media stream is used to encode extension layer data.

[0085] After the display device establishes a second media asset track based on the second media stream corresponding to the second channel tag, it registers a calling function in the second media asset track; and calls the second media stream according to the calling function so that the second media asset track is activated synchronously with the first media asset track.

[0086] It should be noted that the second media asset track established in this embodiment is for playing extended layer data. Extended layer data needs to be played on the basis of basic layer data. Therefore, after establishing the second media asset track, it is necessary to synchronize the second media asset track with the first media asset track.

[0087] In related technologies, when the first media asset track and the second media asset track are synchronized in the display device, if the second media asset track is not activated, the display device will directly discard the second media stream corresponding to the second media asset track. Therefore, in related technologies, even if the second media asset track is established, the display device cannot directly play media assets based on the second media asset track.

[0088] In some embodiments, the display device registers a calling function in the second media asset track to invoke a discarded second media stream. For example, the calling function could be: shvc_custorm_data_event.

[0089] Display devices can typically only activate the first media asset track. After the display device calls the second media stream according to the calling function, it can activate both the first and second media asset tracks simultaneously. When the first and second media asset tracks are activated synchronously, the first and second media streams are merged according to the Predetermined Times Standards (PTS) principle so that the first and second media streams are decoded synchronously to play the media asset file.

[0090] Figure 12 An exemplary schematic diagram of a media asset playback device according to an embodiment of this application is shown. This media asset playback device, applicable to the aforementioned display device, is a player device based on the GStreamer playback pipeline and includes:

[0091] The data source component is used to obtain the media display specification file of the media asset file. The media display specification file is used to determine the video encoding format contained in the media asset file.

[0092] The first demultiplexing element is used to obtain a first description tag and a first channel tag according to the media presentation specification document. The first description tag is used to characterize that the media asset file includes extended layer data, and the extended layer data is used to characterize that the video encoding format can decode the media asset file to obtain ultra-high definition resolution. The first description tag is inserted into the first channel tag to transform the first description tag into a second channel tag. For example, the first demultiplexing element in the embodiments of this application can be a Dash demultiplexing element.

[0093] In some embodiments, the first demultiplexing element is further configured to: determine whether the media asset display description file contains an efficient video encoding format; if the media asset display description file contains the efficient video encoding format, obtain at least one candidate description tag in the media asset display description file; the candidate description tag is used to describe the bitstream, resolution, and encoder type of the media asset display description file; if the media asset display description file does not contain an efficient video encoding format, decode the media asset display description file to play the media asset file; traverse the candidate description tags; determine whether the candidate description tags include a first identifier; the first identifier is used to identify that the candidate description tag includes extended layer data; if the candidate description tag includes the first identifier, determine the candidate description tag as the first description tag.

[0094] In some embodiments, the media asset playback device further includes: a buffer element; after the first demultiplexing element determines the candidate description tag as the first description tag, the first description tag is stored in the buffer element; when the first description tag is stored in the buffer element, the buffer element triggers a modification instruction for modifying the media asset display description file, and in response to the modification instruction, the first description tag is inserted into the first channel tag so that the first description tag is converted into the second channel tag.

[0095] It should be noted that while storing the first description tag in the buffer element, the media asset playback device also stores other candidate description tags in different areas of the buffer element. This is because the extended layer data corresponding to the first description tag cannot be decoded independently, but needs to rely on the base layer data corresponding to other candidate description tags for decoding. This application embodiment only describes the operation process of the first description tag.

[0096] The second demultiplexing element is used to establish a first media asset track based on the first media stream corresponding to the first channel tag, and to establish a second media asset track based on the second media stream corresponding to the second channel tag; wherein the first media stream and the second media stream are used to encode different versions of media asset files. For example, the second demultiplexing element in this application can be a media setting demultiplexing element. It should be noted that establishing the first media asset track based on the first media stream corresponding to the first channel tag requires a separate second demultiplexing element for demultiplexing, and establishing the second media asset track based on the second media stream corresponding to the second channel tag requires a separate second demultiplexing element for demultiplexing.

[0097] The input selection element is used to synchronously activate the first and second media tracks.

[0098] In some embodiments, the input selection element is further configured to register a call function and send the call function to the receiver element. It should be noted that when the two second demultiplexing elements demultiplex the first media stream and the second media stream respectively, and then input them to the input selection element at the same time, the display device activates the first media asset track where the first media stream is located, and calls the second media stream through the call function to activate the second media asset track where the second media stream is located.

[0099] A receiver element is configured to merge a first media stream and a second media stream according to a predetermined time standard setting principle; and send the first media stream and the second media stream to a decoding element; in some embodiments, the receiver element is further configured to, upon receiving a calling function, invoke the second media stream according to the calling function so that the second media asset track is synchronously activated with the first media asset track; when the first media asset track and the second media asset track are synchronously activated, merge the first media stream and the second media stream according to the predetermined time standard setting principle.

[0100] A decoding element receives a first media stream and a second media stream to synchronously decode the first media stream and the second media stream.

[0101] The specific implementation methods of each component in the embodiments of this application correspond to the specific implementation methods in the display device, and will not be described in detail here.

[0102] This application also illustrates a media asset playback method, including:

[0103] The process involves: obtaining a media display description file for the media asset file, which determines the video encoding format contained in the media asset file; obtaining a first description tag and a first channel tag based on the media display description file, whereby the first description tag indicates that the media asset file includes extended layer data, and the extended layer data indicates that the video encoding format can be decoded to obtain an ultra-high-definition resolution media asset file; inserting the first description tag into the first channel tag to transform the first description tag into a second channel tag; establishing a first media asset track based on the first media stream corresponding to the first channel tag, and establishing a second media asset track based on the second media stream corresponding to the second channel tag; wherein the first media stream and the second media stream are used to encode different versions of the media asset file; and synchronizing and activating the first and second media asset tracks to enable the first and second media streams to be decoded synchronously for playback of the media asset file.

[0104] The display device, media asset playback device, and media asset playback method shown above can modify the media display specification document, establish a first media asset track and a second media asset track based on the modified media display specification document, demultiplex the first media asset track and the second media asset track respectively, and finally merge the first media stream and the second media stream in the first media asset track and the second media track, and simultaneously decode them, so that the display device can play media asset files with ultra-high resolution, thereby improving the user experience.

[0105] The technical solutions illustrated in this application are backward compatible with playing media assets in other video encoding formats during specific implementation, demonstrating good compatibility.

[0106] It should be noted that the technical solutions shown above are applicable to the ROUTE (Real time Object delivery Over Unidirectional Transport) / DASH protocol in the ATSC 3.0 standard, and the technical solutions in this application include, but are not limited to, those applicable only to this protocol. In specific applications, the technical solutions shown in this application are universal and can also be applied to the field of real-time clocks. For example, integrating the media playback device using the technology shown in this application into a conferencing system allows users to view ultra-high-definition media materials with lower network bandwidth.

[0107] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A display device, characterized by comprising: include: monitor; The controller is configured as follows: Receive control commands sent by users to retrieve media asset files; In response to the control command, a media display description file of the media asset file is obtained, the media display description file being used to determine the video encoding format contained in the media asset file; According to the media presentation description document, a first description tag and a first channel tag are obtained. The first description tag is used to characterize that the media asset file includes extended layer data. The extended layer data is used to characterize that the video encoding format can be decoded to obtain an ultra-high definition resolution media asset file. Insert the first description label into the first channel label, so that the first description label is transformed into the second channel label; A first media asset track is established based on the first media stream corresponding to the first channel tag, and a second media asset track is established based on the second media stream corresponding to the second channel tag; wherein, the first media stream and the second media stream are respectively used to encode different versions of the media asset file; The first media asset track and the second media asset track are activated synchronously so that the first media stream and the second media stream are decoded synchronously to play the media asset file.

2. The display device of claim 1, wherein, The controller, which performs the step of obtaining the first description tag and the first channel tag according to the media display specification file, is further configured to: Determine whether the media display description file contains an efficient video encoding format; If the media presentation description file contains the high-efficiency video encoding format, obtain at least one candidate description tag from the media presentation description file; the candidate description tag is used to describe the bitstream, resolution, and encoder type of the media presentation description file. If the media presentation description file does not contain the high-efficiency video encoding format, the media presentation description file is decoded to play the media asset file.

3. The display device according to claim 2, characterized in that, The controller's step of retrieving at least one candidate description tag from the media display description file is further configured to: Iterate through the candidate description tags; Determine whether the candidate description tag includes a first identifier; the first identifier is used to identify that the candidate description tag includes extended layer data. If the candidate description label includes a first identifier, the candidate description label is determined as the first description label.

4. The display device according to claim 3, characterized in that, The controller is further configured to: Once the candidate description tag is determined to be the first description tag, the first description tag is stored in the storage space; the storage space is used to cache the first description tag. When the first description tag is stored in the storage space, a modification instruction for modifying the media display description file is triggered; In response to the modification instruction, the step of inserting the first description tag into the first channel tag to convert the first description tag into a second channel tag is performed to generate a modified media display description file.

5. The display device according to claim 4, characterized in that, The controller's steps of establishing a first media asset track based on the first media stream corresponding to the first channel tag and establishing a second media asset track based on the second media stream corresponding to the second channel tag are further configured as follows: Parse the modified media presentation specification file to obtain the first channel label and the second channel label; A first media asset track is established based on the first media stream corresponding to the first channel tag, and a second media asset track is established based on the second media stream corresponding to the second channel tag; wherein, the first media stream is obtained from local storage or a server, and the second media stream is obtained from local storage or a server.

6. The display device according to claim 5, characterized in that, The controller's step of synchronously activating the first media asset track and the second media asset track to enable synchronous decoding of the first media stream and the second media stream for playback of the media asset file is further configured to: After establishing the second media asset track based on the second media stream corresponding to the second channel label, register the calling function in the second media asset track; The second media stream is invoked according to the aforementioned function to synchronize the activation of the second media asset track with the first media asset track. When the first media asset track and the second media asset track are activated synchronously, the first media stream and the second media stream are merged according to a predetermined time standard setting principle so that the first media stream and the second media stream are decoded synchronously to play the media asset file.

7. A media asset playback device, characterized in that, Applied to the display device as described in any one of claims 1-6, comprising: A data source element, wherein the data source element is used to obtain a media display specification file of a media asset file, and the media display specification file is used to determine the video encoding format contained in the media asset file; A first demultiplexing element is used to obtain a first description tag and a first channel tag according to the media presentation specification file. The first description tag is used to characterize that the media asset file includes extended layer data, and the extended layer data is used to characterize that the video encoding format can decode the media asset file to obtain ultra-high definition resolution. The first description tag is inserted into the first channel tag to convert the first description tag into a second channel tag. The second demultiplexing element is used to establish a first media asset track based on the first media stream corresponding to the first channel tag, and to establish a second media asset track based on the second media stream corresponding to the second channel tag; wherein the first media stream and the second media stream are respectively used to encode different versions of the media asset file; An input selection element is used to synchronously activate the first media asset track and the second media asset track. A receiver element is configured to merge the first media stream and the second media stream according to a predetermined time standard setting principle; and send the first media stream and the second media stream to a decoding element; A decoding element receives the first media stream and the second media stream to synchronously decode the first media stream and the second media stream.

8. The media asset playback device according to claim 7, characterized in that, The first demultiplexing element is further configured to: determine whether the media presentation description file contains an efficient video encoding format; if the media presentation description file contains the efficient video encoding format, obtain at least one candidate description tag in the media presentation description file; the candidate description tag is used to describe the bitstream, resolution, and encoder type of the media presentation description file; If the media presentation description file does not contain the high-efficiency video encoding format, the media presentation description file will be decoded to play the media asset file; Traverse the candidate description tags; determine whether the candidate description tags include a first identifier; the first identifier is used to identify that the candidate description tags include extended layer data; if the candidate description tags include the first identifier, determine the candidate description tags as the first description tags.

9. The media asset playback device according to claim 7, characterized in that, The input selection element is also configured to register a calling function and send the calling function to the receiver element; The receiver element is further configured to, upon receiving the calling function, invoke the second media stream according to the calling function, so that the second media asset track is synchronously activated with the first media asset track; and when the first media asset track and the second media asset track are synchronously activated, merge the first media stream and the second media stream according to a predetermined time standard setting principle.

10. A method for playing media assets, characterized in that, include: Obtain the media display description file of the media asset file, wherein the media display description file is used to determine the video encoding format contained in the media asset file; According to the media presentation description document, a first description tag and a first channel tag are obtained. The first description tag is used to characterize that the media asset file includes extended layer data. The extended layer data is used to characterize that the video encoding format can be decoded to obtain an ultra-high definition resolution media asset file. Insert the first description label into the first channel label, so that the first description label is transformed into the second channel label; A first media asset track is established based on the first media stream corresponding to the first channel tag, and a second media asset track is established based on the second media stream corresponding to the second channel tag; wherein, the first media stream and the second media stream are respectively used to encode different versions of the media asset file; The first media asset track and the second media asset track are activated synchronously so that the first media stream and the second media stream are decoded synchronously to play the media asset file.

Images