Video metadata transmission method and device

Abstract

The invention discloses a video metadata transmission method and equipment, and relates to the technical field of multimedia. In the embodiment of the invention, the source equipment directly reads the capability description information of the display equipment through the bus so as to obtain the HDR display capability of the display equipment, and the source equipment sends the video stream to the display equipment under the condition that the display equipment supports the display of the cyanine color HDR video. A video metadata message in a video stream comprises an organization to which a standard adopted by the message belongs and a metadata type under the standard, so that a source device can adopt one encapsulation mode to encapsulate different types of metadata, and the standard adopted by the video metadata message and the metadata type are marked in the video metadata message. The problem that the source device needs to consume more computing power to package the video metadata message is avoided, and the transmission efficiency of the video carrying the video metadata message among different devices is improved.

Description

Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a method and device for transmitting video metadata. Background Art

[0002] With the development of big data, artificial intelligence (AI), and cloud computing technology, various types of multimedia data continue to emerge. Taking the wired transmission of video data between different devices as an example, different devices can use the display port (DP) bus to transmit video data. In the DP protocol, when the display device that plays the video supports high-dynamic range (HDR) display, in order to enable the video to have a better display effect on the display device, the source device will also send video metadata to the display device. However, video metadata includes dynamic metadata and static metadata. In the process of transmitting video metadata between the source device and the display device, the source device encapsulates different types of video metadata into metadata packets of different formats, and then sends the encapsulated metadata packets to the display device, and the display device selects different parsing methods according to the format of the metadata packet to optimize the video played in the display device. Since the source device needs to select different encapsulation methods for different types of video metadata, it is necessary to consume the computing power in the source device, which is easy to cause the process of the source device receiving or pushing the video stream to be affected, and the transmission efficiency of the video stream and video metadata is reduced. Summary of the invention

[0003] The present application provides a video metadata data transmission method and device, which solves the problem that the transmission process of the video stream between the source device and the display device is affected and the transmission efficiency of the video stream and metadata is reduced.

[0004] This application adopts the following technical solution.

[0005] In the first aspect, the present application provides a method for transmitting video metadata. The method for transmitting video metadata is executed by a source device, and the source device is connected to a display device via a first bus. The method for transmitting video metadata includes: the source device reads the capability description information of the display device via the first bus, and the capability description information is used to indicate the HDR display capability of the display device. If the capability description information indicates that the display device supports displaying colorful HDR video, the source device generates a video metadata message for the video, and the video metadata message includes: the organization to which the standard adopted by the video metadata message belongs, and the metadata type under the standard. And, the source device sends a video stream carrying the video metadata message to the display device via the first bus.

[0006] In the present application, the source device can directly read the capability description information of the display device through the bus, thereby obtaining the HDR display capability of the display device, and only when the display device supports the display of the Jingcai HDR video, the source device sends the video stream to the display device. In this way, the invalid communication process between devices caused by the display device not supporting the processing of Jingcai HDR video and the problem of the bus bandwidth resources between devices being occupied are avoided. In addition, compared with the conventional technology that requires different encapsulation methods to encapsulate metadata packets, the video metadata message in the video stream generated by the present application includes the organization to which the standard adopted by the video metadata message belongs and the metadata type under the standard, so that the source device can adopt a packaging method to encapsulate different types of metadata, and the video metadata message is marked with the standard and metadata type adopted by the video metadata message, thereby avoiding the source device from consuming more computing power to realize the encapsulation of the video metadata message, which is conducive to improving the transmission efficiency of the video carrying the video metadata message between different devices. Furthermore, the display device can parse the video metadata message based on the aforementioned standards and metadata types, so as to process and display the video in the video stream according to the video metadata carried by the video metadata message, so as to improve the display effect of the video in the display device and enhance the user's experience of watching the video.

[0007] In combination with the video metadata transmission method provided in the first aspect, in an optional implementation, the video metadata transmission method provided in the present application also includes: the source device determines the HDR working mode of the display device according to the capability description information. The HDR working mode is a first mode or a second mode, and the first mode is characterized by: the image rendering parameters of the video are provided by the display device, and the second mode is characterized by: the image rendering parameters of the video are provided by the source device. In the present application, when the display device works in different HDR working modes, different devices provide image rendering parameters, that is: the source device can determine the generation end of the image rendering parameters according to the HDR working mode of the display device, so as to reasonably allocate the processing resources of the source device and the display device, which is conducive to reducing the rendering delay during video transmission and video display, and improving the image rendering efficiency of the video, improving the display effect of the video in the display device, and improving the user's experience of watching the video.

[0008] In combination with the video metadata transmission method provided in the first aspect, in an optional implementation, if the HDR operating mode of the display device is the first mode, the source device generates a video metadata message for the video, including: the source device generates the first video metadata for the video, and determines that the value of the metadata type is the first value; the first value indicates that the first video metadata is static metadata, and the first video metadata is used to indicate: the association between the electrical parameters provided by the display device for the video and the display brightness of the video. And, the source device generates a video metadata message based on the first value and the first video metadata. In some examples, the association between the electrical parameters provided by the display device for the video and the display brightness of the video can also be referred to as: the image transfer characteristics of the video.

[0009] In combination with the video metadata transmission method provided in the first aspect, in an optional implementation, if the HDR working mode of the display device is the second mode, the source device generates a video metadata message for the video, including: the source device generates the second video metadata of the video, and determines that the value of the metadata type is the second value; the second value indicates that the second video metadata is dynamic metadata, and the second video metadata is used to indicate: the display device displays multiple groups of image rendering parameters of multiple frames of images in the video, and one frame of image corresponds to a group of image rendering parameters. And the source device generates a video metadata message according to the second value and the second video metadata.

[0010] Exemplarily, the first value is 0x0 and the second value is 0x1.

[0011] In the above implementation, the source device determines the metadata type of the video metadata according to the HDR working mode of the display device, so that the video metadata message of the metadata type can adapt to the HDR working mode of the display device, such as static metadata adapts to the first mode, and dynamic metadata adapts to the second mode, thereby avoiding the problem that the display device cannot use the video metadata message to improve the display quality of the video due to the incompatibility of the video metadata type with the HDR working mode of the display device, which is conducive to improving the display effect of the video in the display device and enhancing the user's experience of watching the video.

[0012] In the second aspect, the present application provides another method for transmitting video metadata. The method for transmitting video metadata is executed by a display device, the display device is connected to a source device via a first bus, and the display device provides: capability description information indicating the HDR display capability of the display device, and the method for transmitting video metadata includes: the display device receives a video stream sent by the source device via the first bus, the video stream carries a video metadata message, and the video metadata message includes: the organization to which the standard adopted by the video metadata message belongs, and the metadata type under the standard. And, the display device processes and displays the video stream according to the video metadata message carried by the video stream.

[0013] In the present application, the display device is provided with: capability description information indicating the HDR display capability of the display device, the source device can directly read the capability description information of the display device through the bus, so as to know the HDR display capability of the display device, and the source device sends the video stream to the display device only when the display device supports the display of the Brilliant HDR video, so as to avoid the invalid communication process between devices caused by the display device not supporting the processing of the Brilliant HDR video, and the problem of the bus bandwidth resources between devices being occupied. In the process of the display device parsing the video metadata message, since the video metadata message in the video stream of the present application includes the organization to which the standard adopted by the video metadata message belongs and the metadata type under the standard, the display device can adopt a parsing method to parse different types of metadata, and the video metadata message is marked with the standard and metadata type adopted by the video metadata message, thereby avoiding the display device from consuming more computing power to realize the parsing of the video metadata message. Exemplarily, the display device can parse the video metadata message based on the standard and metadata type adopted by the video metadata message, so as to process and display the video in the video stream according to the video metadata carried by the video metadata message, so as to improve the display effect of the video in the display device and improve the user's experience of watching the video.

[0014] In combination with the video metadata data transmission method provided in the second aspect, in an optional implementation, the video metadata is used to indicate: the association between the electrical parameters provided by the display device for the video and the display brightness of the video. The aforementioned display device processes and displays the video stream according to the video metadata message carried by the video stream, including: the display device generates image rendering parameters of the video according to the video metadata and the device information of the display device. And, the display device processes the video stream according to the image rendering parameters to obtain a rendered video, and displays the rendered video.

[0015] In combination with the video metadata transmission method provided in the first aspect and the second aspect, in an optional implementation manner, the video metadata message includes a message header, and the message header carries an organization and metadata type.

[0016] In combination with the video metadata transmission method provided in the first aspect and the second aspect, in an optional implementation, the video metadata message includes a data portion, the data portion is used to carry the video metadata, the size of the data portion is N bytes, and N is any value between 1 and 32. In some possible examples, the data portion is also referred to as the payload of the video metadata message.

[0017] In combination with the video metadata transmission method provided in the first aspect and the second aspect, in an optional implementation, the video stream carries multiple video metadata messages. The first video metadata message in the multiple video metadata messages carries the first S / E standard information, and the first S / E standard information is: S=1, E=0. The last video metadata message in the multiple video metadata messages carries the second S / E standard information, and the second S / E standard information is: S=0, E=1. In the present application, different S / E standard information is used to mark the start position and the end position in the multiple video metadata messages, thereby avoiding the situation where the display device cannot identify the start and end of the video metadata message, resulting in errors in the image rendering of the video. Subsequently, the display device can effectively identify the video metadata message in the video stream, so as to render the video according to the video metadata carried by the video metadata message, thereby improving the display effect of the video in the display device and the user's experience of watching the colorful HDR video.

[0018] In combination with the video metadata transmission method provided by the first aspect and the second aspect, in an optional implementation manner, the above-mentioned multiple video metadata messages also include: an intermediate video metadata message located between the first video metadata message and the last video metadata message, the intermediate video metadata message carries third S / E standard information, and the third S / E standard information is: S=0, E=0.

[0019] In combination with the video metadata transmission method provided in the first aspect and the second aspect, in an optional implementation, each of the multiple video metadata messages carries an index, which is used to indicate the position or order of each video metadata message in the multiple video metadata messages.

[0020] In combination with the video metadata transmission method provided in the first aspect and the second aspect, in an optional implementation manner, the first bus provided in the present application is a unified multimedia interconnection interface (UMI) bus.

[0021] In combination with the video metadata transmission method provided in the first aspect and the second aspect, in an optional implementation, the above-mentioned video metadata message includes: a cyclic redundancy check (CRC) set at the end, and the CRC is used to verify the accuracy of the message header and data part in the video metadata message.

[0022] In a third aspect, the present application provides a video device, which includes a module for executing the method of any one of the implementation modes of the first aspect or the second aspect.

[0023] In a fourth aspect, the present application provides a video processing device. The video processing device includes: a memory, a transceiver, and a processor. The processor is used to generate a video stream to be sent, the memory is used to cache the video stream, and the transceiver is used to send the video stream. The memory, the transceiver, and the processor are used to collaboratively execute the method of any one of the implementation modes in the first aspect.

[0024] In a fifth aspect, the present application provides a display device, comprising: a transceiver and a display unit. The transceiver is used to receive a video stream carrying a video metadata message, wherein the video metadata message includes an organization to which a standard adopted by the video metadata message belongs, and a metadata type under the standard. The display unit is used to process and display the video stream according to the video metadata message carried by the video stream. The transceiver and the display unit are used to collaboratively execute the method of any implementation method in the second aspect.

[0025] In a sixth aspect, the present application provides a video transmission system. The audio transmission system includes a source device provided in the fourth aspect and a display device provided in the fifth aspect, the source device and the display device are connected via a bus, the source device can be used to implement the function of the source device in the first aspect or the second aspect, and the display device can be used to implement the function of the display device in the first aspect or the second aspect. Therefore, the video transmission system can also achieve the beneficial effects of the video metadata transmission method in the first aspect and the second aspect, which will not be described in detail here.

[0026] In a seventh aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes: computer software instructions.

[0027] When the computer software instructions are executed in a computing device, the computing device executes the operation steps of the method in the first aspect or any possible implementation of the first aspect, and executes the operation steps of the method in the second aspect or any possible implementation of the second aspect. If the computing device is the aforementioned source device or display device, etc.

[0028] In an eighth aspect, the present application provides a computer program product. When the computer program product is run on a computer, the computing device executes the operation steps of the method in the first aspect or any possible implementation of the first aspect, and executes the operation steps of the method described in the second aspect or any possible implementation of the second aspect. For example, the computer is the aforementioned source device or display device, etc.

[0029] Regarding the beneficial effects of the third to eighth aspects, reference may be made to the description of any implementation in the first or second aspect, and no further description is given here. Based on the implementations provided in the above aspects, this application may be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of a video transmission system provided for this application;

[0031] Figure 2 A schematic diagram of an audio and video encoding and decoding system provided by the present application;

[0032] Figure 3 A schematic diagram of a video metadata data transmission method provided in this application Figure 1 ;

[0033] Figure 4 A schematic diagram of the structure of a video metadata message provided for this application;

[0034] Figure 5 A schematic diagram of a video metadata data transmission method provided in this application Figure 2 ;

[0035] Figure 6 A schematic diagram of the structure of a video device provided by the present application;

[0036] Figure 7 A schematic diagram of the structure of the video processing device provided for this application;

[0037] Figure 8 This is a schematic diagram of the structure of the display device provided in this application. DETAILED DESCRIPTION

[0038] The present application provides a method for transmitting video metadata, whereby a source device can directly read the capability description information of a display device through a bus, thereby acquiring the HDR display capability of the display device, and sending a video stream to the display device only when the display device supports displaying colorful HDR video. In this way, the invalid communication process between devices caused by the display device not supporting processing colorful HDR video and the problem of bus bandwidth resources being occupied between devices are avoided.

[0039] Specifically, compared to the conventional technology that requires different encapsulation methods to encapsulate metadata packets, the video metadata message in the video stream generated by the present application includes the organization to which the standard adopted by the video metadata message belongs and the metadata type under the standard, so that the source device can use one encapsulation method to encapsulate different types of metadata, and the video metadata message is marked with the standard and metadata type adopted by the video metadata message, thereby avoiding the source device from consuming more computing power to realize the encapsulation of the video metadata message, which is conducive to improving the transmission efficiency of the video carrying the video metadata message between different devices. Furthermore, the display device can parse the video metadata message based on the aforementioned standard and metadata type, so as to process and display the video in the video stream according to the video metadata carried by the video metadata message, so as to improve the display effect of the video in the display device and improve the user's experience of watching the video.

[0040] The technical solutions involved in this application may not only be applied to current audio and video transmission technologies or audio and video standards, but may also be applied to future audio and video transmission technologies or audio and video standards. The terms used in the implementation methods of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0041] Video streaming refers to the transmission of video data, for example, a video stream can be processed as a stable and continuous stream through a network. A video stream includes multiple video frames, each of which corresponds to a frame of image.

[0042] Video metadata: Descriptive data used to describe video frames. For example, the video metadata carries the image rendering parameters of each video frame in the video stream. When the display device parses and plays the video frame based on the video metadata, the video can have a better display effect on the display device.

[0043] Descriptive information packet (DIP or DIP message): used to describe different data in a data packet.

[0044] In this embodiment, video is a general term, and video includes a sequence of multiple continuous frames, where one frame corresponds to one image. Video and audio are information application technology terms, and refer to video, audio, or multimedia content including video and audio.

[0045] In order to make the description of the following embodiments clear and concise, an introduction to the related art is first given.

[0046] Figure 1A schematic diagram of a video transmission system provided by the present application. The video processing process may include but is not limited to: video acquisition, video encoding, video transmission, video decoding and playback process.

[0047] Figure 1 The video and audio transmission system in the embodiment includes a set-top box 110, a smart TV 120, a plurality of video and audio playback devices and a server 130. The set-top box 110 is connected to the operator network through a network cable and can receive video and audio streams from the server 130. The network can realize the function of video and audio transmission, and the network can include one or more network devices, such as a network device 131 which can be a router or a switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate through wireless communication, which is not limited in this application.

[0048] The set-top box 110 is a video and audio processing device, which is used to realize functions such as receiving, processing and pushing video streams or video and audio streams. In some possible situations, the set-top box 110 may also be called an Internet TV set-top box, a network high-definition player or other. Exemplarily, the set-top box 110 may refer to a TV box provided by a network operator or a TV box purchased by a user. For the hardware implementation of the set-top box 110, please refer to the following Figure 7 The description is not repeated here.

[0049] Smart TV 120 is a display device with video and audio processing functions, which realizes functions such as receiving, processing, pushing and playing video streams or video and audio streams. In some possible situations, the smart TV 120 may refer to video and audio devices such as conference tablets, smart TVs or projectors, which are not limited in this application. For the hardware implementation of smart TV 120, please refer to the following Figure 7 or Figure 8 The description is not repeated here.

[0050] The plurality of video and audio playback devices include: video and audio playback devices 121 to video and audio playback devices 124. For example, these video and audio playback devices may include but are not limited to: a multimedia control platform or other devices supporting video and audio playback functions, such as a virtual reality (VR) terminal device or an augmented reality (AR) terminal device, etc. For the hardware implementation of the video and audio playback device, please refer to the following Figure 8 The description is not repeated here.

[0051] In this embodiment, the set-top box 110 and the smart TV 120 may be connected via a bus 125, the set-top box 110 and each video and audio playback device may also be connected via the bus 125, and the smart TV 120 and each video and audio playback device may also be connected via the bus 125. Exemplarily, the bus 125 may be a data bus supporting video and video and audio transmission, for example, the bus 125 is a UMI bus.

[0052] The UMI bus can support uncompressed video and compressed video transmission, and can also support various advanced features such as Quick Video Transport (QVT), Auto Low Latency Mode (ALLM), Dynamic Frame Rate Refresh (DFR), etc. In addition, the UMI bus can also support LPCM format audio and video defined by IEC60958 and various HDR protocols, such as the HDR protocol specified in T / UWA 005.1-2022, such as HDR Vivid, etc. UMI also supports encryption control protection for data transmission such as audio and video. In some optional implementations, the bus 125 may also refer to other types of buses that can implement the functions supported by the above-mentioned UMI bus.

[0053] The server 130 may be an application server or an authentication and authorization server. The server 130 may provide video services, game services, message services, music services, authentication and authorization services, etc. In one example, the functions of multiple services may be integrated on the server 130, for example, game services and music services may be deployed on the server 130. In another example, the functions of some services may be integrated on the server 130, for example, some services of the game service and some services of the video service may be deployed on the server 130. The server 130 may also provide multiple virtual machines using virtualization technology, and various services are provided by the virtual machines. The embodiments of the present application do not limit the deployment form of the server. The network device 131 is connected to the server 130 wirelessly or wiredly. Figure 1 This is just a schematic diagram. The network can also include other devices. Figure 1 Not drawn in.

[0054] It can be understood that the above-mentioned video and audio is a general term. The video and audio include multiple video frames. One video frame corresponds to a group of messages, and the group of messages carries the video and audio data to be parsed and played. Figure 1 This is just a schematic diagram. The video and audio transmission system may also include other equipment. Figure 1 The embodiments of the present application do not limit the number and type of each device included in the system.

[0055] exist Figure 1 The audio and video transmission system shown is based on Figure 2 This is a schematic diagram of an audio and video codec system provided by the present application. The audio and video codec system includes a source device 210 and a sink device 220. The source device 210 establishes a communication connection with the sink device 220 via a UMI bus.

[0056] The source device 210 can implement the video and audio encoding function, such as Figure 1 As shown, the source device 210 may be a set-top box 110 or a smart TV 120 . The source device 210 may also be an audio and video control center with audio and video encoding capabilities. For example, the audio and video control center includes one or more servers.

[0057] The source device 210 may include a data source 211 , a pre-processing module 212 , an audio and video transmission adapter 213 , and a communication interface 214 .

[0058] The data source 211 may include or may be any type of electronic device for collecting audio and video, and / or any type of source audio and video generating device, such as a computer graphics processor for generating computer animation scenes or any type of device for acquiring and / or providing source audio and video, or computer generating source audio and video. The data source 211 may be any type of memory or storage for storing the above-mentioned source audio and video. The above-mentioned source audio and video may include multiple audio and video streams or images obtained by multiple audio and video acquisition devices (such as cameras), such as ultra high definition (UHD) video, high definition (HD) video, 4K video, etc.

[0059] The preprocessing module 212 is used to receive source video and audio, and preprocess the source video and audio to obtain video and audio or multiple frames of images. For example, the preprocessing performed by the preprocessing module 212 may include color format conversion (e.g., conversion from RGB to YCbCr), octree structuring, video and audio splicing, audio track merging and deletion, or channel number adjustment, etc.

[0060] The video and audio transmission adapter 213 is used to receive video and audio or images, and encode the video and audio, images or images to obtain coded data. In some optional situations, the coded stream (coded data) obtained by encoding can also be called a bit stream. If the coded data is obtained by encoding the video and audio data, the bit stream refers to the video and audio stream.

[0061] The communication interface 214 in the source device 210 may be used to receive encoded data (such as a video stream or an audio / video stream), and send the encoded data (or a version of the encoded data after any other processing) to another device such as the sink device 220 or any other device through a UMI bus for storage, display, playback, or image reconstruction.

[0062] Optionally, the source device 210 includes a bitstream buffer, which is used to store bitstreams corresponding to one or more coding units.

[0063] The above-mentioned sink device 220 can realize the function of video and audio decoding, such as Figure 1 As shown, the sink device 220 may be Figure 1 Any of the smart TV 120 or audio and video playback devices shown.

[0064] The sink device 220 may include a video and audio playback unit 221 , a post-processing module 222 , a video and audio receiving adapter 223 , and a communication interface 224 .

[0065] The communication interface 224 in the sink device 220 is used to receive the encoded data (or a version of the encoded data after any other processing is performed on the encoded data) from the source device 210 or any other source device such as a storage device.

[0066] The communication interface 214 and the communication interface 224 may be used to communicate via a direct communication link between the source device 210 and the sink device 220, such as a direct wired connection. Figure 2 For more information about the UMI bus, see Figure 1 The description is not repeated here.

[0067] The communication interface 224 corresponds to the communication interface 214, for example, and can be used to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded data (such as a video stream or an audio / video stream).

[0068] The communication interface 224 and the communication interface 214 can be configured as follows: Figure 2 The unidirectional communication interface indicated by the arrow pointing from the source device 210 to the corresponding UMI bus of the sink device 220, or the bidirectional communication interface, can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link, or data transmission such as encoded compressed data transmission, etc.

[0069] The video and audio receiving adapter 223 is used to receive encoded data and decode the encoded data to obtain decoded data (video or video and audio, etc.).

[0070] The post-processing module 222 is used to post-process the decoded data to obtain post-processed data (such as an image to be displayed or an audio and video to be played). The post-processing performed by the post-processing module 222 may include, for example, color format conversion (such as conversion from YCbCr to RGB), octree reconstruction, audio and video splitting and fusion, etc., or any other processing for generating data for output by the audio and video playback unit 221.

[0071] The video and audio playback unit 221 is used to receive post-processed data for display or playback to a user or viewer, etc. The video and audio playback unit 221 may be or include any type of display for representing the reconstructed image, such as an integrated or external display screen or display. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen. The video and audio playback unit 221 may also include one or more video and audio playback modules, each of which may refer to a speaker, an intelligent speaker, an amplifier, etc.

[0072] As an optional implementation, the source device 210 and the sink device 220 may transmit the encoded data through a data forwarding device. For example, the data forwarding device may be a router or a switch. It is worth noting that the data forwarding device needs to support the UMI interface.

[0073] The implementation of the video metadata transmission method provided in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0074] Here, the video metadata transmission method of the embodiment of the present application is as follows: Figure 2 The source device 210 and the sink device 220 are shown as an example for explanation. Figure 3 A schematic diagram of a video metadata data transmission method provided in this application Figure 1 . Among them, the source device 31 is used to implement the function of the source device 210, and the display device 32 is used to implement the function of the sink device 220. In this embodiment, the source device 31 may also be called an audio-visual sending device, an audio-visual sending terminal, or an audio-visual sending device, and the display device 32 may also be called an audio-visual receiving device, an audio-visual playing device, or an audio-visual receiving device. In this embodiment, the source device 31 and the display device 32 are connected via a bus 33, and the bus 33 may refer to a UMI bus. In some examples, the bus 33 may also be called a connection bus, a first bus, or other names.

[0075] In a first possible application scenario, the source device 31 may be Figure 1 The set-top box 110 and the display device 32 may be Figure 1 The smart TV 120 in the embodiment of the present invention pushes video and audio data to the smart TV, such as a set-top box.

[0076] In a second possible application scenario, the source device 31 may be Figure 1 The set-top box 110 and the display device 32 may be Figure 1 Any video and audio playback device in the video and audio playback device, such as any one of the video and audio playback devices 121 to the video and audio playback device 124. For example, a set-top box pushes video and audio data to the video and audio playback device.

[0077] In a third possible application scenario, the source device 31 may be Figure 1 The smart TV 120 and the display device 32 may be Figure 1 Any video and audio playback device in the video and audio playback device, such as any one of the video and audio playback devices 121 to the video and audio playback device 124. For example, a smart TV pushes video and audio data to the video and audio playback device.

[0078] The above three possible application scenarios are only examples provided in this embodiment and should not be construed as limiting the present application. In other possible examples, the source device 31 may be Figure 1 Any one of the audio and video playback devices (such as the audio and video playback device 121), the display device 32 is another audio and video playback device (such as the audio and video playback device 122) different from the aforementioned audio and video playback device.

[0079] See also Figure 3 The video metadata data transmission method provided in this embodiment includes the following steps S310 to S340.

[0080] S310 , the source device 31 reads capability description information of the display device 32 through the bus 33 .

[0081] The capability description information is used to indicate the wide dynamic range (HDR) display capability of the display device 32. In some possible scenarios, the capability description information may also be referred to as a capability descriptor, which may include one or more bits of information.

[0082] In an optional example, the display device 32 includes a memory in which the capability description information is stored. The memory may include, but is not limited to, random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, mobile hard disk, CD-ROM or any other form of storage medium known in the art.

[0083] In another optional example, the display device 32 includes a dedicated register, and the state of the dedicated register is used to indicate the above-mentioned capability description information. If the value indicated by the register is 1, the capability description information indicates that the display device 32 has the ability to display the colorful HDR video; if the value indicated by the register is 0, the capability description information indicates that the display device 32 does not have the ability to display the colorful HDR video.

[0084] It is worth noting that since the capability description information indicates the HDR display capability of the device, in some cases, the capability description information is also called device comprehensive capability description information, and the area in the display device that stores the DCCD information is called a DCCD area or simply DCCD, which is not limited in this application.

[0085] If the capability description information indicates that the display device 32 supports displaying colorful HDR video, the source device 31 executes S320 ; if the capability description information indicates that the display device 32 does not support displaying colorful HDR video, the source device 31 sends a regular video stream to the display device 32 .

[0086] S320: The source device 31 generates a video metadata message for the video.

[0087] The video metadata message includes: the organization to which the standard adopted by the video metadata message belongs, and the metadata type under the aforementioned standard.

[0088] The organization to which the standard adopted by the video metadata message belongs may include, but is not limited to, the following organizations: UHD World Association (UWA), S organization, C organization, V organization or other organizations.

[0089] The metadata type is static metadata or dynamic metadata. Static metadata uses the same value at all times when the video data is used. For example, the same video metadata is used in a video to control the color and details of each frame, and the video metadata does not change between frames. Dynamic metadata will be changed as needed (it may be modified frame by frame), for example, allowing specific video metadata to be switched for each frame or each scene. The following embodiments provide exemplary descriptions of video metadata of different metadata types, which will not be repeated here.

[0090] Combine the following Figure 4 , the structure of the video metadata message of this embodiment is described, Figure 4 A schematic diagram of the structure of a video metadata message provided by this application. Figure 4In the video metadata message, the video metadata message includes a tunnel header, a message header and a data part.

[0091] like Figure 4 As shown, the tunnel message header includes a reserved field (RSVD, 1 byte), an adapter ID (Shuttle ID, 7 bytes), an E flag (1 byte), an S flag (1 byte), an R flag (1 byte), a CP flag (1 byte), a type (type, 4 bytes), a length (length, 9 bytes), a T flag (1 byte) and an error correction code (ECC), and the ECC occupies 6 bytes.

[0092] The message header is also called the head byte (HB) part of the video metadata message, and the HB part includes one or more header bytes, such as HBx. The message header carries the aforementioned organization and metadata type.

[0093] The structure of the message header of the video metadata message is exemplarily described below in conjunction with Table 1.

[0094] Table 1

[0095]

[0096] Among them, HB1 includes an index (INDEX) field, and the INDEX refers to the position or order of the current video metadata message in the multiple video metadata messages when multiple video metadata messages are generated. For example, the receiving end (such as the display device 32) can determine whether there is a video metadata message lost during the transmission process based on the value of the index, such as the INDEX number of the current video metadata message. The length in HB3 records the effective data length of the data part. For example, when the E flag is 1, the value of the length field in HB3 records the number of effective bytes (effective data length) carried in the video metadata message.

[0097] Please continue to see Figure 4 The data part is also called the data byte (DB) part of the video metadata message, and the DB part includes one or more data bytes, such as DBx. The data part is used to carry video metadata, so in some cases, the data part is also called the payload, payload area or payload region of the video metadata message.

[0098] Exemplarily, the size of the data portion is N bytes, where N is any value in the range of 1-32, such as 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31 or 32 bytes. Figure 4 As shown, the data portion includes 32 bytes, such as DB0 to DB31.

[0099] In one possible scenario, if the information length of the video metadata is less than or equal to 32 bytes, one video metadata message can carry it, and in this case, both the S flag and the E flag in the message header of the video metadata message need to be set to 1. At the same time, the actual effective metadata length is stored in the HB3 byte of the message header.

[0100] In another possible scenario, if the information length of the video metadata is greater than 32 bytes, multiple video metadata messages need to be encapsulated, each video metadata message transmits 32 bytes of video metadata (wherein the HB3 byte is 32), and if the video metadata included in the last message of the multiple video metadata messages is less than 32 bytes, the remaining bytes are filled with 0. At the same time, the actual effective metadata length is stored in the HB3 byte of the message header.

[0101] It is worth noting that when the information length of the video metadata is greater than 32 bytes, since multiple video metadata messages are used for encapsulation, the index (INDEX) in the message header can be used to distinguish these video metadata messages. In order to further determine the first video metadata message, the middle video metadata message and the last video metadata of these multiple video metadata messages, the S / E standard information can be used to distinguish them. The middle video metadata message refers to the video metadata message located between the first video metadata message and the last video metadata message.

[0102] Exemplarily, the first video metadata message among the multiple video metadata messages carries first S / E standard information, and the first S / E standard information is: S=1, E=0.

[0103] As another example, the last video metadata message among the multiple video metadata messages carries the second S / E standard information, and the second S / E standard information is: S=0, E=1.

[0104] Also exemplarily, the middle video metadata message among the multiple video metadata messages carries the third S / E standard information, and the third S / E standard information is: S=0, E=0.

[0105] In this embodiment, different S / E standard information is used to mark the start and end positions of multiple video metadata messages, thereby avoiding the situation where the display device cannot identify the start and end of the video metadata message, resulting in errors in the video image rendering. Subsequently, the display device can effectively identify the video metadata message in the video stream, so as to render the video image according to the video metadata carried by the video metadata message, thereby improving the display effect of the video in the display device and the user's experience of watching the brilliant HDR video.

[0106] In an optional implementation, the video metadata message includes: a cyclic redundancy check code (CRC) set at the end, and the CRC is used to check the accuracy of the message header and data part in the video metadata message.

[0107] Combined with this CRC, Figure 4 As well as the content of Table 1, the following Table 2 shows a possible complete structure of the video metadata message.

[0108] Table 2

[0109]

[0110] The contents repeated in Table 2 and Table 1 are not described here. For the DB part and CRC part, the description of the above embodiment can be referred to, and the header byte (HB or message header) and data byte (DB or payload) are checked by CRC, which is beneficial to improve the robustness of the video metadata message.

[0111] Please continue to see Figure 3 The video metadata data transmission method provided in this embodiment also includes the following steps S330 and S340.

[0112] S330 , the source device 31 sends a video stream carrying a video metadata message to the display device 32 through the bus 33 .

[0113] Corresponding to the process of S330 , the display device 32 receives the video stream sent by the source device 31 through the bus 33 .

[0114] The video stream also includes video data, which includes multiple frames of image data or compressed or encoded image data. For example, in this embodiment, the video data is carried in an active video packet (Active Video Packet, AVP or AVP message) in the video stream, and the AVP message is used to transmit effective video pixel data (pixel data that the display device will actually display).

[0115] In this way, the source device can directly read the capability description information of the display device through the bus, thereby learning the HDR display capability of the display device, and only sends the video stream to the display device when the display device supports displaying Colorful HDR video. In this way, the invalid communication process between devices caused by the display device not supporting the processing of Colorful HDR video and the problem of bus bandwidth resources being occupied between devices are avoided.

[0116] Compared with the conventional technology that needs to use different encapsulation methods to encapsulate metadata data packets, the video metadata message in the video stream generated by the present application includes the organization to which the standard adopted by the video metadata message belongs and the metadata type under the standard, so that the source device can use one encapsulation method to encapsulate different types of metadata, that is: a video metadata message of one format only needs to set up a specific field to mark whether it is dynamic metadata or static metadata, without using two different packages to carry it, thereby avoiding the source device from consuming more computing power to realize the encapsulation of the video metadata message, which is conducive to improving the transmission efficiency of the video carrying the video metadata message between different devices.

[0117] S340: The display device 32 processes and displays the video stream according to the video metadata message carried by the video stream.

[0118] Exemplarily, the display device can parse the video metadata message based on the aforementioned standards and metadata types, so as to process and display the video in the video stream according to the video metadata carried by the video metadata message, so as to improve the display effect of the video in the display device and enhance the user's experience of watching the video.

[0119] Regarding the process of providing the video metadata message in S320 above, a possible example is provided below. Figure 5 As shown, Figure 5 A schematic diagram of a video metadata data transmission method provided in this application Figure 2 . Combined with the above Figure 3 In the illustrated embodiment, after S310, the video metadata data transmission method provided by this embodiment further includes the following step S311.

[0120] S311. The source device 31 determines the HDR working mode of the display device 32 according to the capability description information.

[0121] The HDR working mode is the first mode or the second mode.

[0122] The first mode is characterized by: the display device 32 provides the image rendering parameters of the video, also known as the monitor adaptation mode. The second mode is characterized by: the source device 31 provides the image rendering parameters of the video, also known as the receiving end adaptation mode.

[0123] In an optional implementation, if the HDR operating mode of the display device 32 is the first mode, the above S320 includes the following S321 and S322.

[0124] S321. The source device 31 generates first video metadata of the video, and determines that the value of the metadata type is a first value.

[0125] The first value indicates that the first video metadata is static metadata. For example, the first value is 0x0.

[0126] The first video metadata is used to indicate: the correlation between the electrical parameter provided by the display device 32 for the video and the display brightness of the video. The electrical parameter may refer to voltage or current. Exemplarily, the first video metadata is also called the image transfer characteristic of the video.

[0127] For example, in the data part of the video metadata message, there are two flags (flag bits or identifiers): monitor adaptation mode support flag (monitor mode enable) and image transmission characteristic flag. The monitor adaptation mode support flag is a 1-bit unsigned integer, which is fixed to 1, indicating that the interconnection adaptation processing method of the HDR Vivid signal at this time is the monitor adaptation mode. The image transmission characteristic is a 1-bit unsigned integer, which represents the transmission characteristic of the image, which is fixed to 0, indicating that in the monitor adaptation mode, the image transmission characteristic is the ST2084 EOTF curve.

[0128] S322: The source device 31 generates a video metadata message according to the first value and the first video metadata.

[0129] Corresponding to the video metadata message generated in S322, in the video metadata data transmission method provided by this embodiment, S340 includes the following steps S341 to S343.

[0130] S341 . The display device 32 generates image rendering parameters of the video according to the video metadata and the device information of the display device 32 .

[0131] The device information of the display device 32 may include, but is not limited to: the display panel type, screen size, color gamut standard, color gamut value, screen resolution, screen ratio, brightness, speaker power, number of speakers, screen refresh rate, backlight mode and number of backlight partitions, etc.

[0132] The display panel type refers to LCD, LED, OLED or other types of display screens.

[0133] The screen size refers to the size of the display device 32, such as 85 inches. The color gamut standard refers to the standard adopted by the color display method of the display device 32, such as BT.709. The color gamut value is used to indicate the ability of the display device 32 to display pixels of different colors. The screen resolution refers to the number of pixels in the vertical and horizontal directions, and the unit is px; for screens of the same size, the lower the screen resolution, the fewer pixels are displayed on the screen, and the size of a single pixel is relatively large; the higher the screen resolution, the more pixels are displayed on the screen, and the size of a single pixel is relatively small. The screen ratio refers to the aspect ratio of the display device 32. The unit of the brightness of the display is nit, and the larger the nit value, the higher the brightness of the display device. The refresh rate refers to the number of times the electron beam repeatedly scans the image on the screen, that is, the number of screens switched per unit time, and the unit is Hertz (Hz). The backlight mode is edge-entry backlight or direct-down backlight. Backlight partitioning is to divide the backlight source into multiple areas, each of which can be controlled independently, and the number of backlight partitions refers to the number of areas after the backlight source is divided.

[0134] The image rendering parameters may include, but are not limited to: output image size, brightness, contrast, saturation, rendering order of image blocks, image noise removal information, image filters, and the like.

[0135] S342: The display device 32 processes the video stream according to the image rendering parameters to obtain a rendered video.

[0136] Exemplarily, the display device 32 processes the video data in the video stream according to the image rendering parameters, such as image cropping, image scaling, noise removal, picture optimization, etc., to obtain a rendered video.

[0137] S343: The display device 32 displays the rendered video.

[0138] In this way, the display device generates image rendering parameters according to the device information of the display device and the received video metadata, thereby avoiding the problem of image rendering parameters being lost during transmission, which is beneficial to improving the accuracy of video display by the display device.

[0139] In another optional implementation, if the HDR operating mode of the display device 32 is the second mode, the above S320 includes the following S323 and S324.

[0140] S323: The source device 31 generates second video metadata for the video, and determines that the value of the metadata type is a second value.

[0141] The second value indicates that the second video metadata is dynamic metadata, and the second video metadata is used to indicate that the display device 32 displays multiple sets of image rendering parameters of multiple frames of images in the video, and one frame of image corresponds to one set of image rendering parameters. Exemplarily, the second value is 0x1.

[0142] S324: The source device 31 generates a video metadata message according to the second value and the second video metadata.

[0143] Corresponding to the video metadata message generated by S324, in the video metadata data transmission method provided in this embodiment, S340 includes: the display device 32 processes the video stream according to the image rendering parameters in the video metadata message to obtain a rendered video, and displays the rendered video.

[0144] In this way, the source device determines the metadata type of the video metadata according to the HDR working mode of the display device, so that the video metadata message of this metadata type can adapt to the HDR working mode of the display device, such as static metadata adapts to the first mode, and dynamic metadata adapts to the second mode, thereby avoiding the problem that the display device cannot use the video metadata message to improve the display quality of the video due to the incompatibility of the video metadata type with the HDR working mode of the display device, which is conducive to improving the display effect of the video in the display device and enhancing the user's experience of watching the video.

[0145] It is understandable that, in order to implement the functions in the above embodiments, the source device and the display device include hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0146] Combined with the above Figures 1 to 5 The video metadata transmission method provided by this embodiment is described in detail. Figure 6 , describing the video device provided according to this embodiment.

[0147] Figure 6 600 is a schematic diagram of the structure of a video device provided by the present application. The video device 600 can be used to implement the functions of any one of the devices in the above method embodiments, and thus can also achieve the beneficial effects of the above method embodiments. In this embodiment, the video device 600 can be as follows: Figure 1 The set-top box 110, smart TV 120 or any display device shown may also be Figure 2 The source device 210 or sink device 220 shown may also be a source device or display device provided in subsequent embodiments. It should be understood that the video device 600 may also be a module (such as a chip) applied to any of the aforementioned devices.

[0148] like Figure 6As shown, the video device 600 includes: a transceiver module 610 and a processing module 620. The transceiver module 610 and the processing module 620 can cooperate to implement each step in the above method embodiment. A more detailed description of the above transceiver module 610 and the processing module 620 can be directly obtained by referring to the relevant description of the device in the method embodiment shown in the above figures, and no further description is given here.

[0149] When the video device implements the video metadata data transmission method shown in any of the above-mentioned figures by software, the video device and its various units may also be software modules. The above-mentioned video metadata data transmission method is implemented by calling the software module by a processor. The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD), and the above-mentioned PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0150] Understandably, Figure 6 The video device shown is only an example provided in this embodiment. The video device may include more or fewer units according to different audio transmission processes, and this application is not limited to this.

[0151] When the video device is implemented by hardware, the hardware can be implemented by a processor or a chip system. The chip system includes one or more chips, each chip including an interface circuit and a control circuit. The interface circuit is used to receive data from other devices outside the chip and transmit it to the control circuit, or send data from the control circuit to other devices outside the chip. The control circuit and the interface circuit are used to implement any possible implementation method in the above embodiments through a logic circuit or execution code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments, which will not be repeated here.

[0152] It is understood that the processor in the embodiments of the present application may be a CPU, or other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0153] in addition, Figure 6 The video device 600 shown may also be processed by a video processing device such as Figure 7 As shown, Figure 7 This is a structural diagram of a video processing device provided in the present application. The video processing device 700 includes: a memory 710 and at least one processor 720. The processor 720 can implement the video metadata data transmission method provided in the above embodiment. The memory 710 is used to store software instructions corresponding to the above video metadata data transmission method.

[0154] As an optional implementation, in hardware implementation, the video processing device 700 may refer to a chip or chip system encapsulated with one or more processors 720. For example, when the video processing device 700 is used to implement the method steps in the above embodiment, the processor 720 included in the video processing device 700 executes the steps of the source device 31 in the above method and its possible sub-steps. In an optional scenario, the video processing device 700 may also include a communication interface 730, which may be used to send and receive data. For example, the communication interface 730 is used to receive audio data, or send audio streams, etc.; the communication interface 730 may be implemented by an interface circuit included in the video processing device 700. Therefore, in some examples, the communication interface 730 may also be referred to as a transceiver of the video processing device. In this embodiment, the communication interface 730 supports wired connection using a UMI interface.

[0155] In the embodiment of the present application, the communication interface 730, the processor 720 and the memory 710 can be connected via a bus 740, and the bus 740 can be divided into an address bus, a data bus, a control bus, etc. The bus 740 can be a Peripheral Component Interconnect Express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX) or other types of buses, etc.

[0156] It is worth noting that the video processing device 700 can also perform Figure 6 The functions of the video device 600 shown are not described in detail here.

[0157] The video processing device 700 provided in this embodiment may be the above set-top box 110, smart TV 120, source device 210 or source device 31, etc., or other devices with video processing functions, which are not limited in this application. For example, when the aforementioned display device also has a video processing function, the video processing device 600 may refer to any one of the aforementioned display devices.

[0158] in addition, Figure 6 The video device 600 shown can also be implemented by a display device. When the video device 600 is implemented by a display device, this embodiment provides a possible example, such as Figure 8 As shown, Figure 8 The structural diagram of the display device provided in the present application. The display device includes: a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a UMI interface 231, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio unit 270, a speaker 270A, a receiver 270B, a microphone 270C, a sensor module 280, a button 260, an indicator 262, a camera 263, and a display screen 264, etc.

[0159] Among them, the above-mentioned sensor module 280 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0160] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the display device. In other embodiments, the display device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0161] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0162] The controller can be the nerve center and command center of the display device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0163] The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0164] In some embodiments, the processor 210 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, a UMI interface, etc.

[0165] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation on the display device. In other embodiments, the display device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0166] The wireless communication function of the display device can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor and baseband processor, etc. In some embodiments, antenna 1 of the display device is coupled with mobile communication module 250, and antenna 2 is coupled with wireless communication module 260, so that the display device can communicate with the network and other devices through wireless communication technology.

[0167] The wired communication function of the display device may be implemented through the USB interface 230 or the UMI interface 231. For example, the display device receives or sends a video stream and a video metadata message through a bus connected via the UMI interface 231.

[0168] The display device implements the display function through a GPU, a display screen 264, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 264 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.

[0169] The display screen 264 is used to display images, videos, etc. The display screen 264 includes a display panel.

[0170] The display device can realize the shooting function through ISP, camera 263, video codec, GPU, display screen 264 and application processor. ISP is used to process the data fed back by camera 263. Camera 263 is used to capture static images or videos. In some embodiments, the display device may include 1 or N cameras 263, where N is a positive integer greater than 1.

[0171] In this embodiment, the above display screen 264, video codec, GPU, display screen 264 and application processor may also be collectively referred to as a display unit of the display device 32, which is used to process and display the received video stream.

[0172] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the display device. The external memory card communicates with the processor 210 through the external memory interface 220 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0173] The internal memory 221 may be used to store computer executable program codes, which include instructions. The processor 210 executes various functional applications and data processing of the display device by running the instructions stored in the internal memory 221. For example, in an embodiment of the present application, the processor 210 may execute the instructions stored in the internal memory 221, and the internal memory 221 may include a program storage area and a data storage area.

[0174] The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the display device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0175] The display device can implement audio functions such as music playback and recording through the audio unit 270, the speaker 270A, the receiver 270B, the microphone 270C, and the application processor.

[0176] The button 260 includes a power button, a volume button, etc. The button 260 may be a mechanical button or a touch button. The indicator 262 may be an indicator light, which may be used to indicate the charging status, the change of the power, or may be used to indicate messages, missed calls, notifications, etc.

[0177] The method steps in the embodiments of the present application can also be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and the storage medium can also be present in a video processing device and a display device as discrete components.

[0178] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0179] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A video metadata transmission method, characterized in that: The method is performed by a source device, the source device is connected to a display device via a first bus, and the method includes: Reading capability description information of the display device through the first bus, where the capability description information is used to indicate a wide dynamic range (HDR) display capability of the display device; If the capability description information indicates that the display device supports displaying the HDR video, a video metadata message of the video is generated, wherein the video metadata message includes: the organization to which the standard adopted by the video metadata message belongs, and the metadata type under the standard; A video stream carrying the video metadata message is sent to the display device via the first bus.

2. The method according to claim 1, characterized in that The video metadata message includes a message header, and the message header carries the organization and the metadata type.

3. The method according to claim 1 or 2, characterized in that: The video metadata message includes a data part, and the data part is used to carry the video metadata. The size of the data part is N bytes, and N is any value between 1 and 32.

4. The method according to any one of claims 1 to 3, characterized in that The video stream carries a plurality of video metadata messages; The first video metadata message among the plurality of video metadata messages carries first S / E standard information, wherein the first S / E standard information is: S=1, E=0; The last video metadata message among the multiple video metadata messages carries second S / E standard information, and the second S / E standard information is: S=0, E=1.

5. The method according to claim 4, characterized in that The multiple video metadata messages also include: an intermediate video metadata message located between the first video metadata message and the last video metadata message, the intermediate video metadata message carries third S / E standard information, and the third S / E standard information is: S=0, E=0.

6. The method according to claim 4 or 5, characterized in that: Each of the multiple video metadata messages carries an index.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Determine the HDR working mode of the display device according to the capability description information; the HDR working mode is a first mode or a second mode, the first mode represents: the image rendering parameters of the video are provided by the display device, and the second mode represents: the image rendering parameters of the video are provided by the source device.

8. The method according to claim 7, characterized in that If the HDR working mode of the display device is the first mode, the video metadata message of the generated video includes: Generate first video metadata of the video, and determine that the value of the metadata type is a first value; the first value indicates that the first video metadata is static metadata, and the first video metadata is used to indicate: an association relationship between an electrical parameter provided by the display device for the video and a display brightness of the video; A video metadata message is generated according to the first value and the first video metadata.

9. The method according to claim 7 or 8, characterized in that: If the HDR working mode of the display device is the second mode, the video metadata message of the generated video includes: Generate second video metadata of the video, and determine that the value of the metadata type is a second value; the second value indicates that the second video metadata is dynamic metadata, and the second video metadata is used to indicate that: the display device displays multiple groups of image rendering parameters of multiple frames of images in the video, and one frame of image corresponds to one group of image rendering parameters; A video metadata message is generated according to the second value and the second video metadata.

10. The method according to any one of claims 1 to 9, characterized in that The first bus is a unified multimedia interconnect interface UMI bus.

11. A video metadata transmission method, characterized in that: The method is performed by a display device, the display device is connected to a source device via a first bus, the display device is provided with: capability description information indicating a wide dynamic range (HDR) display capability of the display device, the method comprising: Receiving, via the first bus, a video stream sent by the source device, wherein the video stream carries a video metadata message, wherein the video metadata message includes: an organization to which a standard adopted by the video metadata message belongs, and a metadata type under the standard; The video stream is processed and displayed according to the video metadata message carried by the video stream.

12. The method according to claim 11, characterized in that The video metadata message includes a message header, and the message header carries the organization and the metadata type.

13. The method according to claim 11 or 12, characterized in that: The video metadata message includes a data part, and the data part is used to carry the video metadata. The size of the data part is N bytes, and N is any value between 1 and 32.

14. The method according to claim 13, characterized in that The video metadata message carried by the video stream includes: a plurality of continuous video metadata messages; The first video metadata message of the plurality of consecutive video metadata messages carries first S / E standard information, wherein the first S / E standard information is: S=1, E=0; The last video metadata message among the multiple consecutive video metadata messages carries second S / E standard information, and the second S / E standard information is: S=0, E=1.

15. The method according to claim 14, characterized in that The multiple continuous video metadata messages also include: an intermediate video metadata message located between the first video metadata message and the last video metadata message, the intermediate video metadata message carries third S / E standard information, and the third S / E standard information is: S=0, E=0.

16. The method according to claim 14 or 15, characterized in that Each of the multiple video metadata messages carries an index.

17. The method according to any one of claims 11 to 16, characterized in that The video metadata is used to indicate: the correlation between the electrical parameters provided by the display device for the video and the display brightness of the video; The processing and displaying of the video stream according to the video metadata message carried by the video stream includes: Generate image rendering parameters of the video according to the video metadata and device information of the display device; Processing the video stream according to the image rendering parameters to obtain a rendered video; The rendered video is displayed.

18. The method according to any one of claims 11 to 16, characterized in that: The video metadata is used to indicate that: the display device displays multiple groups of image rendering parameters for multiple frames of images in the video, and one frame of image corresponds to one group of image rendering parameters.

19. The method according to any one of claims 11 to 18, characterized in that The first bus is a unified multimedia interconnect interface UMI bus.

20. A video processing device, characterized in that: include: memories, transceivers, and processors; The processor is used to generate a video stream to be sent; The memory is used to cache the video stream; The transceiver is used to send the video stream; The memory, the transceiver and the processor are configured to collaboratively execute the method of any one of claims 1-10.

21. A display device, characterized in that: include: transceiver and display unit; The transceiver is used to receive a video stream carrying a video metadata message, wherein the video metadata message includes an organization to which a standard adopted by the video metadata message belongs, and a metadata type under the standard; The display unit is used to process and display the video stream according to the video metadata message carried by the video stream; The transceiver and the display unit are used to collaboratively execute the method of any one of claims 11-19.

Images