Video transmission method and device

Abstract

The invention discloses a video transmission method and device. The method is applied to the field of communication. A source device acquires a DCCD of a sink device, the DCCD including a video capability field, and outputs a video according to flag information that whether super-resolution is supported and / or flag information that whether motion estimation and motion compensation capabilities are supported, the flag information being included in the video capability field, the video format of the video including a first resolution and a first frame rate. Since the source equipment acquires the super-resolution capability, dynamic estimation and motion compensation capability of the sink equipment through the DCCD, the source equipment directly outputs the video, that is, the video acquired by the source equipment does not need to be subjected to video system conversion according to a video system negotiated by the source equipment and the sink equipment, the bandwidth occupied by video transmission and the power consumption of the source equipment are reduced, and the video transmission efficiency is improved. And the host equipment performs effective image quality processing on the video, so that the image quality of the video is improved.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular to a video transmission method and device. Background Art

[0002] At present, the video playback function is a basic and important function in smart devices. In the interconnected scenario of source devices (such as personal computers (PCs), mobile phones, set-top boxes, etc.) and sink devices (such as TVs, projectors, monitors, etc.), the source device and the sink device negotiate the supported video formats (such as resolution, frame rate, pixel format, pixel depth, clock, etc.). The source device sends the video according to the negotiated video format. Due to the variety of video formats, the source device uniformly processes videos of different formats according to the negotiated video format, resulting in a waste of bandwidth and power consumption of the source device, as well as various problems such as poor video quality, picture jitter, and video asynchrony of the sink device. Summary of the invention

[0003] The present application provides a video transmission method and device, thereby reducing occupied bandwidth and power consumption and improving video quality.

[0004] In a first aspect, a video transmission method is provided, which is applied to a source device or a chip in the source device, and the method comprises: obtaining a device comprehensive capability description (DCCD) of a host device, the DCCD comprising a video capability field, and outputting a video according to flag information of whether super-resolution (SR) is supported and / or flag information of whether motion estimation and motion compensation (MEMC) capability is supported contained in the video capability field, wherein the video format comprises a first resolution and a first frame rate.

[0005] The video transmission method provided by the present application adds a description of motion estimation and motion compensation capabilities and super-resolution capabilities to the video capability field contained in DCCD. Since the source device learns whether the host device supports super-resolution and / or whether motion estimation and motion compensation are supported through DCCD, the source device does not need to convert the video obtained by the source device according to the video format already negotiated between the source device and the host device, and output the video. This reduces the bandwidth occupied by the transmitted video and the power consumption of the source device, and the host device effectively processes the video quality to improve the video quality.

[0006] In a possible implementation, outputting a video according to flag information indicating whether super-resolution is supported includes: when the flag information indicating whether super-resolution is supported of the sink device indicates that the sink device supports super-resolution capability, outputting the video, wherein the first resolution is the original resolution of the video.

[0007] When the source device learns through DCCD that the sink device supports super-resolution capability, the source device does not need to process the resolution of the video to output the video, that is, there is no need to process the resolution of the video obtained by the source device according to the resolution already negotiated between the source device and the sink device, thereby reducing the bandwidth occupied by the transmitted video. Since there is no need to process the resolution of the video, the power consumption of the source device is reduced. The sink device effectively processes the image quality of the video with the original resolution to improve the image quality of the video.

[0008] In another possible implementation, outputting a video according to flag information indicating whether super-resolution is supported includes: when the flag information indicating whether super-resolution is supported on the host device indicates that the host device supports super-resolution capability and the source device supports super-resolution capability, outputting a video, and the first resolution is the resolution described by DCCD.

[0009] In another possible implementation manner, the first resolution is a resolution described by DCCD, including: the first resolution is an optimal resolution described by DCCD.

[0010] When the sink device supports super-resolution capability and the source device supports super-resolution capability, the source device can comprehensively determine the resolution of the video based on its own capabilities, output the video, and improve the image quality of the video.

[0011] In another possible implementation, outputting a video according to flag information indicating whether super-resolution is supported includes: when the flag information indicating whether super-resolution is supported by the host device indicates that the host device does not support super-resolution capability, outputting the video, and the first resolution is a resolution after processing the original resolution of the video.

[0012] If the sink device does not support super-resolution capabilities, the source device can process the video resolution according to its own capabilities to improve the video quality.

[0013] In another possible implementation, outputting a video according to flag information indicating whether motion estimation and motion compensation capabilities are supported includes: when the flag information indicating whether the motion estimation and motion compensation capabilities of the host device are supported indicates that the host device supports motion estimation and motion compensation capabilities, outputting the video, and the first frame rate is the original frame rate of the video.

[0014] When the source device learns through DCCD that the host device supports motion estimation and motion compensation capabilities, the source device does not need to process the frame rate of the video to output the video, that is, there is no need to process the frame rate of the video obtained by the source device according to the frame rate that has been negotiated between the source device and the host device, thereby reducing the bandwidth occupied by the transmitted video. Since there is no need to process the frame rate of the video, the power consumption of the source device is reduced. The host device effectively processes the image quality of the video with the original frame rate to improve the image quality of the video.

[0015] In another possible implementation, outputting a video according to flag information indicating whether motion estimation and motion compensation capabilities are supported, including: when the flag information indicating whether the motion estimation and motion compensation capabilities of the host device are supported indicates that the host device supports motion estimation and motion compensation capabilities, and the source device supports motion estimation and motion compensation capabilities, outputting a video, and the first frame rate is the frame rate described by DCCD.

[0016] In another possible implementation, the first frame rate is a frame rate described by DCCD, including: the first frame rate is an optimal frame rate described by DCCD.

[0017] When the sink device supports motion estimation and motion compensation capabilities, and the source device supports motion estimation and motion compensation capabilities, the source device can comprehensively determine the frame rate of the video based on its own capabilities, output the video, and improve the image quality of the video.

[0018] In another possible implementation, outputting a video according to flag information indicating whether motion estimation and motion compensation capabilities are supported, including: when the flag information indicating whether the motion estimation and motion compensation capabilities of the host device are supported indicates that the host device does not support motion estimation and motion compensation capabilities, outputting a video, and the first frame rate is a frame rate after processing the original frame rate of the video.

[0019] If the sink device does not support motion estimation and motion compensation capabilities, the source device can process the video frame rate according to its own capabilities to improve the video quality.

[0020] In a second aspect, a video transmission device is provided, comprising modules for executing the operation steps of the method in the first aspect or any possible implementation of the first aspect. For example, the video transmission device comprises a receiving module and a sending module. The receiving module is used to obtain a DCCD of a host device, wherein the DCCD comprises a video capability field. The sending module is used to output a video according to flag information of whether super-resolution is supported and / or flag information of whether motion estimation and motion compensation capabilities are supported contained in the video capability field, wherein the video format of the video comprises a first resolution and a first frame rate.

[0021] In a possible implementation, the sending module is used to output the video according to the flag information of whether super-resolution is supported, and is specifically used to: when the flag information of whether super-resolution of the host device is supported indicates that the host device supports super-resolution capability, output the video, and the first resolution is the original resolution of the video.

[0022] In another possible implementation, the sending module is used to output the video according to the flag information of whether the super-resolution is supported, and is specifically used to: when the flag information of whether the super-resolution of the host device is supported indicates that the host device supports the super-resolution capability and the source device supports the super-resolution capability, output the video, and the first resolution is the resolution described by the DCCD.

[0023] In another possible implementation manner, the first resolution is an optimal resolution described by DCCD.

[0024] In another possible implementation, the sending module is used to output the video according to the flag information of whether super-resolution is supported, and is specifically used to: when the flag information of whether super-resolution is supported by the host device indicates that the host device does not support super-resolution capability, output the video, and the first resolution is the resolution after processing the original resolution of the video.

[0025] In another possible implementation, the sending module is used to output the video according to the flag information of whether the motion estimation and motion compensation capabilities are supported, and is specifically used to: when the flag information of whether the motion estimation and motion compensation capabilities of the host device are supported indicates that the host device supports the motion estimation and motion compensation capabilities, output the video, and the first frame rate is the original frame rate of the video.

[0026] In another possible implementation, the sending module is used to output the video according to the flag information of whether the motion estimation and motion compensation capabilities are supported, and is specifically used to: when the flag information of whether the motion estimation and motion compensation capabilities of the host device are supported indicates that the host device supports the motion estimation and motion compensation capabilities, and the source device supports the motion estimation and motion compensation capabilities, output the video, and the first frame rate is the frame rate described by DCCD.

[0027] In another possible implementation, the first frame rate is an optimal frame rate described by DCCD.

[0028] In another possible implementation, the sending module is used to output the video according to the flag information of whether the motion estimation and motion compensation capabilities are supported, and is specifically used to: when the flag information of whether the motion estimation and motion compensation capabilities of the host device are supported indicates that the host device does not support the motion estimation and motion compensation capabilities, output the video, and the first frame rate is the frame rate after processing the original frame rate of the video.

[0029] In a third aspect, a video transmission device is provided, comprising: a memory, a transceiver and a processor; the memory, the transceiver and the processor are used to collaboratively execute the method described in the first aspect and any one of its possible implementations.

[0030] In a fourth aspect, a video transmission system is provided, the video transmission system comprising a source device and a sink device, the source device can be used to implement the function of the source device in any one of the first aspect and its possible implementation methods, and the sink device effectively processes the resolution and frame rate of the received video to improve the quality of the video. Therefore, the video transmission system can also achieve the beneficial effects of the method in the first aspect, which will not be described in detail here.

[0031] In a fifth aspect, a computer-readable storage medium is provided, which stores computer instructions, and when the computer instructions are executed on a computing device, the first aspect and possible implementations thereof are executed. For example, the computing device is the aforementioned source device.

[0032] In a sixth aspect, a computer program product is provided, the computer program product comprising computer instructions, which, when executed on a computing device, execute the method of the first aspect and any one of its possible implementations, such as the computing device being the aforementioned source device.

[0033] In a seventh aspect, a chip system is provided, comprising: a processor, configured to call and run a computer program from a memory, so that a computing device equipped with the chip system executes the first aspect and possible implementations thereof, such as the computing device being the aforementioned source device or sink device.

[0034] The beneficial effects achieved by the technical solutions of the second to seventh aspects of the present application and the corresponding possible implementation methods can be referred to the technical effects of the first aspect and its corresponding possible implementation methods mentioned above, and will not be repeated here.

[0035] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of an audio and video transmission system provided for this application;

[0037] Figure 2 A schematic diagram of an audio and video encoding and decoding system provided by the present application;

[0038] Figure 3 A schematic diagram of a video processing capability negotiation process provided by this application;

[0039] Figure 4 A schematic diagram of a video transmission method provided by the present application;

[0040] Figure 5 A schematic diagram of the structure of a video transmission device provided in this application;

[0041] Figure 6 A schematic diagram of the structure of a video transmission device provided in this application;

[0042] Figure 7 A schematic diagram of the structure of a display device provided in this application. DETAILED DESCRIPTION

[0043] To facilitate understanding, the main terms involved in this application are first explained.

[0044] Device Comprehensive Capability Description (DCCD): mainly defines a system framework and data structure for declaring the capabilities supported by the device, such as device parameters, performance attributes, and audio and video capabilities. The DCCD adopts a variable-length structure, and the different capabilities supported by the device are encapsulated in different data blocks of the DCCD. The declaration of the capabilities supported by the device can be extended by adding extended data blocks. The DCCD describes the data content structure and is independent of the communication protocol between devices.

[0045] Resolution: Also known as resolution, it refers to the ability of a measurement or display system to distinguish details. It determines the fineness of the details of a bitmap image and is usually used as a measure of the details and clarity displayed in an image, video or display device. The higher the resolution, the clearer the displayed image and the richer the details.

[0046] Frame rate: refers to the frequency (rate) at which bitmap images in units of frames appear continuously on the display.

[0047] Super-resolution (SR): The process of obtaining a high-resolution image by using a series of low-resolution images. This technology improves the quality of low-resolution images and makes them look clearer and more detailed. This technology is used in many fields, such as digital image processing, video processing, imaging, satellite remote sensing, etc.

[0048] Motion Estimation and Motion Compensation (MEMC): is a technology widely used in televisions, monitors and other video playback devices. It is an important indicator for improving image quality. It refers to estimating the trajectory of object movement, inserting new motion compensation frames into the original image, compensating for the images that are not in the video source, reducing the image's ghosting and jitter, and making the motion picture look clearer and smoother.

[0049] At present, the source device converts the video format of the video according to the video format negotiated with the host device. For example, the source device performs super-resolution and / or motion estimation and motion compensation to convert the low resolution of the video into the high resolution required for negotiation with the host device, or processes the frame rate of the video. The source device performs resolution scaling, which fails to fully utilize the super-resolution function of the host device, resulting in poor video quality. The source device achieves frame rate matching by interpolating or dropping frames, which will introduce the problem of "motion jitter".

[0050] The sink device can also restore the resolution of the received video to the original resolution of the video, and then improve the image quality through super-resolution technology and picture quantization (PQ) technology. The sink device can also restore the frame rate of the received video to the original frame rate of the video, and then enable motion estimation and motion compensation technology to make the motion smoother. This increases the cost of the sink device.

[0051] The source device processes videos of different formats in a unified manner according to the negotiated video format, and needs to perform processing such as resolution enlargement, resolution reduction, frame loss, frame insertion, pixel format downsampling, etc. These processes will cause the loss of video details, and the image quality processing cannot be optimized when the sink device restores it, resulting in poor video quality; resolution enlargement will lead to various problems such as increased bandwidth and high device power consumption.

[0052] Among them, the video format described in this application may refer to the format of the video. For example, the video format includes at least one of resolution, frame rate, pixel format, pixel depth and clock.

[0053] In order to solve the problems of wasting bandwidth and power consumption and poor video quality, the video transmission method provided by the present application is that the source device obtains the DCCD of the host device, the DCCD contains a video capability field, and the video capability field contains flag information of whether super-resolution is supported and / or flag information of whether motion estimation and motion compensation capabilities are supported; according to the flag information of whether super-resolution is supported and / or flag information of whether motion estimation and motion compensation capabilities are supported, the video is output, and the video format of the video includes a first resolution and a first frame rate. The first resolution can be the original resolution or the processed resolution of the video, and the first resolution can be the original frame rate or the processed frame rate of the video.

[0054] The video transmission method provided by the present application adds a description of motion estimation and motion compensation capabilities and super-resolution capabilities to the video capability field contained in DCCD. Since the source device learns whether the host device supports super-resolution and / or whether motion estimation and motion compensation are supported through DCCD, the source device does not need to convert the video obtained by the source device according to the video format already negotiated between the source device and the host device, and output the video. This reduces the bandwidth occupied by the transmitted video and the power consumption of the source device, and the host device effectively processes the video quality to improve the video quality.

[0055] The technical solutions involved in the embodiments of the present application can be applied not only to current audio and video transmission technologies or audio and video standards, but also to future audio and video transmission technologies or audio and video standards. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0056] In this embodiment, video is a general term, and video includes a sequence of multiple continuous frames, where one frame corresponds to one image. Video and audio are information application technology terms, and refer to video, audio, or multimedia content including video and audio.

[0057] Video streaming: refers to the transmission of video data. For example, video streaming can be processed as a stable and continuous stream through the network. The video captured by the video capture device (or video data) includes multiple frames of images. The video data is packaged and packaged to obtain a video stream. The video stream includes multiple video frames, and each video frame corresponds to a frame of image.

[0058] The video formats supported by the video transmission method provided in the embodiment of the present application include but are not limited to RGB, YCbCr4:4:4, YCbCr4:2:2, YCbCr4:2:0, ARGB, Y-only, RAW, etc.

[0059] The component bit width (or color depth) of the video data supported by the video transmission method provided in the embodiment of the present application includes but is not limited to one or more of the following: 8bit, 10bit, 12bit or 16bit. For example, for video data in RGB format, the component bit width is 8bit, then the R component of a pixel occupies 8bit, the G component occupies 8bit, and the B component occupies 8bit. It can also be described as: the video transmission method supports 8bpc (bit per component), 10bpc, 12bpc, 16bpc video data.

[0060] The video transmission method provided by the present application is described in detail below with reference to the accompanying drawings.

[0061] Figure 1 The schematic diagram of a video and audio transmission system provided by the present application. The video processing process may include but is not limited to: video acquisition, video encoding, video transmission, video decoding and playback process.

[0062] Figure 1 The video and audio transmission system in the embodiment includes a set-top box 110, a smart TV 120, a plurality of video and audio playback devices and a server 130. The set-top box 110 is connected to the network through a network cable and can receive video and audio streams from the server 130. The network can realize the function of video and audio transmission, and the network can include one or more network devices, such as a network device 131 which can be a router or a switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate through wireless communication, which is not limited in the embodiment of the present application.

[0063] The set-top box 110 is a video and audio processing device, which is used to realize functions such as receiving, processing and pushing video streams or video and audio streams. In some possible situations, the set-top box 110 may also be called an Internet TV set-top box, a network high-definition player or other. Exemplarily, the set-top box 110 may refer to a TV box provided by a network operator or a TV box purchased by a user. For the hardware implementation of the set-top box 110, please refer to the following Figure 6 The description is not repeated here.

[0064] Smart TV 120 is a display device with video and audio processing functions, which realizes functions such as receiving, processing, pushing and playing video streams or video and audio streams. In some possible situations, the smart TV 120 may refer to video and audio devices such as conference tablets, smart TVs or projectors, which are not limited in this embodiment of the present application. For the hardware implementation of smart TV 120, please refer to the following Figure 6 or Figure 7 The description is not repeated here.

[0065] The plurality of video and audio playback devices include: video and audio playback devices 121 to video and audio playback devices 124. For example, these video and audio playback devices may include but are not limited to: a multimedia control platform or other devices supporting video and audio playback functions, such as a virtual reality (VR) terminal device or an augmented reality (AR) terminal device, etc. For the hardware implementation of the video and audio playback device, please refer to the following Figure 7 The description is not repeated here.

[0066] In this embodiment, the set-top box 110 and the smart TV 120 are connected via a network, for example, via an audio and video interface network connection, the set-top box 110 and each audio and video playback device may also be connected via an audio and video interface network connection, and the smart TV 120 and each audio and video playback device may also be connected via an audio and video interface network connection. Exemplarily, the audio and video interface network may be a wired network supporting video and audio and video transmission. Optionally, the audio and video interface network may be referred to as a unified multimedia internet network. In some implementations, the unified multimedia internet network may also be other names.

[0067] The unified multimedia internet network can support both uncompressed and compressed video transmission, as well as various advanced features such as quick video transport (QVT), auto low latency mode (ALLM), and dynamic frame rate refresh (DFR). The unified multimedia internet network can also support LPCM format video and audio defined by IEC 60958 and various HDR protocols, such as the HDR protocol specified in T / UWA 005.1-2022, such as HDR Vivid. The unified multimedia internet network also supports encryption control protection for video and audio data transmission.

[0068] The server 130 may be an application server or an authentication and authorization server. The server 130 may provide video services, game services, message services, music services, authentication and authorization services, etc. In one example, the functions of multiple services may be integrated on the server 130, for example, game services and music services may be deployed on the server 130. In another example, the functions of some services may be integrated on the server 130, for example, some services of the game service and some services of the video service may be deployed on the server 130. The server 130 may also provide multiple virtual machines using virtualization technology, and various services are provided by the virtual machines. The embodiments of the present application do not limit the deployment form of the server. The network device 131 is connected to the server 130 wirelessly or wiredly. Figure 1The schematic diagram is only an example, and the network may also include other devices. Figure 1 Not drawn in.

[0069] Figure 1 This is just a schematic diagram. The video and audio transmission system may also include other equipment. Figure 1 The embodiment of the present application does not limit the number and type of each device included in the system.

[0070] exist Figure 1 The audio and video transmission system shown is based on Figure 2 This is a schematic diagram of an audio and video codec system provided by the present application. The audio and video codec system includes a source device 210 and a sink device 220. The source device 210 establishes a communication connection with the sink device 220 through a unified multimedia Internet network.

[0071] The source device 210 can implement the video and audio encoding function, such as Figure 1 and Figure 2 As shown, the source device 210 may be the above Figure 1 The source device 210 may be a set-top box 110 or a smart TV 120 shown in FIG. 1 , and may also be a video and audio control center having video and audio encoding capabilities. For example, the video and audio control center includes one or more servers.

[0072] The source device 210 may include a data source 211 , a pre-processing module 212 , an audio and video transmission adapter 213 , and a communication interface 214 .

[0073] The data source 211 may include or may be any type of electronic device for collecting audio and video, and / or any type of source audio and video generating device, such as a computer graphics processor for generating computer animation scenes or any type of device for acquiring and / or providing source audio and video, or computer-generated source audio and video. The data source 211 may be any type of memory or storage for storing the above-mentioned source audio and video. The above-mentioned source audio and video may include multiple audio and video streams or images obtained by multiple audio and video acquisition devices (such as cameras), such as ultra high definition (UHD) video, high definition (HD) video, 4K video, 8K video, etc.

[0074] The preprocessing module 212 is used to receive source video and audio, and preprocess the source video and audio to obtain video and audio or multiple frames of images. For example, the preprocessing performed by the preprocessing module 212 may include color format conversion (e.g., conversion from RGB to YCbCr), octree structuring, video and audio splicing, audio track merging and deletion, or channel number adjustment, etc.

[0075] The video and audio transmission adapter 213 is used to receive video and audio or images, and encode the video and audio or images to obtain coded data. In some optional situations, the coded stream (coded data) obtained by encoding can also be called a bit stream. If the coded data is obtained by encoding the video and audio data, the bit stream refers to the video and audio stream, and if the coded data is obtained by encoding the video data, the bit stream refers to the video stream.

[0076] The communication interface 214 in the source device 210 can be used to receive encoded data (such as a video stream or an audio and video stream), and send the encoded data (or a version of the encoded data after any other processing) to another device such as the host device 220 or any other device through the unified multimedia Internet network for storage, display, playback or image reconstruction.

[0077] Optionally, the source device 210 includes a bitstream buffer, which is used to store bitstreams corresponding to one or more coding units.

[0078] The above-mentioned sink device 220 can realize the function of video and audio decoding, such as Figure 1 As shown, when the source device 210 is the set-top box 110, the sink device 220 may be Figure 1 The source device 210 is the smart TV 120 or any one of the video and audio playback devices shown. When the source device 210 is the smart TV 120, the sink device 220 can be any one of the video and audio playback devices.

[0079] The sink device 220 may include a video and audio playback unit 221 , a post-processing module 222 , a video and audio receiving adapter 223 , and a communication interface 224 .

[0080] The communication interface 224 in the sink device 220 is used to receive the encoded data (or a version of the encoded data after any other processing is performed on the encoded data) from the source device 210 or any other source device such as a storage device.

[0081] The communication interface 214 and the communication interface 224 may be used to communicate via a direct communication link between the source device 210 and the sink device 220, such as a direct wired connection. Figure 2 For more information about the unified multimedia internet network, please refer to Figure 1 The description is not repeated here.

[0082] The communication interface 224 corresponds to the communication interface 214, for example, and can be used to transmit data and process the data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain audio and video data.

[0083] The communication interface 224 and the communication interface 214 can be configured as follows: Figure 2The unidirectional communication interface or the bidirectional communication interface indicated by the arrow pointing from the source device 210 to the sink device 220 in the corresponding unified multimedia internet network can be used to send and receive messages, etc. to establish a connection, and transmit other information, etc., such as transmitting information related to the communication link, or, for example, transmitting information related to audio and video data (such as descriptive information DIP of audio and video).

[0084] The video and audio receiving adapter 223 is used to receive encoded data and decode the encoded data to obtain decoded data (video or video and audio, etc.).

[0085] The post-processing module 222 is used to post-process the decoded data to obtain post-processed data (such as an image to be displayed or an audio and video to be played). The post-processing performed by the post-processing module 222 may include, for example, color format conversion (such as conversion from YCbCr to RGB), octree reconstruction, audio and video splitting and fusion, etc., or any other processing for generating data for output by the audio and video playback unit 221.

[0086] The video and audio playback unit 221 is used to receive post-processed data for display or playback to a user or viewer, etc. The video and audio playback unit 221 may be or include any type of display for representing the reconstructed image, such as an integrated or external display screen or display. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen. The video and audio playback unit 221 may also include one or more video and audio playback modules, each of which may refer to a speaker, an intelligent speaker, an amplifier, etc.

[0087] As an optional implementation, the source device 210 and the sink device 220 may transmit the encoded data through a data forwarding device. For example, the data forwarding device may be a router or a switch. It is worth noting that the data forwarding device needs to support a unified multimedia internet network.

[0088] In the embodiment of the present application, the source device may also be referred to as an audio and video sending device, an audio and video sending terminal, etc., and the sink device may also be referred to as an audio and video receiving device, an audio and video playing device, etc. In this embodiment, the source device and the sink device are connected via a unified multimedia Internet network.

[0089] In the first possible application scenario, the source device can be Figure 1 The set-top box 110 in the embodiment of the present invention may be a sink device. Figure 1 In the smart TV 120, if the set-top box negotiates with the smart TV, that is, the set-top box obtains the DCCD of the smart TV, the video capability field included in the DCCD adds the description of the motion estimation and motion compensation capability and the super-resolution capability, and the set-top box pushes the video to the smart TV according to the super-resolution capability and / or motion estimation and motion compensation capability of the smart TV.

[0090] In the second possible application scenario, the source device can be Figure 1 The set-top box 110 in the embodiment of the present invention may be a sink device. Figure 1 Any video and audio playback device in, such as any one of the video and audio playback devices 121 to 124. If the set-top box negotiates with the video and audio playback device, that is, the set-top box obtains the DCCD of the video and audio playback device, the set-top box pushes the video and audio data to the video and audio playback device.

[0091] In the third possible application scenario, the source device can be Figure 1 In the smart TV 120, the sink device may be Figure 1 Any video and audio playback device in, such as any one of the video and audio playback devices 121 to 124. For example, the smart TV negotiates with the video and audio playback device, that is, the smart TV obtains the DCCD of the video and audio playback device, and the smart TV pushes the video and audio data to the video and audio playback device.

[0092] The above three possible application scenarios are only examples provided in this embodiment and should not be construed as limiting the present application. Figure 1 Any one of the video and audio playback devices (such as the video and audio playback device 121), the host device is another video and audio playback device (such as the video and audio playback device 122) different from the aforementioned video and audio playback device.

[0093] The following describes the process of DCCD negotiation between the source device and the sink device.

[0094] Figure 3 A schematic diagram of a video processing capability negotiation process provided by the present application. Here, the video transmission method of the present application is as follows: Figure 2 The source device 210 or a chip in the source device 210 and the sink device 220 or a chip in the sink device 220 are used as examples for explanation. Figure 3 The video transmission method provided in the embodiment of the present application includes steps 310 to 320.

[0095] Step 310: The source device obtains the DCCD of the sink device.

[0096] Step 320: The sink device obtains the DCCD of the source device.

[0097] The following description is made by taking the source device obtaining the DCCD of the sink device as an example. The sink device obtaining the DCCD of the source device can refer to the description of the source device obtaining the DCCD of the sink device.

[0098] The sink device provides information indicating the capability of the sink device. The source device queries the capability of the sink device, including the source device obtaining the capability description information of the sink device. The process of the source device obtaining the capability information of the sink device includes: the source device sends a query message to the sink device, and the query message is used to query the capability of the sink device; then, the sink device sends (feeds back) the capability description information of the sink device to the source device, and the source device receives the capability description information sent by the sink device, and the capability description information can indicate the capability of the sink device.

[0099] The capability description information of the sink device may include information indicating the video processing capability of the sink device. For example, the video processing capability includes an indication of whether the sink device supports super-resolution capability and / or whether it supports motion estimation and motion compensation capability. In some possible situations, the capability description information may also be referred to as a capability descriptor, which may include one or more bits of information.

[0100] In some cases, the capability description information of the sink device may also be referred to as a device comprehensive capability description (device comprehensive capability description), and the area in the sink device storing the DCCD is referred to as a DCCD area or simply DCCD, which is not limited in this application.

[0101] In an embodiment of the present application, a capability declaration is made in the DCCD as to whether super-resolution capability and / or whether motion estimation and motion compensation capability are supported. For example, the host device supports super-resolution capability, supports motion estimation and motion compensation capability, does not support super-resolution capability, or does not support any one of motion estimation and motion compensation capability.

[0102] The source device queries the super-resolution capability and motion estimation and motion compensation capability of the sink device through the DCCD, and then the sink device feeds back its own super-resolution capability and motion estimation and motion compensation capability to the source device.

[0103] In some embodiments, the DCCD includes a video capability field, which mainly describes the video-related capabilities supported by the display. The video capability field includes flag information indicating whether super-resolution is supported and / or flag information indicating whether motion estimation and motion compensation capabilities are supported.

[0104] For example, the data structure of the video capability field is shown in Table 1.

[0105] Table 1

[0106]

[0107] As shown in Table 1, the offset 06h of the video capability field includes flag information of whether MEMC is supported and flag information of whether SR is supported. In some embodiments, different values ​​of the bit can indicate whether super-resolution is supported and / or whether motion estimation and motion compensation are supported.

[0108] For example, a bit value of 1 indicates that super-resolution or motion estimation and motion compensation are supported, and a bit value of 0 indicates that super-resolution or motion estimation and motion compensation are not supported. For another example, a bit value of 0 indicates that super-resolution or motion estimation and motion compensation are supported, and a bit value of 1 indicates that super-resolution or motion estimation and motion compensation are not supported.

[0109] Optionally, the DCCD may also store other content describing the audio and video processing capabilities of the host device, such as audio processing capabilities (whether the host device supports processing pure audio data packets), HDR display capabilities (whether the host device supports displaying specific types of HDR videos, such as Colorful HDR), or others.

[0110] In an optional example, the host device includes a memory in which the capability description information is stored. The memory may include, but is not limited to, random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, mobile hard disk, CD-ROM or any other form of storage medium known in the art.

[0111] In another optional example, the sink device includes a dedicated register, and the state of the dedicated register is used to indicate the above-mentioned capability description information. If the value indicated by the register is 1, the capability description information indicates that the sink device has super-resolution capability; if the value indicated by the register is 0, the capability description information indicates that the sink device does not have super-resolution capability. If the value indicated by the register is 1, the capability description information indicates that the sink device has motion estimation and motion compensation capability; if the value indicated by the register is 0, the capability description information indicates that the sink device does not have motion estimation and motion compensation capability.

[0112] The following describes the process of video transmission between the source device and the sink device.

[0113] The source device obtains the DCCD fed back by the sink device, parses the DCCD, and obtains the flag information of whether the sink device supports and / or the flag information of whether the motion estimation and motion compensation capabilities are supported. In the case where the source device outputs the video, it determines whether to process the resolution and frame rate of the video, output the video, and transmit the video to the sink device according to the flag information of whether the super-resolution is supported and / or the flag information of whether the motion estimation and motion compensation capabilities are supported.

[0114] Figure 4 A flow chart of a video transmission method provided by the present application. Here, the video transmission method of the present application is as follows: Figure 2 The source device 210 or a chip in the source device 210 and the sink device 220 or a chip in the sink device 220 are used as examples for explanation. Figure 4 The video transmission method provided in the embodiment of the present application includes steps 410 to 420.

[0115] Step 410: The source device outputs a video according to the flag information indicating whether the super-resolution is supported and / or the flag information indicating whether the motion estimation and motion compensation capabilities are supported. The video format of the video includes a first resolution and a first frame rate. Accordingly, the sink device receives the video from the source device.

[0116] The video sent by the source device to the sink device may be a frame of data in the video data to be sent by the source device, and the video corresponds to valid data (or valid pixels, or valid pixel data) of a frame image in the video.

[0117] This application does not limit the source of the video obtained by the source device. The source device can obtain the video from a storage device, or receive the video sent by other devices.

[0118] In some embodiments, after the source device acquires the video, the output strategy of the video can be determined according to the flag information of whether the super-resolution of the host device is supported and / or the flag information of whether the motion estimation and motion compensation capabilities are supported, and the output strategy indicates the video format of the video. The source device outputs the video and transmits the video to the host device. The video format of the video can be the video format indicated by the output strategy. For example, the output strategy indicates whether to process the resolution and / or frame rate of the video. If the output strategy indicates that the resolution and / or frame rate of the video are processed, the output strategy can also indicate the processed resolution and / or processed frame rate.

[0119] The following describes how the source device outputs a video based on flag information indicating whether the sink device supports super-resolution.

[0120] In a first possible implementation manner, when the flag information of whether the sink device supports super-resolution indicates that the sink device supports super-resolution capability, a video is output, and a first resolution included in the video format of the video is an original resolution of the video.

[0121] The source device learns that the sink device has super-resolution capability, and after acquiring the video, the source device does not need to process the resolution of the video, and directly outputs the video with the original resolution. For example, the original resolution of the video acquired by the source device is 4K, and the first resolution of the video output by the source device is 4K.

[0122] In some embodiments, the DCCD acquired by the source device further includes a CTA Timing field, and the CTA Timing field is used to describe that the DCCD contains video timing information supporting the CTA protocol. The data structure of the CTA Timing field is shown in Table 2.

[0123] Table 2

[0124]

[0125] The CTA Timing Code refers to the VIC number in Table 1 Video Format Timings in the CTA-861-G protocol. If an unknown VIC is identified during parsing on the other end, the VIC Code can be ignored and subsequent parsing can continue.

[0126] For example, the VIC codes in the CTA Timing field are arranged in a preferred order, that is, the first CTA Timing is the most recommended and preferred Timing in the CTA Timing field, the second one is the second most recommended, and so on.

[0127] The original resolution may also be a resolution that can be recognized by the sink device. For example, the original resolution may be a resolution indicated by any video format shown in Table 2.

[0128] In a second possible implementation, when the flag information of whether the sink device supports super-resolution indicates that the sink device supports super-resolution capability and the source device supports super-resolution capability, a video is output, and a first resolution included in the video format of the video is a resolution described by DCCD.

[0129] The source device supports super-resolution capability, which means that the source device has the ability to process resolution. When the source device learns that the sink device has super-resolution capability, after the source device obtains the video, it makes a comprehensive decision based on its own capabilities and outputs the video according to the resolution described by DCCD. That is, the resolution of the video is processed according to the resolution described by DCCD, and the resolution of the output video is the processed resolution. For example, the original resolution of the video obtained by the source device is 4K, and the first resolution of the video output by the source device is 8K.

[0130] In some embodiments, the first resolution included in the video format of the video is the optimal resolution described by DCCD. For example, Table 2 indicates that the first CTA Timing is the most recommended and preferred Timing in the CTA Timing field, and the original resolution of the video is converted to the resolution indicated by the most recommended and preferred Timing.

[0131] Optionally, the first resolution included in the video format of the video is a suboptimal resolution described by DCCD.

[0132] In a third possible implementation, when the flag information of whether the host device supports super-resolution indicates that the host device does not support super-resolution capability, a video is output, and the first resolution included in the video format of the video is a resolution after processing the original resolution of the video.

[0133] The source device learns that the sink device does not have super-resolution capability. After the source device obtains the video, it processes the resolution of the video, and the resolution of the output video is the processed resolution. For example, the source device can process the original resolution of the video according to the resolution negotiated with the sink device, that is, the resolution of the video output by the source device is the resolution negotiated with the sink device. The resolution negotiated with the sink device can be any of the resolutions shown in Table 2. For another example, the source device can process the original resolution of the video according to the resolution described by the DCCD, and the resolution of the video output by the source device is the suboptimal resolution described by the DCCD. For example, the original resolution of the video obtained by the source device is 2K, and the first resolution of the video output by the source device is 4K.

[0134] In the above solution, the original resolution of the video may be any resolution shown in Table 2. Optionally, the resolution of the video acquired by the source device may not be any resolution shown in Table 2, and the source device may process the original resolution of the video according to the resolution negotiated with the sink device, that is, convert the resolution of the video output by the source device into the resolution negotiated with the sink device. Alternatively, the source device converts the original resolution of the video into any resolution shown in Table 2.

[0135] The following describes how the source device outputs a video based on flag information indicating whether the sink device supports motion estimation and motion compensation capabilities.

[0136] In a first possible implementation, when the flag information indicating whether the host device supports motion estimation and motion compensation capabilities indicates that the host device supports motion estimation and motion compensation capabilities, a video is output, and a first frame rate included in the video format of the video is an original frame rate of the video.

[0137] The source device learns that the sink device has motion estimation and motion compensation capabilities. After the source device obtains the video, it does not need to process the frame rate of the video and directly outputs the video with the original frame rate. For example, the original frame rate of the video obtained by the source device is 60 frames per second, and the first frame rate of the video output by the source device is 4K60 frames per second.

[0138] The original frame rate may also be a frame rate that can be recognized by the sink device. For example, the original frame rate may be a frame rate indicated by any video format shown in Table 2.

[0139] In a second possible implementation, when the flag information indicating whether the motion estimation and motion compensation capabilities of the host device are supported indicates that the host device supports the motion estimation and motion compensation capabilities, and the source device supports the motion estimation and motion compensation capabilities, the video is output, and the first frame rate included in the video format of the video is the frame rate described by the DCCD.

[0140] The source device supports motion estimation and motion compensation capabilities, which means that the source device has the ability to process frame rates. When the source device learns that the host device has motion estimation and motion compensation capabilities, after the source device obtains the video, it makes a comprehensive decision based on its own capabilities and outputs the video at the frame rate described by DCCD. That is, the frame rate of the video is processed according to the frame rate described by DCCD, and the frame rate of the output video is the processed frame rate. For example, the original frame rate of the video obtained by the source device is 30 frames / second, and the first frame rate of the video output by the source device is 60 frames / second.

[0141] In some embodiments, the first frame rate included in the video format of the video is the optimal frame rate described by the DCCD. For example, Table 2 indicates that the first CTA Timing is the most recommended and preferred Timing in the CTA Timing field, and the original frame rate of the video is converted to the frame rate indicated by the most recommended and preferred Timing.

[0142] Optionally, the first frame rate included in the video format of the video is a suboptimal frame rate included in the DCCD.

[0143] In a third possible implementation, when the flag information indicating whether the host device supports motion estimation and motion compensation capabilities indicates that the host device does not support motion estimation and motion compensation capabilities, a video is output, and a first frame rate included in the video format of the video is a frame rate after processing the original frame rate of the video.

[0144] The source device learns that the host device does not have motion estimation and motion compensation capabilities. After the source device obtains the video, it processes the frame rate of the video, and the frame rate of the output video is the processed frame rate. For example, the source device can process the original frame rate of the video according to the frame rate negotiated with the host device, that is, the frame rate of the video output by the source device is the frame rate negotiated with the host device. The frame rate negotiated with the host device can be any of the frame rates shown in Table 2. For another example, the source device can process the original frame rate of the video according to the frame rate described by the DCCD, and the frame rate of the video output by the source device is the suboptimal frame rate described by the DCCD. For example, the original frame rate of the video obtained by the source device is 30 frames / second, and the first frame rate of the video output by the source device is 60 frames / second.

[0145] In the above solution, the original frame rate of the video may be any frame rate shown in Table 2. Optionally, the frame rate of the video acquired by the source device may not be any frame rate shown in Table 2, and the source device may process the original frame rate of the video according to the frame rate negotiated with the sink device, that is, convert the frame rate of the video output by the source device into the frame rate negotiated with the sink device. Alternatively, the source device converts the original frame rate of the video into any frame rate shown in Table 2.

[0146] The possible implementations of the above-mentioned super-resolution capability and motion estimation and motion compensation capabilities may be combined arbitrarily, and are not limited in this application.

[0147] For example, when the sink device supports super-resolution capability and supports motion estimation and motion compensation capability, a video is output, and a video format of the video includes a first resolution that is an original resolution of the video and a first frame rate that is an original frame rate of the video.

[0148] For another example, when the host device does not support super-resolution capability and supports motion estimation and motion compensation capabilities, the video is output, the first resolution is the resolution converted from the original resolution of the video, and the first frame rate is the original frame rate of the video.

[0149] For another example, when the host device supports super-resolution capability but does not support motion estimation and motion compensation capabilities, the video is output, the first resolution is the original resolution of the video, and the first frame rate is the frame rate converted from the original frame rate of the video.

[0150] For another example, when the host device does not support super-resolution capability and does not support motion estimation and motion compensation capabilities, the output video has a first resolution that is a resolution converted from the original resolution of the video, and a first frame rate that is a frame rate converted from the original frame rate of the video.

[0151] The source device transmits the video to the sink device at the original resolution and frame rate of the video, which can effectively reduce the transmission bandwidth and save power consumption. The source device transmits the video to the sink device at the original resolution and frame rate of the video. The sink device can process the image quality more effectively according to the original resolution and frame rate to achieve the best effect.

[0152] Step 420: The sink device processes the video from the source device.

[0153] When the sink device supports super-resolution capability, after receiving the video from the source device, the sink device performs super-resolution processing on the first resolution of the video. For example, the first resolution of the video is converted to a second resolution. The first resolution may be 4K, and the second resolution may be 8K.

[0154] The first resolution included in the video format of the video may be the original resolution of the video. For another example, the first resolution included in the video format of the video may also be a resolution processed from the original resolution.

[0155] In some embodiments, the source device compares the resolution of the received video with the target resolution, and if the resolution of the received video is lower than the target resolution, performs super-resolution processing on the resolution of the video. The target resolution may be the resolution of the display.

[0156] For example, the super-resolution processing mainly includes the following steps.

[0157] 1. Feature extraction: First, the low-resolution image is subjected to feature extraction through a convolutional neural network to obtain a feature map. The purpose of this step is to extract useful feature information from the low-resolution image to prepare for the subsequent generation of high-resolution images.

[0158] 2. Generate high-resolution images: Next, the feature map is input into the generator, which generates high-resolution images layer by layer through a series of convolutional and deconvolutional layers. This step is the core of super-resolution processing. Through the learned feature information, the generator tries to reconstruct a high-resolution image.

[0159] 3. Calculate perspective loss: Compare the generated high-resolution image with the real high-resolution image and calculate the perspective loss. Perspective loss is a loss function used to measure the difference between the image generated by the generator and the real image, taking into account multiple factors such as image brightness, contrast, and structure.

[0160] 4. Optimize parameters: Finally, by optimizing the parameters of the generator and the discriminator, the generated high-resolution image is closer and closer to the real high-resolution image. This step continuously improves the quality of the generated image by adjusting the parameters of the generator and the discriminator, and finally achieves an effect close to the real high-resolution image.

[0161] The entire super-resolution process is an iterative optimization process. By continuously adjusting and optimizing parameters, the quality of the generated image is continuously improved, and the super-resolution goal is finally achieved. This process involves not only image processing technology, but also deep learning and machine learning methods, making it possible to generate high-resolution images from low-resolution images.

[0162] For example, it is processed by the Scaler module of the host device (such as a TV). For example, high-end TVs can perform super-resolution processing through AI SR to achieve better image quality.

[0163] In the case where the sink device supports motion estimation and motion compensation capabilities, after receiving the video from the source device, the sink device performs motion estimation and motion compensation processing on the first frame rate of the video. For example, the sink device turns on the motion estimation and motion compensation capabilities and performs frame insertion processing on the video. For example, the first frame rate of the video is converted to a second frame rate. The first frame rate may be 30 frames per second, and the second frame rate may be 60 frames per second.

[0164] The first frame rate included in the video format of the video is the original frame rate of the video. For another example, the first frame rate included in the video format of the video may also be a frame rate after the original frame rate is processed.

[0165] In some embodiments, the source device compares the frame rate of the received video with the target frame rate, and if the frame rate of the received video is lower than the target frame rate, performs motion estimation and motion compensation on the frame rate of the video. The target frame rate may be the frame rate of the display.

[0166] The main purpose of motion estimation and motion compensation technology is to improve the smoothness and clarity of video playback, especially when dealing with fast-moving scenes. Motion estimation and motion compensation technology predicts the motion trajectory of an object by analyzing the changes between two consecutive frames, and generates one or more intermediate frames on this basis to fill the gaps between the original video frames. That is, the host device turns on the motion estimation and motion compensation capabilities to insert frames. As a result, motion blur is reduced, dynamic clarity is improved, and high-speed motion images look smoother and more natural.

[0167] The process of motion estimation mainly involves finding the best match between the block in the current frame and the block in the reference frame. This process is achieved by calculating the similarity between the current block and the blocks at various positions in the reference frame. The similarity measurement criteria include MAD, MSE, NCCF, etc., while the H.264 standard uses the sum of absolute errors and the sum of absolute transformed differences (SATD). After finding the best match, motion estimation will output a motion vector (MV), which is the position coordinate of the reference block relative to the current block. This motion vector is then used in the motion compensation process.

[0168] Motion compensation involves using the motion vectors obtained from motion estimation to obtain the corresponding blocks from the reference frame and compare them with the corresponding blocks in the current frame to obtain the residual blocks. If the two blocks match very well, the residual block is basically zero, which means that the data compression efficiency is very high. Motion compensation includes global motion compensation and block motion compensation. It is a method of describing the difference between adjacent frames to reduce spatial redundancy in video sequences. This method is used in video compression / video codecs to reduce spatial redundancy in video sequences and can also be used for de-interlacing operations.

[0169] The sink device outputs the image according to the original frame rate or a motion estimation and motion compensation module of the sink device (eg, a television) performs motion compensation, resulting in smoother motion.

[0170] The video transmission method provided by the present application is that the source device uses flag information such as whether MEMC is supported and whether SR is supported in the video capability field 06h in DCCD to comprehensively consider the capabilities of the source device itself to give the best video output strategy. The strategy indicates the video format of the output video, and the video format includes resolution and frame rate.

[0171] When the source device negotiates with the sink device, it obtains the DCCD of the sink device and analyzes the video processing capability information such as whether the sink device supports MEMC and SR.

[0172] If the sink device supports MEMC and / or SR, the source device outputs the original resolution and / or original frame rate when outputting the video, and the sink device enables MEMC for frame insertion and / or enables SR for resolution scaling.

[0173] If the host device supports MEMC and / or SR, and the source device supports MEMC and / or SR, the source device can make a comprehensive decision based on its own capabilities when outputting the video, and output the video according to the resolution and / or frame rate described by DCCD, for example, outputting the video according to the optimal resolution and / or optimal frame rate described by DCCD.

[0174] If the sink device does not support MEMC and / or SR, the source device outputs the processed resolution and / or processed frame rate when outputting the video.

[0175] In this way, the description of motion estimation, motion compensation capability and super-resolution capability is added to the video capability field contained in DCCD. Since the source device learns the video processing capability of the host device through DCCD, and the video processing capability includes super-resolution capability, motion estimation and motion compensation capability, the source device outputs the video according to the video processing capability of the host device, that is, there is no need to convert the video format obtained by the source device according to the video format already negotiated between the source device and the host device, thereby reducing the bandwidth occupied by the transmitted video and the power consumption of the source device, and the host device effectively processes the video quality to improve the video quality.

[0176] It is understandable that, in order to implement the functions in the above embodiments, the source device and the sink device include hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0177] Combined with the above Figures 1 to 4 The video transmission method provided by this embodiment is described in detail. Figure 5 , describing the video transmission device provided according to this embodiment.

[0178] Figure 5 The schematic diagram of the structure of a video transmission device provided in an embodiment of the present application is as follows. The video transmission device can be used to implement the functions of any one of the devices in the above method embodiments, and thus can also achieve the beneficial effects of the above method embodiments. In this embodiment, the video transmission device can be as follows Figure 1 The set-top box 110, smart TV 120 or any display device shown may also be Figure 2 The source device 210 or sink device 220 shown may also be a source device or sink device provided in subsequent embodiments. It should be understood that the video transmission apparatus may also be a module (such as a chip) applied to any of the aforementioned devices.

[0179] like Figure 5As shown, the video transmission device includes: a transceiver module 510 and a processing module 520. The transceiver module 510 and the processing module 520 can cooperate to implement each step in the above method embodiment. A more detailed description of the above transceiver module 510 and the processing module 520 can be directly obtained by referring to the relevant description of the device in the method embodiment shown in the above figures, and no further description is given here.

[0180] For example, the transceiver module 510 is used to execute step 310, step 320, step 410, and step 420. The transceiver module 510 is used to obtain the DCCD of the host device, the DCCD includes a video capability field, and outputs a video according to the flag information of whether the super-resolution is supported and / or the flag information of whether the motion estimation and motion compensation capability is supported contained in the video capability field, wherein the video format of the video includes a first resolution and a first frame rate. The processing module 520 is used to parse the DCCD, and obtain the flag information of whether the super-resolution is supported and / or the flag information of whether the motion estimation and motion compensation capability is supported contained in the video capability field.

[0181] Optionally, the video transmission device may further include a storage module 530 for storing DCCD and video, etc.

[0182] When the video transmission device implements the video transmission method shown in any of the above-mentioned figures through software, the video transmission device and its various units can also be software modules. The above-mentioned video transmission method is implemented by calling the software module through a processor. The processor can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD), and the above-mentioned PLD can be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0183] Understandably, Figure 5 The video transmission device shown is only an example provided in this embodiment. The video transmission device may include more or fewer units according to different video transmission processes, and this application is not limited to this.

[0184] When the video transmission device is implemented by hardware, the hardware can be implemented by a processor or a chip system. The chip system includes one or more chips, each chip including an interface circuit and a control circuit. The interface circuit is used to receive data from other devices outside the chip and transmit it to the control circuit, or send data from the control circuit to other devices outside the chip. The control circuit and the interface circuit are used to implement any possible implementation method in the above embodiments through a logic circuit or execution code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments, which will not be repeated here.

[0185] It is understood that the processor in the embodiments of the present application may be a CPU, or other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0186] Figure 5 The video transmission device shown can also be implemented by a video transmission device. Figure 6 This is a structural diagram of a video transmission device provided in the present application, the video transmission device includes: a memory 601 and at least one processor 602, the processor 602 can implement the video transmission method provided in the above embodiment, and the memory 601 is used to store software instructions corresponding to the above video transmission method.

[0187] As an optional implementation, in hardware implementation, the video transmission device may refer to a chip or chip system encapsulated with one or more processors 602. For example, when the video transmission device is used to implement the method steps in the above embodiment, the processor 602 included in the video transmission device executes the steps of the source device in the above method and its possible sub-steps. In an optional scenario, the video transmission device may also include a communication interface 603, which can be used to send and receive data. For example, the communication interface 603 is used to receive DCCD, audio and video data, or send audio and video streams, etc.; the communication interface 603 can be implemented by the interface circuit included in the video transmission device. Therefore, in some examples, the communication interface 603 may also be referred to as a transceiver of the video transmission device. In this embodiment, the communication interface 603 supports the use of a unified multimedia Internet network.

[0188] In the embodiment of the present application, the communication interface 603, the processor 602 and the memory 601 can be connected via a bus 604, and the bus 604 can be divided into an address bus, a data bus, a control bus, etc. The bus 604 can be a Peripheral Component Interconnect Express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX) or other types of buses, etc.

[0189] It is worth noting that the video transmission device can also perform Figure 5 The functions of the video transmission device shown are not described in detail here.

[0190] The video transmission device provided in this embodiment may be the above set-top box 110, smart TV 120, source device 210, etc., or other devices with video processing functions, which are not limited in this application. For example, when the aforementioned display device also has a video processing function, the video transmission device may refer to any one of the aforementioned display devices.

[0191] in addition, Figure 5 The video transmission device shown can also be implemented by a display device. When the video transmission device is implemented by a display device, this embodiment provides a possible example, such as Figure 7 As shown, Figure 7 The structural diagram of the display device provided in the present application. The display device includes: a processor 710, an external memory interface 720, an internal memory 721, a universal serial bus (USB) interface 730, a unified multimedia interconnection interface 731, an antenna 1, an antenna 2, a mobile communication module 750, a wireless communication module 760, an audio module 770, a speaker 770A, a receiver 770B, a microphone 770C, a sensor module 780, a button 790, an indicator 792, a camera 793, a display screen 794, a subscriber identification module (SIM) card interface 1-N 795, etc.

[0192] Among them, the above-mentioned sensor module 780 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0193] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the display device. In other embodiments, the display device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0194] The processor 710 may include one or more processing units, for example, the processor 710 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0195] The controller can be the nerve center and command center of the display device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0196] The processor 710 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 710 is a cache memory. The memory may store instructions or data that the processor 710 has just used or cyclically used. If the processor 710 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 710, and thus improves the efficiency of the system.

[0197] In some embodiments, the processor 710 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, a unified multimedia interconnect interface, etc.

[0198] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation on the display device. In other embodiments, the display device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0199] The wireless communication function of the display device can be implemented through antenna 1, antenna 2, mobile communication module 750, wireless communication module 760, modem processor, baseband processor, etc. In some embodiments, antenna 1 of the display device is coupled with mobile communication module 750, and antenna 2 is coupled with wireless communication module 760, so that the display device can communicate with the network and other devices through wireless communication technology.

[0200] The wired communication function of the display device can be implemented through the USB interface 730 or the unified multimedia interconnection interface 731. For example, the display device receives or sends DCCD, video stream, etc. through the bus connected by the unified multimedia interconnection interface 731.

[0201] The display device implements the display function through a GPU, a display screen 794, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 794 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 710 may include one or more GPUs that execute program instructions to generate or change display information.

[0202] The display screen 794 is used to display images, videos, etc. The display screen 794 includes a display panel.

[0203] The display device can realize the shooting function through ISP, camera 793, video codec, GPU, display screen 794 and application processor. ISP is used to process the data fed back by camera 793. Camera 793 is used to capture static images or videos. In some embodiments, the display device may include 1 or N cameras 793, where N is a positive integer greater than 1.

[0204] In this embodiment, the above display screen 794, video codec, GPU, display screen 794 and application processor may also be collectively referred to as a display unit of the host device, which is used to process and display the received video stream.

[0205] The external memory interface 720 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the display device. The external memory card communicates with the processor 710 through the external memory interface 720 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0206] The internal memory 721 may be used to store computer executable program codes, which include instructions. The processor 710 executes various functional applications and data processing of the display device by running the instructions stored in the internal memory 721. For example, in an embodiment of the present application, the processor 710 may execute the instructions stored in the internal memory 721, and the internal memory 721 may include a program storage area and a data storage area.

[0207] The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the display device (such as video and audio data, a phone book, etc.), etc. In addition, the internal memory 721 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0208] The display device can implement audio functions such as music playback and recording through the audio module 770, the speaker 770A, the receiver 770B, the microphone 770C, and the application processor.

[0209] The button 790 includes a power button, a volume button, etc. The button 790 may be a mechanical button or a touch button. The indicator 792 may be an indicator light, which may be used to indicate the charging status, the change of the power, or may be used to indicate messages, missed calls, notifications, etc.

[0210] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instruction is loaded and executed on a computer, the process or function in accordance with the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instruction can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) mode or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a magnetic disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state drive (SSD)).

[0211] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0212] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0213] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0214] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0215] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, or all or part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk, and other media that can store program codes.

[0216] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video transmission method, characterized in that: Applied to a source device or a chip in the source device, the method comprises: Acquire a device comprehensive capability description DCCD of the sink device, where the DCCD includes a video capability field, where the video capability field includes flag information of whether super-resolution is supported and / or flag information of whether motion estimation and motion compensation capabilities are supported; A video is output according to the flag information indicating whether the super-resolution is supported and / or the flag information indicating whether the motion estimation and motion compensation capabilities are supported, wherein the video format includes a first resolution and a first frame rate.

2. The method according to claim 1, characterized in that Outputting a video according to the flag information indicating whether the super-resolution is supported includes: When the flag information of whether the super-resolution of the sink device is supported indicates that the sink device supports the super-resolution capability, the video is output, and the first resolution is the original resolution of the video.

3. The method according to claim 1 or 2, characterized in that: Outputting a video according to the flag information indicating whether the super-resolution is supported includes: When the flag information of whether the sink device supports super-resolution indicates that the sink device supports super-resolution capability and the source device supports super-resolution capability, the video is output, and the first resolution is the resolution described by the DCCD.

4. The method according to claim 3, characterized in that The first resolution is the resolution described by the DCCD, including: The first resolution is the optimal resolution described by the DCCD.

5. The method according to any one of claims 1 to 4, characterized in that Outputting a video according to the flag information indicating whether the super-resolution is supported includes: When the flag information of whether the sink device supports super-resolution indicates that the sink device does not support super-resolution capability, the video is output, and the first resolution is a resolution obtained by processing the original resolution of the video.

6. The method according to any one of claims 1 to 5, characterized in that Output video based on flag information indicating whether motion estimation and motion compensation capabilities are supported, including: In a case where the flag information indicating whether the sink device supports motion estimation and motion compensation capabilities indicates that the sink device supports motion estimation and motion compensation capabilities, the video is output, and the first frame rate is an original frame rate of the video.

7. The method according to claim 6, characterized in that Output video based on flag information indicating whether motion estimation and motion compensation capabilities are supported, including: When the flag information indicating whether the host device supports motion estimation and motion compensation capabilities indicates that the host device supports motion estimation and motion compensation capabilities, and the source device supports motion estimation and motion compensation capabilities, the video is output, and the first frame rate is the frame rate described by the DCCD.

8. The method according to claim 7, characterized in that The first frame rate is a frame rate described by the DCCD, including: The first frame rate is an optimal frame rate described by the DCCD.

9. The method according to any one of claims 6 to 8, characterized in that: Output video based on flag information indicating whether motion estimation and motion compensation capabilities are supported, including: When the flag information indicating whether the host device supports motion estimation and motion compensation capabilities indicates that the host device does not support motion estimation and motion compensation capabilities, the video is output, and the first frame rate is a frame rate obtained by processing an original frame rate of the video.

10. A video transmission device, characterized in that: include: A memory, a transceiver and a processor; the memory, the transceiver and the processor are used to collaboratively execute the method according to any one of claims 1 to 9.

11. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal comprising a computer instruction stored in the memory; when the processor executes the computer instruction, the processor executes the operating steps of the method described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: Computer instructions are stored, and when the computer instructions are executed on a computing device, the operating steps of the method according to any one of claims 1 to 9 are executed.

Images