Video transmission method and device

Abstract

The present application discloses a video transmission method and device. The method is applied to the field of communications. A source device acquires a DCCD of a sink device, the DCCD comprising a video capability field. On the basis of flag information included in the video capability field and indicating whether super-resolution is supported and / or whether a motion estimation and motion compensation capability is supported, the source device outputs a video, a video format of the video comprising a first resolution and a first frame rate. Because the source device learns of the super-resolution capability and the motion estimation and motion compensation capability of the sink device via the DCCD, the source device outputs the video directly. That is, there is no need to perform video format conversion on the video acquired by the source device to conform to a video format previously negotiated between the source device and the sink device. Thus, the bandwidth occupied for video transmission and the power consumption of the source device are reduced, and the sink device performs effective image quality processing on the video, thereby improving the image quality of the video.

Description

Video transmission method and device

[0001] The present application claims priority from the Chinese patent application No. 202411281075.1 filed on September 12, 2024, and entitled "Video transmission method and device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of communication, and in particular to a video transmission method and device. BACKGROUND

[0003] At present, the video playing function is a basic and important function in intelligent devices. In the interconnection scene of a source device (such as a personal computer (PC), a mobile phone, a set-top box, etc.) and a sink device (such as a television, a projector, a display, etc.), the source device and the sink device negotiate the supported video format (such as resolution, frame rate, pixel format, pixel depth, clock, etc.). The source device sends the video according to the negotiated video format. Since there are various video formats, the source device uniformly processes different formats of videos according to the negotiated video format, which leads to waste of bandwidth and power consumption of the source device, and various problems such as poor video quality, picture jitter, and video out of synchronization of the sink device. SUMMARY

[0004] The present application provides a video transmission method and device, thereby reducing the occupied bandwidth and power consumption, and improving the video quality.

[0005] In a first aspect, a video transmission method is provided, applied to a source device or a chip in the source device, and the method comprises: obtaining a device comprehensive capability description (DCCD) of a sink device, the DCCD containing a video capability field; outputting a video according to flag information of whether super-resolution (SR) is supported and / or flag information of whether motion estimation and motion compensation (MEMC) capability is supported contained in the video capability field, wherein the video format of the video comprises a first resolution and a first frame rate.

[0006] The video transmission method provided in the application adds the description of the motion estimation and motion compensation capability and the super resolution capability in the video capability field of the DCCD. Since the source device learns whether the super resolution is supported and / or whether the motion estimation and motion compensation are supported by the sink device through the DCCD, the source device does not need to perform the video format conversion on the video acquired by the source device according to the video format which has been negotiated by the source device and the sink device, and outputs the video, thereby reducing the bandwidth occupied by the transmitted video and reducing the power consumption of the source device, and the sink device effectively processes the quality of the video and improves the quality of the video.

[0007] In a possible implementation, the outputting of the video according to the flag information of whether the super resolution is supported comprises: in a case where the flag information of whether the super resolution is supported of the sink device indicates that the sink device supports the super resolution capability, outputting the video, and the first resolution being the original resolution of the video.

[0008] In a case where the source device learns that the sink device supports the super resolution capability through the DCCD, the source device does not need to process the resolution of the video and output the video, that is, does not need to perform the resolution processing on the video acquired by the source device according to the resolution which has been negotiated by the source device and the sink device, thereby reducing the bandwidth occupied by the transmitted video, reducing the power consumption of the source device since the resolution of the video does not need to be processed, and effectively processing the quality of the original resolution video by the sink device and improving the quality of the video.

[0009] In another possible implementation, the outputting of the video according to the flag information of whether the super resolution is supported comprises: in a case where the flag information of whether the super resolution is supported of the sink device indicates that the sink device supports the super resolution capability and the source device supports the super resolution capability, outputting the video, and the first resolution being the resolution described in the DCCD.

[0010] In another possible implementation, the first resolution being the resolution described in the DCCD comprises: the first resolution being the optimal resolution described in the DCCD.

[0011] In a case where the sink device supports the super resolution capability and the source device supports the super resolution capability, the source device can comprehensively decide the resolution of the video according to the capability of the source device, output the video, and improve the quality of the video.

[0012] In another possible implementation, the outputting of the video according to the flag information of whether the super resolution is supported comprises: in a case where the flag information of whether the super resolution is supported of the sink device indicates that the sink device does not support the super resolution capability, outputting the video, and the first resolution being the resolution processed on the original resolution of the video.

[0013] When the sink device does not support the super-resolution capability, the source device can process the resolution of the video according to the capability of the source device, and improve the picture quality of the video.

[0014] In another possible implementation, outputting the video according to the flag information about whether the motion estimation and motion compensation capability is supported, comprises: when the flag information about whether the motion estimation and motion compensation capability of the sink device is supported indicates that the motion estimation and motion compensation capability of the sink device is supported, outputting the video, and the first frame rate is the original frame rate of the video.

[0015] When the source device learns from the DCCD that the motion estimation and motion compensation capability of the sink device is supported, the source device does not need to process the frame rate of the video, and outputs the video, that is, does not need to process the frame rate of the video obtained by the source device according to the frame rate negotiated by the source device and the sink device, thereby reducing the bandwidth occupied by the transmission of the video, reducing the power consumption of the source device due to the fact that the frame rate of the video does not need to be processed, and effectively processing the picture quality of the original frame rate of the video by the sink device, thereby improving the picture quality of the video.

[0016] In another possible implementation, outputting the video according to the flag information about whether the motion estimation and motion compensation capability is supported, comprises: when the flag information about whether the motion estimation and motion compensation capability of the sink device is supported indicates that the motion estimation and motion compensation capability of the sink device is supported, and the source device supports the motion estimation and motion compensation capability, outputting the video, and the first frame rate is the frame rate described by the DCCD.

[0017] In another possible implementation, the first frame rate is the optimal frame rate described by the DCCD.

[0018] When the sink device supports the motion estimation and motion compensation capability, and the source device supports the motion estimation and motion compensation capability, the source device can comprehensively decide the frame rate of the video according to the capability of the source device, and output the video, thereby improving the picture quality of the video.

[0019] In another possible implementation, outputting the video according to the flag information about whether the motion estimation and motion compensation capability is supported, comprises: when the flag information about whether the motion estimation and motion compensation capability of the sink device is supported indicates that the motion estimation and motion compensation capability of the sink device is not supported, outputting the video, and the first frame rate is the frame rate processed according to the original frame rate of the video.

[0020] When the sink device does not support the motion estimation and motion compensation capability, the source device can process the frame rate of the video according to the capability of the source device, and improve the picture quality of the video.

[0021] In a second aspect, a video transmission apparatus is provided, which comprises various modules for performing the operation steps of the method in the first aspect or any possible implementation manner of the first aspect. For example, the video transmission apparatus comprises a receiving module and a sending module. The receiving module is configured to acquire the DCCD of the sink device, and the DCCD comprises the video capability field. The sending module is configured to output the video according to the flag information about whether the super resolution is supported and / or the flag information about whether the motion estimation and motion compensation capability is supported, which are contained in the video capability field, and the video format of the video comprises the first resolution and the first frame rate.

[0022] In a possible implementation manner, when the sending module is configured to output the video according to the flag information about whether the super resolution is supported, the sending module is specifically configured to: output the video in a case where the flag information about whether the super resolution is supported of the sink device indicates that the sink device supports the super resolution capability, and the first resolution is the original resolution of the video.

[0023] In another possible implementation manner, when the sending module is configured to output the video according to the flag information about whether the super resolution is supported, the sending module is specifically configured to: output the video in a case where the flag information about whether the super resolution is supported of the sink device indicates that the sink device supports the super resolution capability, the source device supports the super resolution capability, and the first resolution is the resolution described in the DCCD.

[0024] In another possible implementation manner, the first resolution is the optimal resolution described in the DCCD.

[0025] In another possible implementation manner, when the sending module is configured to output the video according to the flag information about whether the super resolution is supported, the sending module is specifically configured to: output the video in a case where the flag information about whether the super resolution is supported of the sink device indicates that the sink device does not support the super resolution capability, and the first resolution is the resolution after processing the original resolution of the video.

[0026] In another possible implementation manner, when the sending module is configured to output the video according to the flag information about whether the motion estimation and motion compensation capability is supported, the sending module is specifically configured to: output the video in a case where the flag information about whether the motion estimation and motion compensation capability is supported of the sink device indicates that the sink device supports the motion estimation and motion compensation capability, and the first frame rate is the original frame rate of the video.

[0027] In another possible implementation manner, when the sending module is configured to output the video according to the flag information about whether the motion estimation and motion compensation capability is supported, the sending module is specifically configured to: output the video in a case where the flag information about whether the motion estimation and motion compensation capability is supported of the sink device indicates that the sink device supports the motion estimation and motion compensation capability, the source device supports the motion estimation and motion compensation capability, and the first frame rate is the frame rate described in the DCCD.

[0028] In another possible implementation, the first frame rate is an optimal frame rate described by the DCCD.

[0029] In another possible implementation, the sending module is configured to output the video according to the flag information indicating whether the motion estimation and motion compensation capability of the sink device is supported, and in particular configured to output the video when the flag information indicating whether the motion estimation and motion compensation capability of the sink device is supported indicates that the motion estimation and motion compensation capability of the sink device is not supported, and the first frame rate is a frame rate processed from an original frame rate of the video.

[0030] In a third aspect, a video transmission device is provided, which includes a memory, a transceiver and a processor, and the memory, the transceiver and the processor are configured to cooperatively execute the method in the first aspect and any one of the possible implementation manners thereof.

[0031] In a fourth aspect, a video transmission system is provided, which includes a source device and a sink device, the source device is configured to implement the functions of the source device in the first aspect and any one of the possible implementation manners thereof, and the sink device is configured to effectively perform quality processing on the resolution and frame rate of the received video to improve the quality of the video. Therefore, the video transmission system also has the beneficial effects of the method in the first aspect, which will not be described herein.

[0032] In a fifth aspect, a computer readable storage medium is provided, which stores computer instructions, and the computer instructions are configured to execute the first aspect and the possible implementation manners thereof when running on a computing device. For example, the computing device is the source device described above.

[0033] In a sixth aspect, a computer program product is provided, which includes computer instructions, and the computer instructions are configured to execute the method in the first aspect and any one of the possible implementation manners thereof when running on a computing device. For example, the computing device is the source device described above.

[0034] In a seventh aspect, a chip system is provided, which includes a processor configured to call and run a computer program from a memory, so that a computing device installed with the chip system executes the first aspect and the possible implementation manners thereof. For example, the computing device is the source device or the sink device described above.

[0035] The beneficial effects of the second aspect to the seventh aspect and the corresponding possible implementation manners of the present application will be described above in the first aspect and the corresponding possible implementation manners thereof, which will not be described herein.

[0036] On the basis of the implementation manners of the above aspects, the present application can be further combined to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0037] Fig. 1 is a schematic diagram of a video and audio transmission system according to the present application;

[0038] Fig. 2 is a schematic diagram of a video and audio codec system according to the present application;

[0039] Fig. 3 is a schematic diagram of a video processing capability negotiation process according to the present application;

[0040] Fig. 4 is a schematic diagram of a video transmission method according to the present application;

[0041] Fig. 5 is a schematic diagram of a video transmission device according to the present application;

[0042] Fig. 6 is a schematic diagram of a video transmission apparatus according to the present application;

[0043] Fig. 7 is a schematic diagram of a display apparatus according to the present application. DETAILED DESCRIPTION

[0044] For the convenience of understanding, the main terms involved in the present application are first explained.

[0045] Device Comprehensive Capability Description (DCCD): mainly defines a system framework and data structure for declaring the capabilities supported by a device, such as device parameters, performance attributes, and audio and video capabilities. The DCCD adopts a variable-length structure, and different capabilities supported by a device are encapsulated in different data blocks of the DCCD. The declaration of the capabilities supported by a device can be extended by adding extension data blocks. The DCCD describes the data content structure, and is independent of the communication protocol between devices.

[0046] Resolution: also known as resolution or resolving power, generally refers to the resolution capability of a measurement or display system, which determines the degree of detail of a bitmap image. Resolution is often used as a measure of the detail and clarity of an image, video, or display device. The higher the resolution, the clearer the displayed image and the more detailed the details.

[0047] Frame rate: refers to the frequency (rate) at which bitmap images in frames appear on a display in succession.

[0048] Super-Resolution (SR): a process of obtaining a high-resolution image from a series of low-resolution images. This technology improves the quality of low-resolution images, making them appear clearer and more detailed. This technology has applications in many fields, such as digital image processing, video processing, imaging, satellite remote sensing, etc.

[0049] Motion Estimation and Motion Compensation (MEMC): is a technology widely used in television, display and other video playing devices, is an important indicator to improve the quality of the track to estimate the movement of objects, in the original image frame inserted new motion compensation frame, compensation for the video source itself does not have the picture, reduce the picture of the drag and jitter phenomenon, so that the motion picture looks more clear and smooth.

[0050] At present, the source device converts the video format of the video according to the video format negotiated with the sink device. For example, super resolution and / or motion estimation and motion compensation are performed by the source device to convert the low resolution of the video into the high resolution required by the negotiation with the sink device, or to process the frame rate of the video. The scaling of the resolution performed by the source device fails to fully utilize the super resolution function of the sink device, resulting in poor video quality. The frame rate matching achieved by the source device through the interpolation or frame dropping method may introduce the "motion jitter" problem.

[0051] The sink device can also restore the resolution of the received video to the original resolution of the video, and then implement quality improvement through super resolution technology and picture quality (PQ) technology. The sink device can also restore the frame rate of the received video to the original frame rate of the video, and then enable motion estimation and motion compensation technology to make the motion smoother. In this way, the cost of the sink device is increased.

[0052] The source device uniformly processes different formats of videos according to the negotiated video format, and needs to perform amplification resolution, reduction resolution, frame dropping, frame insertion, pixel format downsampling and other processes. These processes may cause loss of video picture details, and the restoration and quality processing in the sink device cannot be optimal, resulting in poor video quality; the resolution amplification may cause the bandwidth to be large, the device power consumption to be large and other problems.

[0053] Among them, the video format described in the present application can refer to the format of the video, for example, the video format includes at least one of resolution, frame rate, pixel format, pixel depth and clock, etc.

[0054] In order to solve the problems of waste bandwidth and power consumption and poor video quality, the video transmission method provided in the application is as follows: a source device acquires a DCCD of a sink device, the DCCD contains a video capability field, the video capability field contains flag information of whether super resolution is supported and / or flag information of whether motion estimation and motion compensation capability is supported; and a video is output according to the flag information of whether super resolution is supported and / or the flag information of whether motion estimation and motion compensation capability is supported, the video format of the video includes a first resolution and a first frame rate. The first resolution can be an original resolution or a processed resolution of the video, and the first frame rate can be an original frame rate or a processed frame rate of the video.

[0055] The video transmission method provided in the application adds the description of the motion estimation and motion compensation capability and the super resolution capability in the video capability field of the DCCD. Since the source device learns from the DCCD whether the super resolution is supported and / or whether the motion estimation and motion compensation are supported by the sink device, the source device does not need to perform video format conversion on the video acquired by the source device according to the video format that has been negotiated by the source device and the sink device, and output the video, thereby reducing the bandwidth occupied by the transmitted video and reducing the power consumption of the source device, and the sink device effectively processes the quality of the video and improves the quality of the video.

[0056] The technical scheme related in the embodiments of the application can not only be applied to the current video and audio transmission technology or video and audio standard, but also can be applied to the future video and audio transmission technology or video and audio standard. The terms used in the embodiment part of the application are only used for explaining the specific embodiments of the application, and are not intended to limit the application.

[0057] In the embodiment, the video is a general term, and the video includes a sequence of multiple continuous frames of images, one frame corresponding to one image. The video and audio is an information application technology term, and refers to video, audio or multimedia content including video and audio.

[0058] Video streaming refers to the transmission of video data, for example, the video streaming can be processed as a stable and continuous stream through the network. The video (or video data) captured by a video capture device includes multiple frames of images, and the video data obtains a video stream after being encapsulated and packaged, the video stream includes multiple video frames, and each video frame corresponds to one frame of image.

[0059] The video transmission method provided in the embodiments of the application supports the video formats including but not limited to RGB, YCbCr4:4:4, YCbCr4:2:2, YCbCr4:2:0, ARGB, Y-only, RAW and the like.

[0060] The component bit width (or color depth) of the video data supported by the video transmission method provided in the embodiments of the present application includes, but is not limited to, one or more of the following: 8 bits, 10 bits, 12 bits or 16 bits. For example, for video data in RGB format, if the component bit width is 8 bits, then the R component of one pixel point occupies 8 bits, the G component occupies 8 bits, and the B component occupies 8 bits. It can also be described as: the video transmission method supports video data with 8bpc (bit per component), 10bpc, 12bpc or 16bpc.

[0061] The video transmission method provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0062] FIG. 1 is a schematic diagram of a video and audio transmission system provided in the embodiments of the present application. The processing process of the video can include, but is not limited to, video acquisition, video encoding, video transmission, video decoding and playing processes.

[0063] The video and audio transmission system in FIG. 1 includes a set top box 110, a smart television 120, a plurality of video and audio playing devices and a server 130. The set top box 110 accesses a network through a network cable and can receive a video and audio stream from the server 130. The network can implement the function of video and audio transmission, and can include one or more network devices, such as a network device 131 which can be a router or a switch, etc. In some optional implementation manners, the set top box 110 and the server 130 can also communicate with each other through wireless communication, which is not limited in the embodiments of the present application.

[0064] The set top box 110 is a video and audio processing device, which is used to implement the functions of receiving, processing and pushing a video stream or a video and audio stream, etc. In some possible cases, the set top box 110 can also be referred to as an Internet television set top box, a network high-definition player or other devices. For example, the set top box 110 can be a television box provided by a network operator, or a television box purchased by a user. The hardware implementation of the set top box 110 can refer to the description of FIG. 6 below, which is not described herein.

[0065] The smart television 120 is a display device with a video and audio processing function, which implements the functions of receiving, processing, pushing and playing a video stream or a video and audio stream, etc. In some possible cases, the smart television 120 can be referred to as a conference panel, a smart television or a projector, etc. The hardware implementation of the smart television 120 can refer to the description of FIG. 6 or FIG. 7 below, which is not described herein.

[0066] The plurality of audio-video playback devices include audio-video playback device 121 to audio-video playback device 124. For example, the audio-video playback devices can include, but are not limited to, a multimedia control platform or other devices supporting audio-video playback functions, such as a virtual reality (VR) terminal device or an augmented reality (AR) terminal device, and the like. The hardware implementation of the audio-video playback devices can refer to the description of FIG. 7 below, which is not described here.

[0067] In this embodiment, the set-top box 110 and the smart television 120 are connected through a network, for example, can be connected through an audio-video interface network, the set-top box 110 and each audio-video playback device can also be connected through an audio-video interface network, and the smart television 120 and each audio-video playback device can also be connected through an audio-video interface network. For example, the audio-video interface network can be a wired network supporting video and audio-video transmission. Alternatively, the audio-video interface network can be referred to as a unified multimedia interconnection network. In some implementations, the unified multimedia interconnection network can also be other names.

[0068] The unified multimedia interconnection network can support non-compressed video and compressed video transmission, and can also support a variety of advanced features such as quick video transport (QVT), auto low latency (ALLM), dynamic frame rate refresh (DFR), etc. The unified multimedia interconnection network can also support LPCM format audio-video defined by IEC 60958 and a variety of HDR protocols, such as the HDR protocol specified in T / UWA 005.1-2022, such as HDR Vivid, etc. The unified multimedia interconnection network also supports encryption control protection and the like for audio-video data transmission.

[0069] The server 130 can be an application server or an authentication and authorization server. The server 130 can provide video services, game services, message services, music services, authentication and authorization services, etc. In one example, the functions of multiple services can be integrated on the server 130, for example, game services and music services can be deployed on the server 130. In another example, it can also be that the server 130 integrates the functions of part of the services, for example, the server 130 deploys part of the services of the game services and part of the services of the video services. The server 130 can also provide multiple virtual machines using virtualization technology, and each virtual machine provides a service. The deployment form of the server is not limited by the embodiments of the present application. The network device 131 is connected to the server 130 through wireless or wired means. The schematic diagram of FIG. 1 is only an example, and other devices can also be included in the network, which are not shown in FIG. 1.

[0070] FIG. 1 is only a schematic diagram, and other devices can be included in the audiovisual transmission system, which are not shown in FIG. 1. Embodiments of the present application do not limit the number and types of devices included in the system.

[0071] Based on the audiovisual transmission system shown in FIG. 1, FIG. 2 is a schematic diagram of an audiovisual coding system provided by the present application, which includes a source device 210 and a sink device 220. The source device 210 establishes a communication connection with the sink device 220 through a unified multimedia interconnection network.

[0072] The source device 210 described above can implement the function of audiovisual coding. As shown in FIGS. 1 and 2, the source device 210 can be the set-top box 110 or the smart television 120 shown in FIG. 1, and the source device 210 can also be an audiovisual control center with audiovisual coding capability, for example, the audiovisual control center includes one or more servers.

[0073] The source device 210 can include a data source 211, a preprocessing module 212, an audiovisual sending adapter 213, and a communication interface 214.

[0074] The data source 211 can include or be any type of electronic device for collecting audiovisuals, and / or any type of source audiovisual generation device, such as a computer graphics processor for generating a computer animation scene or any type of device for obtaining and / or providing source audiovisuals, computer-generated source audiovisuals. The data source 211 can be any type of memory or storage for storing the source audiovisuals described above. The source audiovisuals described above can include multiple audiovisual streams or images collected by multiple audiovisual collection devices (such as video cameras), such as Ultra High Definition (UHD) video, High Definition (HD) video, 4K video, 8K video, and the like.

[0075] The preprocessing module 212 is configured to receive source audiovisuals and pre-process the source audiovisuals to obtain audiovisuals or multiple frames of images. For example, the preprocessing performed by the preprocessing module 212 can include color format conversion (such as conversion from RGB to YCbCr), octree structuring, audiovisual splicing, etc., audio track merging and deletion or channel number adjustment, and the like.

[0076] The audiovisual sending adapter 213 is configured to receive audiovisuals or images and encode the audiovisuals or images to obtain encoded data. In some optional cases, the encoded data can also be referred to as a bitstream. If the encoded data is obtained by encoding audiovisual data, the bitstream refers to an audiovisual stream, and if the encoded data is obtained by encoding video data, the bitstream refers to a video stream.

[0077] The communication interface 214 in the source device 210 can be configured to receive encoded data (e.g., a video stream or an audio-visual stream) and transmit the encoded data (or a version of the encoded data after any other processing) to another device, such as the sink device 220 or any other device, via a unified multimedia interconnect network for storage, display, playback, or image reconstruction, etc.

[0078] Optionally, the source device 210 includes a bitstream buffer configured to store a bitstream corresponding to one or more encoded units.

[0079] The sink device 220 described above can be configured to perform audio-visual decoding. As shown in FIG. 1, when the source device 210 is a set-top box 110, the sink device 220 can be any one of the smart television 120 or the audio-visual playback devices shown in FIG. 1. When the source device 210 is the smart television 120, the sink device 220 can be any one of the audio-visual playback devices.

[0080] The sink device 220 can include an audio-visual playback unit 221, a post-processing module 222, an audio-visual reception adapter 223, and a communication interface 224.

[0081] The communication interface 224 in the sink device 220 can be configured to receive encoded data (or a version of the encoded data after any other processing) from the source device 210 or from any other source device, such as a storage device.

[0082] The communication interface 214 and the communication interface 224 can be configured to transmit data via a direct communication link between the source device 210 and the sink device 220, such as a direct wired connection, etc., via the unified multimedia interconnect network shown in FIG. 2. The unified multimedia interconnect network can be configured as described above with reference to FIG. 1, and thus will not be described again here.

[0083] The communication interface 224 can correspond to the communication interface 214 and can be configured to transmit data and process the data using any type of corresponding transmission decoding or processing and / or de-encapsulation to obtain audio-visual data.

[0084] The communication interface 224 and the communication interface 214 can each be configured as a unidirectional communication interface or a bidirectional communication interface as indicated by the corresponding arrows of the unified multimedia interconnect network from the source device 210 to the sink device 220 in FIG. 2, and can be configured to transmit and receive messages, etc., to establish a connection and transmit other information, such as information related to the communication link or information related to the audio-visual data (e.g., descriptive information DIP of the audio-visual data).

[0085] The audio-visual reception adapter 223 can be configured to receive encoded data and decode the encoded data to obtain decoded data (e.g., video or audio-visual).

[0086] The post-processing module 222 is configured to perform post-processing on the decoded data to obtain post-processed data (e.g., an image to be displayed or a video / audio to be played). The post-processing performed by the post-processing module 222 can include, for example, color format conversion (e.g., from YCbCr to RGB), octree reconstruction, video / audio splitting and fusion, etc., or any other processing for generating data to be output by the video / audio playback unit 221.

[0087] The video / audio playback unit 221 is configured to receive the post-processed data for display or playback to a user or viewer, etc. The video / audio playback unit 221 can be or include any type of display for representing a reconstructed image, such as an integrated or external display screen or monitor. For example, the display screen can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display screen. The video / audio playback unit 221 can also include one or more video / audio playback modules, each of which can be a speaker, a smart speaker, a power amplifier, etc.

[0088] As an optional implementation, the source device 210 and the sink device 220 can transmit the encoded data through a data forwarding device. For example, the data forwarding device can be a router or a switch, etc. It is worth noting that the data forwarding device needs to support the unified multimedia interconnection network.

[0089] In the embodiments of the present application, the source device can also be referred to as a video / audio sending device, a video / audio sending end, etc., and the sink device can also be referred to as a video / audio receiving device, a video / audio playback device, etc. In the embodiments, the source device and the sink device are connected through the unified multimedia interconnection network.

[0090] In a first possible application scenario, the source device can be the set-top box 110 in FIG. 1, and the sink device can be the smart television 120 in FIG. 1. The set-top box negotiates with the smart television, i.e., the set-top box obtains the DCCD of the smart television, the video capability field contained in the DCCD is increased to include descriptions of the super-resolution capability and the motion estimation and motion compensation capability, and the set-top box pushes a video to the smart television according to the super-resolution capability and / or the motion estimation and motion compensation capability of the smart television.

[0091] In a second possible application scenario, the source device can be the set-top box 110 in FIG. 1, and the sink device can be any of the audio-video playback devices in FIG. 1, such as any one of the audio-video playback device 121 to the audio-video playback device 124. The set-top box negotiates with the audio-video playback device, i.e., the set-top box acquires the DCCD of the audio-video playback device, and the set-top box pushes the audio-video data to the audio-video playback device.

[0092] In a third possible application scenario, the source device can be the smart television 120 in FIG. 1, and the sink device can be any of the audio-video playback devices in FIG. 1, such as any one of the audio-video playback device 121 to the audio-video playback device 124. The smart television negotiates with the audio-video playback device, i.e., the smart television acquires the DCCD of the audio-video playback device, and the smart television pushes the audio-video data to the audio-video playback device.

[0093] The above three possible application scenarios are only examples provided by the present embodiment, and should not be construed as a limitation on the present application. In other possible examples, the source device can be any one of the audio-video playback devices in FIG. 1 (such as the audio-video playback device 121), and the sink device can be another audio-video playback device different from the aforementioned audio-video playback device (such as the audio-video playback device 122).

[0094] The process of the source device and the sink device negotiating the DCCD is described below.

[0095] FIG. 3 is a flow diagram of a video processing capability negotiation provided by the present application. Here, the video transmission method of the present application is taken as an example performed by the source device 210 or the chip in the source device 210, and the sink device 220 or the chip in the sink device 220. Referring to FIG. 3, the video transmission method provided by the present embodiment includes steps 310 to 320.

[0096] In step 310, the source device acquires the DCCD of the sink device.

[0097] In step 320, the sink device acquires the DCCD of the source device.

[0098] The process of the source device acquiring the DCCD of the sink device is described below. The process of the sink device acquiring the DCCD of the source device can be referred to the description of the process of the source device acquiring the DCCD of the sink device.

[0099] The sink device is provided with information indicating the capability of the sink device, and the source device queries the capability of the sink device, including that the source device acquires capability description information of the sink device, and the process of the source device acquiring the capability information of the sink device includes: the source device sends a query message to the sink device, the query message being used for querying the capability of the sink device; and then, the sink device sends (feedbacks) the capability description information of the sink device to the source device, and the source device receives the capability description information sent by the sink device, which can indicate the capability of the sink device.

[0100] The capability description information of the sink device can include information indicating the video processing capability of the sink device. For example, the video processing capability includes information indicating whether the sink device supports the super-resolution capability and / or whether the sink device supports the motion estimation and motion compensation capability. In some possible cases, the capability description information can also be referred to as a capability descriptor, which can include one or more bits of information.

[0101] In some cases, the capability description information of the sink device can also be referred to as a device comprehensive capability description (DCCD), and a region in the sink device where the DCCD is stored is referred to as a DCCD region or simply a DCCD, which is not limited in the present application.

[0102] In the embodiments of the present application, whether the super-resolution capability is supported and / or whether the motion estimation and motion compensation capability is supported is declared in the DCCD, for example, the sink device supports the super-resolution capability, supports the motion estimation and motion compensation capability, does not support the super-resolution capability, or does not support the motion estimation and motion compensation capability.

[0103] The source device queries the super-resolution capability and the motion estimation and motion compensation capability of the sink device through the DCCD, and then the sink device feeds back the super-resolution capability and the motion estimation and motion compensation capability of the sink device to the source device.

[0104] In some embodiments, the DCCD contains a video capability field, which mainly describes the video-related capability supported by the display. The video capability field contains flag information indicating whether the super-resolution is supported and / or flag information indicating whether the motion estimation and motion compensation capability is supported.

[0105] For example, the data structure of the video capability field is shown in Table 1.

[0106] Table 1

[0107] As shown in Table 1, the offset 06h of the video capability field includes the flag information of whether the MEMC is supported and the flag information of whether the SR is supported. In some embodiments, according to different values of the bit, it can be indicated whether the super resolution is supported and / or whether the motion estimation and motion compensation are supported.

[0108] For example, the value of the bit is 1, which indicates that the super resolution or the motion estimation and motion compensation are supported, and the value of the bit is 0, which indicates that the super resolution or the motion estimation and motion compensation are not supported. For another example, the value of the bit is 0, which indicates that the super resolution or the motion estimation and motion compensation are supported, and the value of the bit is 1, which indicates that the super resolution or the motion estimation and motion compensation are not supported.

[0109] Optionally, the DCCD can also store other content in the audio and video processing capability of the sink device, such as the audio processing capability (whether the sink device supports processing pure audio data packets), the HDR display capability (whether the sink device supports displaying a specific type of HDR video, such as the PQ HDR), or the like.

[0110] In an optional example, the sink device includes a memory in which the capability description information is stored. The memory can include, but is not limited to, a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium well known in the art, and the like.

[0111] In another optional example, the sink device includes a dedicated register, and the state of the dedicated register is used to indicate the capability description information. For example, if the value indicated by the register is 1, the capability description information indicates that the sink device has the super resolution capability; if the value indicated by the register is 0, the capability description information indicates that the sink device does not have the super resolution capability. For example, if the value indicated by the register is 1, the capability description information indicates that the sink device has the motion estimation and motion compensation capability; if the value indicated by the register is 0, the capability description information indicates that the sink device does not have the motion estimation and motion compensation capability.

[0112] The process of transmitting the video by the source device and the sink device is described below.

[0113] The source device obtains the DCCD fed back by the sink device, analyzes the DCCD, and learns the flag information indicating whether the sink device supports and / or the flag information indicating whether the motion estimation and motion compensation capability is supported. In the case that the source device outputs the video, whether the resolution and frame rate of the video are processed is determined according to the flag information indicating whether the super resolution is supported and / or the flag information indicating whether the motion estimation and motion compensation capability is supported, the video is output, and the video is transmitted to the sink device.

[0114] FIG. 4 is a flowchart of a video transmission method provided by the present application. Here, the video transmission method provided by the present application is taken as an example performed by the source device 210 or the chip in the source device 210, the sink device 220 or the chip in the sink device 220. Referring to FIG. 4, the video transmission method provided by the embodiment of the present application includes steps 410 to 420.

[0115] In step 410, the source device outputs the video according to the flag information indicating whether the super resolution is supported and / or the flag information indicating whether the motion estimation and motion compensation capability is supported. The video format of the video includes a first resolution and a first frame rate. Correspondingly, the sink device receives the video from the source device.

[0116] The video sent by the source device to the sink device can be one frame of data in the video data to be sent by the source device, and the video corresponds to the effective data (or effective pixels, or effective pixel data) of one frame image in the video.

[0117] The source device provided by the present application is not limited in terms of the source of the video. The source device can obtain the video from a storage device, or receive the video sent by another device.

[0118] In some embodiments, after the source device obtains the video, the output strategy of the video can be determined according to the flag information indicating whether the super resolution is supported and / or the flag information indicating whether the motion estimation and motion compensation capability is supported by the sink device. The output strategy indicates the video format of the video. The source device outputs the video, transmits the video to the sink device, and the video format of the video can be the video format indicated by the output strategy. For example, the output strategy indicates whether the resolution and / or frame rate of the video is processed. If the output strategy indicates that the resolution and / or frame rate of the video is processed, the output strategy can further indicate the processed resolution and / or processed frame rate.

[0119] The output of the video by the source device according to the flag information indicating whether the super resolution is supported by the sink device is described below.

[0120] In a first possible implementation, in the case that the flag information indicating whether the super resolution is supported by the sink device indicates that the sink device supports the super resolution capability, the video is output, and the first resolution included in the video format of the video is the original resolution of the video.

[0121] The source device learns that the sink device has super-resolution capability, and after the source device obtains the video, the source device directly outputs the video with the original resolution without processing the resolution of the video. For example, the original resolution of the video obtained by the source device is 4K, and the first resolution of the video output by the source device is 4K.

[0122] In some embodiments, the DCCD obtained by the source device further includes a CTA Timing field, and the CTA Timing field is used to describe video timing information supported by the CTA protocol included in the DCCD. The data structure of the CTA Timing field is shown in Table 2.

[0123] Table 2

[0124] The CTA Timing Code refers to the VIC number of Table 1 Video Format Timings in the CTA-861-G protocol. When an unknown VIC is identified during analysis at the opposite end, the VIC Code can be ignored, and subsequent analysis can continue.

[0125] For example, the VIC codes in the CTA Timing field are arranged in a preferred order, i.e., the first CTA Timing is the most preferred Timing in the CTA Timing field, the second is the second preferred, and so on.

[0126] The original resolution can also be a resolution that can be recognized by the sink device. For example, the original resolution can be a resolution indicated by any one of the video formats shown in Table 2.

[0127] In the second possible implementation, in a case where the flag information indicating whether the super-resolution of the sink device is supported indicates that the sink device supports the super-resolution capability and the source device supports the super-resolution capability, the video is output, and the video format of the video includes a first resolution that is the resolution described by the DCCD.

[0128] The source device supports the super-resolution capability, which means that the source device has the capability of resolution processing, and in a case where the source device learns that the sink device has the super-resolution capability, after the source device obtains the video, the source device outputs the video according to the capability of the source device, i.e., the resolution of the video is processed according to the resolution described by the DCCD, and the resolution of the output video is the processed resolution. For example, the original resolution of the video obtained by the source device is 4K, and the first resolution of the video output by the source device is 8K.

[0129] In some embodiments, the first resolution included in the video format of the video is the optimal resolution described in the DCCD. For example, the first CTA Timing in Table 2 indicates that the most recommended preferred Timing in the CTA Timing field, and the original resolution of the video is converted to the resolution indicated by the most recommended preferred Timing.

[0130] Alternatively, the first resolution included in the video format of the video is the sub-optimal resolution described in the DCCD.

[0131] In a third possible implementation, in a case where the flag information of whether the super resolution of the sink device is supported indicates that the sink device does not support the super resolution capability, the video is output, and the first resolution included in the video format of the video is a resolution processed from the original resolution of the video.

[0132] The source device learns that the sink device does not have the super resolution capability, and the source device processes the resolution of the video after obtaining the video. The resolution of the output video is the processed resolution. For example, the source device can process the original resolution of the video according to the resolution negotiated with the sink device, that is, the resolution of the video output by the source device is the resolution negotiated with the sink device. The resolution negotiated with the sink device can be any resolution shown in Table 2. For another example, the source device can process the original resolution of the video according to the resolution described in the DCCD, and the resolution of the video output by the source device is the sub-optimal resolution described in the DCCD. For example, the original resolution of the video obtained by the source device is 2K, and the first resolution of the video output by the source device is 4K.

[0133] In the above scheme, the original resolution of the video can be any resolution shown in Table 2. Alternatively, the resolution of the video obtained by the source device can also not be any resolution shown in Table 2. In this case, the source device can process the original resolution of the video according to the resolution negotiated with the sink device, that is, the resolution of the video output by the source device is converted to the resolution negotiated with the sink device. Alternatively, the source device converts the original resolution of the video to any resolution shown in Table 2.

[0134] The following describes the output of the video by the source device according to the flag information of whether the motion estimation and motion compensation capability of the sink device is supported.

[0135] In a first possible implementation, in a case where the flag information of whether the motion estimation and motion compensation capability of the sink device is supported indicates that the sink device supports the motion estimation and motion compensation capability, the video is output, and the first frame rate included in the video format of the video is the original frame rate of the video.

[0136] The source device learns that the sink device has the motion estimation and motion compensation capability, and after the source device acquires the video, the source device directly outputs the video with the original frame rate without processing the frame rate of the video. For example, the original frame rate of the video acquired by the source device is 60 frames per second, and the first frame rate of the video output by the source device is 4K60 frames per second.

[0137] The original frame rate can also be a frame rate that can be recognized by the sink device. For example, the original frame rate can be a frame rate indicated by any one of the video formats shown in Table 2.

[0138] In the second possible implementation, in a case where the flag information indicating whether the motion estimation and motion compensation capability of the sink device is supported indicates that the sink device supports the motion estimation and motion compensation capability and the source device supports the motion estimation and motion compensation capability, the video is output, and the first frame rate included in the video format of the video is the frame rate described in the DCCD.

[0139] The source device supports the motion estimation and motion compensation capability, which means that the source device has the capability of frame rate processing, and in a case where the source device learns that the sink device has the motion estimation and motion compensation capability, after the source device acquires the video, the source device makes a comprehensive decision according to its own capability and outputs the video according to the frame rate described in the DCCD. That is, the frame rate of the video is processed according to the frame rate described in the DCCD, and the frame rate of the output video is the processed frame rate. For example, the original frame rate of the video acquired by the source device is 30 frames per second, and the first frame rate of the video output by the source device is 60 frames per second.

[0140] In some embodiments, the first frame rate included in the video format of the video is the optimal frame rate described in the DCCD. For example, in Table 2, the first CTA Timing is indicated as the most recommended preferred Timing in the CTA Timing field, and the original frame rate of the video is converted into the frame rate indicated by the most recommended preferred Timing.

[0141] Optionally, the first frame rate included in the video format of the video is a suboptimal frame rate included in the DCCD.

[0142] In the third possible implementation, in a case where the flag information indicating whether the motion estimation and motion compensation capability of the sink device is supported indicates that the sink device does not support the motion estimation and motion compensation capability, the video is output, and the first frame rate included in the video format of the video is a frame rate processed from an original frame rate of the video.

[0143] The source device learns that the sink device does not have the motion estimation and motion compensation capability. After the source device obtains the video, the source device processes the frame rate of the video, and the frame rate of the output video is the processed frame rate. For example, the source device can process the original frame rate of the video according to the frame rate negotiated with the sink device, that is, the frame rate of the video output by the source device is the frame rate negotiated with the sink device. The frame rate negotiated with the sink device can be any one of the frame rates shown in Table 2. For another example, the source device can process the original frame rate of the video according to the frame rate described in the DCCD, and the frame rate of the video output by the source device is the suboptimal frame rate described in the DCCD. For example, the original frame rate of the video obtained by the source device is 30 frames per second, and the first frame rate of the video output by the source device is 60 frames per second.

[0144] In the above scheme, the original frame rate of the video can be any one of the frame rates shown in Table 2. Alternatively, the frame rate of the video obtained by the source device can also not be any one of the frame rates shown in Table 2. In this case, the source device can process the original frame rate of the video according to the frame rate negotiated with the sink device, that is, convert the frame rate of the video output by the source device to the frame rate negotiated with the sink device. Alternatively, the source device converts the original frame rate of the video to any one of the frame rates shown in Table 2.

[0145] The above possible implementations of the super-resolution capability and the motion estimation and motion compensation capability can be combined in any manner, which is not limited herein.

[0146] For example, in the case where the sink device supports the super-resolution capability and supports the motion estimation and motion compensation capability, the output video has a video format including a first resolution being the original resolution of the video and a first frame rate being the original frame rate of the video.

[0147] For another example, in the case where the sink device does not support the super-resolution capability and supports the motion estimation and motion compensation capability, the output video has a first resolution being the original resolution of the video converted, and a first frame rate being the original frame rate of the video.

[0148] For another example, in the case where the sink device supports the super-resolution capability and does not support the motion estimation and motion compensation capability, the output video has a first resolution being the original resolution of the video, and a first frame rate being the original frame rate of the video converted.

[0149] For another example, in the case where the sink device does not support the super-resolution capability and does not support the motion estimation and motion compensation capability, the output video has a first resolution being the original resolution of the video converted, and a first frame rate being the original frame rate of the video converted.

[0150] The source device transmits the video to the sink device at the original resolution and original frame rate of the video, which can effectively reduce the transmission bandwidth and save power consumption. The source device transmits the video to the sink device at the original resolution and original frame rate of the video, and the sink device can more effectively process the picture quality according to the original resolution and original frame rate to achieve the optimal effect.

[0151] Step 420, the sink device processes the video from the source device.

[0152] In the case that the sink device supports super-resolution capability, after receiving the video from the source device, the sink device performs super-resolution processing on the first resolution of the video. For example, the first resolution of the video is converted to a second resolution. The first resolution can be 4K, and the second resolution can be 8K.

[0153] For example, the first resolution included in the video format of the video can be the original resolution of the video. For another example, the first resolution included in the video format of the video can also be the resolution after the original resolution processing.

[0154] In some embodiments, the source device compares the resolution of the received video with a target resolution, and if the resolution of the received video is lower than the target resolution, performs super-resolution processing on the resolution of the video. The target resolution can be the resolution of the display.

[0155] For example, the super-resolution processing process mainly includes the following steps.

[0156] 1. Feature extraction: first, the low-resolution image is extracted through the convolutional neural network to obtain a feature map. The purpose of this step is to extract useful feature information from the low-resolution image to prepare for the subsequent high-resolution image generation.

[0157] 2. Generate high-resolution image: then, the feature map is input into the generator, and the generator generates high-resolution images layer by layer through a series of convolutional layers and deconvolutional layers. This step is the core of the super-resolution processing, and the generator tries to reconstruct a high-resolution image through the learned feature information.

[0158] 3. Calculate perspective loss: compare the generated high-resolution image with the real high-resolution image to calculate the perspective loss. The perspective loss is a loss function used to measure the difference between the image generated by the generator and the real image, which takes into account factors such as brightness, contrast and structure of the image.

[0159] 4. Optimize parameters: finally, the parameters of the generator and the discriminator are optimized so that the generated high-resolution image is closer and closer to the real high-resolution image. This step adjusts the parameters of the generator and the discriminator to continuously improve the quality of the generated image, and finally achieves an effect similar to the real high-resolution image.

[0160] The entire super-resolution processing process is an iterative optimization process, by continuously adjusting and optimizing parameters, so that the generated image quality is continuously improved, and finally the super-resolution target is achieved. This process not only involves image processing techniques, but also includes deep learning and machine learning methods, making it possible to generate high-resolution images from low-resolution images.

[0161] For example, the Scaler module of the sink device (such as a television) processes, such as high-end televisions can perform super-resolution processing through AI SR, and the picture quality is better.

[0162] In the case where the sink device supports motion estimation and motion compensation capabilities, the sink device receives video from the source device and performs motion estimation and motion compensation processing on the first frame rate of the video. For example, the sink device enables motion estimation and motion compensation capabilities and performs frame interpolation processing on the video. For example, the first frame rate of the video is converted to a second frame rate. The first frame rate can be 30 frames per second, and the second frame rate can be 60 frames per second.

[0163] For example, the first frame rate included in the video format of the video is the original frame rate of the video. For another example, the first frame rate included in the video format of the video can also be the frame rate after processing the original frame rate.

[0164] In some embodiments, the source device compares the frame rate of the received video with the target frame rate, and if the frame rate of the received video is lower than the target frame rate, the frame rate of the video is processed by motion estimation and motion compensation. The target frame rate can be the frame rate of the display.

[0165] The main purpose of the motion estimation and motion compensation technology is to improve the smoothness and clarity of video playback, especially in handling fast motion scenes. Motion estimation and motion compensation technology predicts the motion trajectory of objects by analyzing the changes between two consecutive frames, and generates one or more intermediate images based on this, filling the gap between the original video frames, i.e. the sink device enables motion estimation and motion compensation capabilities to perform frame interpolation. Thus, it reduces motion blur and improves dynamic clarity, making high-speed motion pictures look smoother and more natural.

[0166] The process of motion estimation mainly includes finding the best match between a block in the current frame and a block in the reference frame, which is achieved by calculating the similarity between the current block and the block at each position in the reference frame. The similarity measurement includes MAD, MSE, NCCF, etc., and the sum of absolute transformed differences (SATD) is used in the H.264 standard. After finding the best match, motion estimation outputs a motion vector (MV), which is the position coordinate of the reference block relative to the current block. This motion vector is then used in the motion compensation process.

[0167] Motion compensation includes using the motion vector obtained by motion estimation to obtain the corresponding block from the reference frame and comparing it with the corresponding block in the current frame to obtain a residual block. If the matching degree of the two blocks is high, the residual block is basically 0, which means that the data compression efficiency is high. Motion compensation includes global motion compensation and block motion compensation, which is a method of describing the difference between adjacent frames and is used to reduce spatial redundancy in video sequences. This method is used in video compression / video codec to reduce spatial redundancy in video sequences and can also be used for deinterlacing.

[0168] The sink device outputs according to the original frame rate or performs motion compensation by the motion estimation and motion compensation module of the sink device (such as a television set), and the motion is smoother.

[0169] The video transmission method provided in the application gives the best video output strategy by comprehensively considering the capability of the source device itself through the flag information such as whether the MEMC is supported in the video capability field 06h in the DCCD, whether the SR is supported, etc., and the strategy indicates the video system of the output video, which includes resolution and frame rate.

[0170] When the source device negotiates with the sink device, the DCCD of the sink device is obtained, and whether the MEMC is supported, whether the SR is supported and other video processing capability information are analyzed.

[0171] If the sink device supports the MEMC or / and the SR, the source device outputs the original resolution or / and the original frame rate when outputting the video, and the MEMC is started by the sink device to perform the interpolation or / and the SR is started by the sink device to perform the resolution scaling processing.

[0172] If the sink device supports the MEMC or / and the SR, the source device supports the MEMC or / and the SR, and the source device can comprehensively decide according to its own capability when outputting the video, and outputs the video according to the resolution or / and the frame rate described in the DCCD, for example, outputs the video according to the optimal resolution or / and the optimal frame rate described in the DCCD.

[0173] If the sink device does not support MEMC or / and SR, the source device outputs the processed resolution or / and the processed frame rate when outputting the video.

[0174] Thus, the video capability field included in the DCCD is increased to describe the motion estimation and motion compensation capability and the super resolution capability. Since the source device learns the video processing capability of the sink device through the DCCD, the video processing capability includes the super resolution capability, the motion estimation and motion compensation capability, the source device outputs the video according to the video processing capability of the sink device, that is, the source device does not need to perform video format conversion on the video obtained by the source device according to the video format negotiated by the source device and the sink device, thereby reducing the bandwidth occupied by the transmitted video and reducing the power consumption of the source device, and the sink device effectively processes the video to improve the video quality.

[0175] It can be understood that, in order to implement the functions in the above-described embodiments, the source device and the sink device include corresponding hardware structures and / or software modules for performing various functions. Those skilled in the art should easily realize that, in combination with the units and method steps of the examples described in the embodiments disclosed in the present application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application scenario and design constraints of the technical solution.

[0176] The video transmission method provided by the present embodiment is described in detail above in combination with FIGS. 1 to 4. The video transmission apparatus provided by the present embodiment will be described below in combination with FIG. 5.

[0177] FIG. 5 is a structural schematic diagram of a video transmission apparatus provided by an embodiment of the present application. The video transmission apparatus can be used to implement the functions of any one of the devices in the above-described method embodiments, and thus can also achieve the beneficial effects possessed by the above-described method embodiments. In the present embodiment, the video transmission apparatus can be the set-top box 110, the smart television 120 or any one of the display devices shown in FIG. 1, or can be the source device 210 or the sink device 220 shown in FIG. 2, or can be the source device or the sink device provided in subsequent embodiments. It should be understood that the video transmission apparatus can also be a module (such as a chip) applied to any one of the foregoing devices.

[0178] As shown in FIG. 5, the video transmission apparatus includes a transceiver module 510 and a processing module 520. The transceiver module 510 and the processing module 520 can cooperatively implement various steps in the above-described method embodiments. More detailed descriptions of the transceiver module 510 and the processing module 520 can be directly obtained by referring to the related descriptions of the devices in the method embodiments shown in the foregoing figures, which will not be described herein.

[0179] For example, the transceiver module 510 is configured to perform the step 310, the step 320, the step 410 and the step 420. The transceiver module 510 is configured to acquire the DCCD of the sink device, the DCCD containing the video capability field, and output a video according to the flag information of whether the super resolution is supported and / or the flag information of whether the motion estimation and motion compensation capability is supported contained in the video capability field, the video format of the video including the first resolution and the first frame rate. The processing module 520 is configured to parse the DCCD, and acquire the flag information of whether the super resolution is supported and / or the flag information of whether the motion estimation and motion compensation capability is supported contained in the video capability field.

[0180] Optionally, the video transmission apparatus can further include a storage module 530 configured to store the DCCD, the video, and the like.

[0181] When the video transmission apparatus is implemented by software, the video transmission apparatus and each unit thereof can be a software module. The processor invokes the software module to implement the video transmission method described above. The processor can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD), and the PLD can be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0182] It can be understood that the video transmission apparatus shown in FIG. 5 is merely an example provided by the present embodiment, and a video transmission apparatus can include more or fewer units according to different video transmission processes, which is not limited in the present application.

[0183] When the video transmission apparatus is implemented by hardware, the hardware can be implemented by a processor or a chip system. The chip system includes one or more chips, and each chip includes an interface circuit and a control circuit. The interface circuit is configured to receive data from other devices outside the chip and transmit the data to the control circuit, or send data from the control circuit to other devices outside the chip. The control circuit and the interface circuit are configured to implement the method of any possible implementation manner of the above-described embodiments by a logic circuit or an execution code instruction. The beneficial effects can be referred to the description of any aspect of the above-described embodiments, which will not be described here again.

[0184] It can be understood that the processor in the embodiments of the present application can be a CPU, and can also be other general-purpose processors, digital signal processors (DSP), ASIC, FPGA or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0185] The video transmission apparatus shown in FIG. 5 can also be implemented by a video transmission device. FIG. 6 is a structural schematic diagram of a video transmission device provided by the present application. The video transmission device includes a memory 601 and at least one processor 602. The processor 602 can implement the video transmission method provided by the above-mentioned embodiments. The memory 601 is used to store software instructions corresponding to the above-mentioned video transmission method.

[0186] As an optional implementation, in hardware implementation, the video transmission device can be a chip or a chip system packaged with one or more processors 602. For example, when the video transmission device is used to implement the method steps in the above-mentioned embodiments, the processor 602 included in the video transmission device executes the steps of the source device in the above-mentioned method and possible sub-steps thereof. In an optional case, the video transmission device can also include a communication interface 603, which can be used to transceive data. For example, the communication interface 603 is used to receive DCCD, audio and video data, or transmit audio and video streams, etc. The communication interface 603 can be implemented by an interface circuit included in the video transmission device. Therefore, in some examples, the communication interface 603 can also be referred to as a transceiver of the video transmission device. In the present embodiment, the communication interface 603 supports the use of a unified multimedia inter-network.

[0187] In the embodiments of the present application, the communication interface 603, the processor 602 and the memory 601 can be connected through a bus 604, which can be divided into an address bus, a data bus, a control bus, etc. The bus 604 can be a Peripheral Component Interconnect Express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX) or other types of buses, etc.

[0188] It is worth noting that the video transmission device can also perform the functions of the video transmission apparatus shown in FIG. 5, which will not be repeated here.

[0189] The video transmission device provided by the embodiment can be the set-top box 110, the smart television 120, the source device 210, or other devices with video processing functions, which are not limited in the present application. For example, when the display device also has video processing functions, the video transmission device can refer to any of the aforementioned display devices.

[0190] In addition, the video transmission apparatus shown in FIG. 5 can also be implemented by a display device. When the video transmission apparatus is implemented by a display device, the present embodiment provides a possible example, as shown in FIG. 7, which is a structural schematic diagram of a display device provided by the present application. The display device includes a processor 710, an external memory interface 720, an internal memory 721, a universal serial bus (USB) interface 730, a uniform multimedia interface 731, an antenna 1, an antenna 2, a mobile communication module 750, a wireless communication module 760, an audio module 770, a speaker 770A, a receiver 770B, a microphone 770C, a sensor module 780, a key 790, an indicator 792, a camera 793, and a display screen 794, a subscriber identification module (SIM) card interface 1-N 795, and the like.

[0191] Among them, the sensor module 780 can include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor.

[0192] It can be understood that the structure shown in the embodiment does not constitute a specific limitation on the display device. In other embodiments, the display device can include more or fewer components than those shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0193] The processor 710 can include one or more processing units, for example: the processor 710 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0194] The controller can be the nerve center and command center of the display device. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0195] The processor 710 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 710 is a cache memory. The memory can save instructions or data that the processor 710 has just used or repeatedly uses. If the processor 710 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 710, thereby improving the efficiency of the system.

[0196] In some embodiments, the processor 710 can include one or more interfaces. The interface can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, a universal multimedia interface, etc.

[0197] It can be understood that the interface connection relationship between the modules in the embodiment is only illustrative and does not constitute a structural limitation on the display device. In other embodiments, the display device can also use different interface connection modes or combinations of multiple interface connection modes in the above embodiments.

[0198] The wireless communication function of the display device can be implemented by the antenna 1, the antenna 2, the mobile communication module 750, the wireless communication module 760, the modem processor, and the baseband processor, and the like. In some embodiments, the antenna 1 and the mobile communication module 750 of the display device are coupled, and the antenna 2 and the wireless communication module 760 are coupled, so that the display device can communicate with the network and other devices through the wireless communication technology.

[0199] The wired communication function of the display device can be implemented by the USB interface 730 or the unified multimedia interface 731. For example, the display device receives or transmits DCCD, video stream, and the like through the bus connected by the unified multimedia interface 731.

[0200] The display function of the display device can be implemented by the GPU, the display screen 794, and the application processor, and the like. The GPU is a microprocessor for image processing, connected to the display screen 794 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 710 can include one or more GPUs that execute program instructions to generate or change display information.

[0201] The display screen 794 is used to display images, videos, and the like. The display screen 794 includes a display panel.

[0202] The display device can implement the shooting function by the ISP, the camera 793, the video codec, the GPU, the display screen 794, and the application processor, and the like. The ISP is used to process the data fed back by the camera 793. The camera 793 is used to capture still images or videos. In some embodiments, the display device can include one or N cameras 793, N being a positive integer greater than 1.

[0203] In the embodiment, the display screen 794, the video codec, the GPU, the display screen 794, and the application processor, and the like above can also be collectively referred to as a display unit of the sink device, which is used to process and display the received video stream.

[0204] The external memory interface 720 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the display device. The external storage card communicates with the processor 710 through the external memory interface 720 to implement the data storage function. For example, music, video, and the like files are saved in the external storage card.

[0205] The internal memory 721 can be used to store computer executable program codes, which include instructions. The processor 710 performs various functional applications of the display device and data processing by running the instructions stored in the internal memory 721. For example, in the embodiments of the present application, the processor 710 can perform the instructions stored in the internal memory 721, which can include a storage program area and a storage data area.

[0206] The storage program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The storage data area can store data created during use of the display device (such as audio and video data, a phone book, etc.), etc. In addition, the internal memory 721 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0207] The display device can realize audio functions through an audio module 770, a speaker 770A, a receiver 770B, a microphone 770C, an application processor, etc. For example, music playing, recording, etc.

[0208] The keys 790 include a power-on key, a volume key, etc. The keys 790 can be mechanical keys. They can also be touch keys. The indicator 792 can be an indicator light, which can be used to indicate a charging state, a power change, and can also be used to indicate a message, a missed call, a notification, etc.

[0209] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, all or part generates the processes or functions in the embodiments of the present application. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a magnetic disk, a magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid state drive (SSD)), etc.

[0210] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional modules is taken as an example, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0211] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0212] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0213] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0214] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a flash memory, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various program code storage media.

[0215] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of video transmission, characterized by, The method is applied to a source device or a chip in the source device, and comprises: obtaining device comprehensive capability description (DCCD) of a sink device, wherein the DCCD comprises a video capability field, and the video capability field comprises flag information of whether super resolution is supported and / or flag information of whether motion estimation and motion compensation capability is supported; outputting a video according to the flag information of whether super resolution is supported and / or the flag information of whether motion estimation and motion compensation capability is supported, wherein a video format of the video comprises a first resolution and a first frame rate.

2. The method of claim 1, wherein, According to the flag information of whether super resolution is supported, outputting a video, comprising: in the case that the flag information of whether super resolution is supported of the sink device indicates that the sink device supports super resolution capability, outputting the video, and the first resolution is a resolution described in the DCCD.

3. The method according to claim 1 or 2, characterized in that, According to the flag information of whether super resolution is supported, outputting a video, comprising: in the case that the flag information of whether super resolution is supported of the sink device indicates that the sink device does not support super resolution capability, outputting the video, and the first resolution is a resolution processed from an original resolution of the video.

4. The method of claim 3, wherein, According to the flag information of whether motion estimation and motion compensation capability is supported, outputting a video, comprising: in the case that the flag information of whether motion estimation and motion compensation capability is supported of the sink device indicates that the sink device supports motion estimation and motion compensation capability, outputting the video, and the first frame rate is an original frame rate of the video.

5. The method according to any one of claims 1-4, characterized in that, According to the flag information of whether motion estimation and motion compensation capability is supported, outputting a video, comprising: in the case that the flag information of whether motion estimation and motion compensation capability is supported of the sink device indicates that the sink device supports motion estimation and motion compensation capability, and the source device supports motion estimation and motion compensation capability, outputting the video, and the first frame rate is a frame rate described in the DCCD.

6. The method according to any one of claims 1-5, characterized in that, The first frame rate is the frame rate described in the DCCD, comprising: the first frame rate is an optimal frame rate described in the DCCD.

7. The method of claim 6, wherein, According to the flag information of whether motion estimation and motion compensation capability is supported, outputting a video, comprising: in the case that the flag information of whether motion estimation and motion compensation capability is supported of the sink device indicates that the sink device does not support motion estimation and motion compensation capability, outputting the video, and the first frame rate is a frame rate processed from an original frame rate of the video.

8. The method of claim 7, wherein, comprising: a memory, a transceiver and a processor; the memory, the transceiver and the processor are used to cooperatively execute the method in any one of claims 1 to 9.

9. The method according to any one of claims 6-8, characterized in that, ​ ​ 10. A video transmission device, comprising: ​ ​ 11. A chip, characterized by comprise one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from a memory of the electronic device and send the signals to the processors, the signals comprising computer instructions stored in the memory; when the processors execute the computer instructions, the processors are caused to perform the operational steps of the method of any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, computer instructions are stored, which, when executed on a computing device, perform the operational steps of the method of any one of claims 1-9.

Images