Video transmission method and device

Abstract

The invention discloses a video transmission method and equipment, and relates to the technical field of multimedia. A source device can directly query a processing capability of a sink device on a compressed video, and the source device transmits a first VBP, a compression parameter message and the compressed video to the sink device only when the sink device has a decompression capability on the compressed video, the first VBP comprising a compressed video mode field being a first value, the compression parameter message comprising a second value being a second value, and the compressed video comprising a second value being a third value. The first value is used for indicating that the video compression mode is a compressed video, and the compression parameter message is used for indicating compression parameter information of the compressed video. In this way, the problems that an invalid communication process is generated between the devices due to the fact that the sink device does not have the decompression capacity of the compressed video, and the transmission resource bandwidth between the devices is occupied are solved. And moreover, the compressed video can be transmitted, so that the problem that the transmission efficiency is affected due to the fact that the compressed video is transmitted by adopting a DP protocol or an HDMI protocol is avoided, and the robustness of data transmission and the efficiency of transmitting the compressed data can be improved.

Description

Video transmission method and device Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a video transmission method and device. Background Art

[0002] With the development of big data, artificial intelligence (AI), and cloud computing technologies, various types of multimedia data are constantly emerging. For example, for wired transmission of video data between different devices, different devices can use the display port (DP) bus or the high-definition multimedia interface (HDMI) bus to transmit video data. However, while the DP protocol or HDMI protocol generally supports video transmission, transmission efficiency is affected when using the DP protocol or HDMI protocol to transmit compressed video.

[0003] Summary of the Invention

[0004] The present application provides a video transmission method and device to solve the problem that the DP protocol or the HDMI protocol usually supports video transmission, but the transmission efficiency is affected when the DP protocol or the HDMI protocol is used to transmit compressed video.

[0005] This application adopts the following technical solution.

[0006] In a first aspect, the present application provides a video transmission method. The video transmission method is executed by a source device or a chip in the source device, such as a set-top box. Exemplarily, the video transmission method includes querying the sink device's processing capability for compressed video, and transmitting a first vertical blanking packet (VBP), a compression parameter packet, and compressed video to the sink device if the sink device has the capability to decompress the compressed video. The compressed video mode field included in the first VBP is a first value, and the first value is used to indicate that the compressed video mode is compressed video. The compression parameter packet is used to indicate: compression parameter information of the compressed video.

[0007] In this application, the source device can directly query the sink device's processing capability for compressed video, and only when the sink device has the ability to decompress the compressed video will the source device transmit the first vertical blanking message VBP, the compression parameter message, and the compressed video to the sink device. This avoids the problem of invalid communication between devices caused by the sink device's lack of decompression capability for compressed video, as well as the problem of occupied transmission resource bandwidth between devices. In addition, this application can transmit compressed video, avoiding the problem of transmission efficiency being affected by using the DP protocol or HDMI protocol to transmit compressed video, which is conducive to improving the robustness of data transmission and the efficiency of transmitting compressed data.

[0008] Exemplarily, the first value may be 1b.

[0009] In a possible example, the video stream transmitted by the source device to the sink device includes the first VBP, the compression parameter message, and the compressed video.

[0010] Exemplarily, the video stream includes multiple video frames, each of which includes a first VBP, a compression parameter message, and compressed video.

[0011] In a possible implementation, transmitting the compressed video to the sink device includes: transmitting an audio sample packet (AVP) to the sink device, where the audio sample packet AVP is used to carry the compressed video.

[0012] In this application, compressed video is carried by AVP, so that the source device can transmit the compressed video to the sink device, which is beneficial to improving the transmission efficiency of the compressed video.

[0013] In a possible example, one AVP carries multiple groups of 8-bit data. In other words, the AVP carries compressed video at a granularity of 8 bits.

[0014] In a possible implementation, the compressed video complies with a perceptual lossless compression (PLC) standard.

[0015] Compared with the DP protocol or HDMI protocol, which only transmits compressed videos obtained by specific compression methods (such as display stream compression (DSC)), the present application can transmit compressed videos that comply with the PLC standard. The above-mentioned specific compression method does not include PLC. Therefore, it avoids the problem of being affected when transmitting compressed videos that comply with the PLC standard, improves the robustness of data transmission and the efficiency of transmitting compressed videos that comply with the PLC standard.

[0016] In a possible implementation, the descriptive information type byte included in the compression parameter message is the second value.

[0017] For example, the descriptive information type byte in the compression parameter message is fixed to 0x02.

[0018] In a possible implementation, when a compression parameter message is transmitted to a sink device, a message start flag included in the compression parameter message is 1, and a message end flag included in the compression parameter message is 1.

[0019] For example, in the flags field of the message header of the compression parameter message, S=1 and E=1. S indicates the message start flag, and E indicates the message end flag.

[0020] In a possible implementation, if multiple compression parameter messages are transmitted to the sink device, the message start flag included in the first compression parameter message among the multiple compression parameter messages is 1, and the message end flag included in the last compression parameter message among the multiple compression parameter messages is 1.

[0021] In the present application, since the compression parameter message includes a message start flag and a message end flag, the message start flag and the message end flag can enable the host device to clearly understand the start or end of the compression parameter message sent by the source device to the host device, which is conducive to improving the host device's parsing of the compression parameter message, thereby improving the efficiency of the host device in decoding the compressed video according to the compression message parameters carried by the compression parameter message.

[0022] For example, if the compression parameter information of a compressed video exceeds 32 bytes, multiple compression parameter messages are required to carry it. The flags field in the first compression parameter message among the multiple compression parameter messages corresponds to S=1 and E=0. The flags field in the last compression parameter message among the multiple compression parameter messages corresponds to S=0 and E=1.

[0023] In a possible implementation, the source device and the sink device are connected via an audio and video interface network.

[0024] For example, the audio and video interface network is a unified multimedia interconnection interface bus.

[0025] In one possible implementation, if the compressed video transmission is completed, the above-mentioned video transmission method further includes: transmitting a second vertical blanking message VBP and uncompressed video to the host device, the compressed video mode field included in the second VBP is a third value, and the third value is used to indicate that the compressed video mode is uncompressed video.

[0026] Exemplarily, the third value may be 0b.

[0027] In the present application, after the compressed video is transmitted, by changing the compressed video mode field in the VBP from the first value to the third value, the host device can be reminded that the video data currently transmitted is uncompressed video, and then the host device can promptly adopt corresponding processing for the uncompressed video, thereby improving the processing efficiency of the host device for the uncompressed video.

[0028] In a second aspect, the present application provides a video transmission method. The video transmission method is executed by a sink device or a chip in the sink device, such as a display. Exemplarily, the sink device provides compressed video processing capabilities that support source device queries. If the sink device has the ability to decompress compressed video, the video transmission method includes receiving a first vertical blanking message (VBP), a compression parameter message, and compressed video transmitted by a source device, wherein the first VBP includes a compressed video mode field with a first value indicating that the compressed video mode is compressed video, and the compression parameter message indicates compression parameter information of the compressed video.

[0029] In the present application, the sink device has the ability to process compressed video that supports source device query, and then the source device can query the sink device's processing capability for compressed video. Only when the sink device has the ability to decompress the compressed video, the source device transmits the first vertical blanking message VBP, the compression parameter message, and the compressed video to the sink device. In this way, the invalid communication process between devices caused by the sink device's lack of decompression capability of the compressed video and the problem of occupied transmission resource bandwidth between devices are avoided. In addition, the present application can transmit compressed video, which is beneficial to avoid the problem of transmission efficiency being affected by using the DP protocol or HDMI protocol to transmit compressed video, and improves the robustness of data transmission and the efficiency of transmitting compressed data.

[0030] For other possible implementations of the second aspect, reference may be made to any possible implementation of the first aspect described above, and details will not be repeated here.

[0031] In a third aspect, the present application provides a video device, which includes a module for executing the method of the first aspect or any possible implementation of the first aspect, or includes a module for executing the method of the second aspect or any possible implementation of the second aspect.

[0032] In a fourth aspect, the present application provides a chip. The chip includes a control circuit and an interface circuit. The control circuit is used to query the host device's processing capability for compressed video. The control circuit and the interface circuit cooperate to execute the above-mentioned first aspect or any possible implementation of the first aspect, and / or the interface circuit is used to receive a first vertical blanking message VBP, a compression parameter message, and compressed video transmitted by a source device. The compressed video mode field included in the first VBP is a first value, and the first value is used to indicate that the compressed video mode is compressed video. The compression parameter message is used to indicate: compression parameter information of the compressed video. The control circuit and the interface circuit cooperate to execute the above-mentioned second aspect or any possible implementation of the second aspect. Exemplarily, the chip can be the audio and video sending adapter in the host device shown in the first aspect, or the chip can be the audio and video receiving adapter in the host device shown in the second aspect.

[0033] In a fifth aspect, the present application provides a video processing device. The video processing device may be the source device in the first aspect. The video processing device includes: a memory and a transceiver. The memory is used to store compressed video; the transceiver is used to transmit a first vertical blanking message VBP, a compression parameter message and compressed video. The compressed video mode field included in the first VBP is a first value, and the first value is used to indicate that the compressed video mode is compressed video. The compression parameter message is used to indicate: compression parameter information of the compressed video. The memory and the transceiver are used to collaboratively execute the method in the first aspect or any optional implementation method of the first aspect. Exemplarily, the video processing device is used to implement the function of the source device in the first aspect.

[0034] In a sixth aspect, the present application provides a display device. The display device may also be referred to as an audio-visual playback device, a video playback device, or the host device in the second aspect, and the display device includes: a transceiver and a display unit. The transceiver is used to receive a first vertical blanking message VBP, a compression parameter message, and a compressed video. The compressed video mode field included in the first VBP is a first value, and the first value is used to indicate that the compressed video mode is compressed video. The compression parameter message is used to indicate: compression parameter information of the compressed video. The display unit is used to process and display the first vertical blanking message VBP, the compression parameter message, and the compressed video. The transceiver and the display unit are used to collaboratively execute the method in the second aspect or any optional implementation method in the second aspect. Exemplarily, the display device is used to implement the functions of the host device in the second aspect.

[0035] In a seventh aspect, the present application provides a video transmission system. This audio transmission system includes the video processing device provided in the fifth aspect and the display device provided in the sixth aspect. The video processing device and the display device are connected via an audio / video interface network. The video processing device can be used to implement the functions of the source device in the first aspect, and the display device can be used to implement the functions of the sink device in the second aspect. Therefore, this video transmission system can also achieve the beneficial effects of the methods in the first and second aspects above, which will not be elaborated here.

[0036] In an eighth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes computer software instructions. When the computer software instructions are executed in a computing device, the computing device executes the operating steps of the method described in the first aspect or any possible implementation of the first aspect, or executes the operating steps of the method described in the second aspect or any possible implementation of the second aspect. For example, the computing device is the aforementioned source device or sink device.

[0037] In a ninth aspect, the present application provides a computer program product. When the computer program product is executed on a computer, the computer program product causes the computing device to perform the operating steps of the method described in the first aspect or any possible implementation of the first aspect, or to perform the operating steps of the method described in the second aspect or any possible implementation of the second aspect. For example, the computer is the aforementioned source device or sink device.

[0038] Regarding the beneficial effects of the third to ninth aspects, reference may be made to the description of any implementation in the first or second aspects, and no further details will be given here. Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a schematic diagram of a video transmission system provided by the present application;

[0040] FIG2 is a schematic diagram of an audio and video encoding and decoding system provided by the present application;

[0041] FIG3 is a flow chart of a video transmission method provided by the present application;

[0042] FIG4 is a schematic diagram of the structure of a video frame provided by the present application;

[0043] FIG5 is a schematic diagram of AVP bearer provided by this application;

[0044] FIG6 is a flow chart of a video transmission method provided by this application.

[0045] FIG7 is a schematic structural diagram of a video device provided by the present application;

[0046] FIG8 is a schematic structural diagram of a video processing device provided by this application;

[0047] FIG9 is a schematic structural diagram of a sink device provided in this application. DETAILED DESCRIPTION

[0048] The present application provides a video transmission method, which includes: a source device queries a sink device for its processing capability of compressed video; if the sink device has the capability to decompress the compressed video, the source device transmits a first VBP, a compression parameter message, and compressed video to the sink device; the first VBP includes a compressed video mode field with a first value, which is used to indicate that the compressed video mode is compressed video; and the compression parameter message is used to indicate: compression parameter information of the compressed video.

[0049] In this application, the source device can directly query the sink device's processing capability for compressed video, and only when the sink device has the ability to decompress the compressed video will the source device transmit the first VBP, compression parameter message, and compressed video to the sink device. This avoids the problem of invalid communication between devices caused by the sink device's lack of decompression capability for compressed video, as well as the problem of occupied transmission resource bandwidth between devices. In addition, this application can transmit compressed video, avoiding the problem of transmission efficiency being affected by using the DP protocol or HDMI protocol to transmit compressed video, which is conducive to improving the robustness of data transmission and the efficiency of transmitting compressed data.

[0050] The technical solutions involved in this application may be applicable not only to current audio and video transmission technologies or audio and video standards, but also to future audio and video transmission technologies or audio and video standards. The terms used in the implementation methods of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0051] Video streaming refers to the transmission of video data, i.e., a video stream can be processed as a stable and continuous flow over a network. A video stream consists of multiple video frames, each of which corresponds to an image.

[0052] Descriptive information packet (DIP or DIP message): used to describe different data in a data packet.

[0053] Perceptual lossless compression (PLC) video is video (or audio and video data) encoded using the PLC standard. This PLC video can meet subjective losslessness criteria. The PLC compression process primarily consists of four parts: signal processing, pattern prediction, quantization, and coefficient encoding.

[0054] During signal processing, PLC supports dividing the signal into equal-sized rectangular slices for parallel processing. The smallest unit in the encoding process is a group of 16×2 pixels. At the interface level, the output bit stream is allowed to be divided into fixed-size sub-stream segments according to the channel to which it belongs, and output after interleaving and reorganization. In the mode prediction process, PLC includes three types of prediction modes: point prediction, ordinary intra-frame prediction, and block copy intra-frame prediction. The bit cost of each mode (including the code length after quantization of the residual) and the degree of reconstruction distortion jointly determine the optimal coding mode for the coding unit. It is worth noting that while retaining the texture fitting advantage of the largest-sized coding unit in similar standards, PLC supports highly parallel hardware processing with its prediction mode dependency elimination design.

[0055] During quantization and coefficient encoding, PLC uses shifted quantization and semi-fixed-length coefficient coding to achieve energy aggregation of prediction residuals while ensuring low hardware costs. Furthermore, PLC supports pixel-by-pixel fine-tuning of quantization parameters at specific locations, significantly improving the subjective reconstruction quality of texture details without increasing bit overhead.

[0056] In this embodiment, video is a general term that includes a sequence of multiple consecutive frames, with one frame corresponding to one image. Video and audio are information application technology terms that refer to video, audio, or multimedia content including video and audio.

[0057] In order to make the description of the following embodiments clear and concise, an introduction to related technologies is first given.

[0058] Figure 1 is a schematic diagram of a video transmission system provided by the present application. The video processing process may include but is not limited to: video acquisition, video encoding, video transmission, video decoding and playback.

[0059] The video transmission system in Figure 1 includes a set-top box 110, a smart TV 120, multiple video and audio playback devices, and a server 130. The set-top box 110 is connected to the operator's network via a network cable and can receive video and audio streams from the server 130. The network can implement video and audio transmission functions and can include one or more network devices, such as a router or switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate via wireless communication, which is not limited in this application.

[0060] The set-top box 110 is a video and audio processing device that is used to implement functions such as receiving, processing, and pushing video streams or video and audio streams. In some possible scenarios, the set-top box 110 may also be called an Internet TV set-top box, a network high-definition player, or other similar devices. For example, the set-top box 110 may refer to a TV box provided by a network operator or a TV box purchased by a user. For the hardware implementation of the set-top box 110, reference may be made to the description of FIG. 7 below, which will not be described in detail here.

[0061] Smart TV 120 is a sink device with video and audio processing capabilities that implements functions such as receiving, processing, pushing, and playing video or audio streams. In some possible scenarios, Smart TV 120 may refer to a video and audio device such as a conference tablet, smart TV, or projector, but this application is not limited to this. The hardware implementation of Smart TV 120 can be referred to in the description of Figures 7 or 8 below and will not be detailed here.

[0062] The multiple audio and video playback devices include audio and video playback devices 121 to 124. For example, these audio and video playback devices may include, but are not limited to, multimedia control platforms or other devices that support audio and video playback functions, such as virtual reality (VR) terminal devices or augmented reality (AR) terminal devices. The hardware implementation of the audio and video playback devices can be found in the description of FIG. 8 below and is not detailed here.

[0063] In this embodiment, the set-top box 110 and the smart TV 120 can be connected via a channel 125. The set-top box 110 and each audio and video playback device can also be connected via the channel 125. The smart TV 120 and each audio and video playback device can also be connected via the channel 125. Exemplarily, the channel 125 can be a signal line / link / network / channel that supports video and audio transmission. For example, the channel 125 refers to a unified multimedia interconnection channel, which can also be called a unified multimedia interconnection network.

[0064] A unified multimedia interconnection channel (UMC) is a channel connected based on the unified multimedia interconnection interface provided by the source and sink devices. This UMC can be used to connect a charger to charge electronic devices (such as the aforementioned smart TVs, set-top boxes, or audio and video playback devices), to transfer data between electronic devices and peripheral devices, and to connect headphones to play audio. This unified multimedia interconnection interface can also be used to connect other electronic devices, such as augmented reality devices. When the UMC is used to implement data communication between devices, it can support both uncompressed and compressed video transmission, as well as advanced features such as quick video transport (QVT), auto low latency mode (ALLM), and dynamic frame rate refresh (DFR). Furthermore, the UMC can support LPCM audio and video formats defined by IEC 60958, as well as various high-dynamic range (HDR) protocols, such as those specified in T / UWA 005.1-2022, such as HDR Vivid. The UMC also supports encryption control and protection for audio and video data transmission. In some optional implementations, the channel 125 may also refer to other types of channels that can implement the functions supported by the above-mentioned unified multimedia interconnection channel.

[0065] In this embodiment, the unified multimedia interconnection interface is suitable for transmitting high-speed signals between audio and video devices, third-party protocol devices (such as universal serial bus (USB) / peripheral component interconnect express (PCIe) / Ethernet), etc. A device that uses the unified multimedia interconnection interface is called a unified multimedia interconnection device, and a system composed of multiple unified multimedia interconnection devices connected together is called a unified multimedia interconnection system. The capabilities provided by the unified multimedia interconnection interface include: ①, bidirectional transmission of audio and video, meeting the audio and video transmission requirements of devices such as TVs, PCs, and mobile phones, and supporting audio and video content transmission protection, visual lossless compression, etc. ②, third-party protocol data transmission, realizing the interaction between the unified multimedia interconnection device and third-party protocol devices, such as supported third-party protocols including USB3, and Ethernet, PCIe, etc. will be supported in the future. ③, bidirectional power supply, meeting the power supply requirements of electronic devices whose power does not exceed a certain power value, such as the certain power value of 480W, 500W or other values.

[0066] It is worth noting that in some examples, the communication network composed of channels for realizing audio and video transmission between devices is also called an audio and video interface network or an audio and video interface network.

[0067] The server 130 may be an application server or an authentication and authorization server. The server 130 may provide video services, game services, message services, music services, authentication and authorization services, and the like. In one example, the functions of multiple services may be integrated on the server 130. For example, a game service and a music service may be deployed on the server 130. In another example, the functions of some services may be integrated on the server 130. For example, some services of the game service and some services of the video service may be deployed on the server 130. The server 130 may also utilize virtualization technology to provide multiple virtual machines, and the virtual machines provide various services. The embodiments of the present application do not limit the deployment form of the server. The network device 131 is connected to the server 130 via a wireless or wired mode. FIG1 is merely a schematic diagram. Other devices may also be included in the network, which are not shown in FIG1.

[0068] It is understood that the above-mentioned audio and video is a general term. Audio and video include multiple video frames, each of which corresponds to a group of packets carrying the audio and video data to be parsed and played. Figure 1 is merely a schematic diagram. The audio and video transmission system may also include other devices, which are not shown in Figure 1. The embodiments of this application do not limit the number and type of each device included in this system.

[0069] FIG2 is a schematic diagram of an audio / video codec system provided by the present application, based on the audio / video transmission system shown in FIG1 . The audio / video codec system includes a source device 210 and a sink device 220. The source device 210 establishes a communication connection with the sink device 220 via an audio / video interface network. The audio / video interface network can be a unified multimedia interconnect signal line, a unified multimedia interconnect link, a unified multimedia interconnect network, or a unified multimedia interconnect channel. The unified multimedia interconnect channel is used as an example for description below.

[0070] The above-mentioned source device 210 can realize the function of audio and video encoding. As shown in Figure 1, the source device 210 can be a set-top box 110 or a smart TV 120. The source device 210 can also be an audio and video control center with audio and video encoding capabilities. For example, the audio and video control center includes one or more servers.

[0071] The source device 210 may include a data source 211 , a pre-processing module 212 , an audio and video transmission adapter 213 , and a communication interface 214 .

[0072] The data source 211 may include or may be any type of electronic device for capturing audio and video, and / or any type of source audio and video generating device, such as a computer graphics processor for generating computer animation scenes, or any type of device for acquiring and / or providing source audio and video, or for computer-generating source audio and video. The data source 211 may be any type of memory or storage for storing the aforementioned source audio and video. The aforementioned source audio and video may include multiple audio and video streams or images captured by multiple audio and video capturing devices (such as cameras), such as ultra high definition (UHD) video, high definition (HD) video, 4K video, and the like.

[0073] The pre-processing module 212 is configured to receive source audio and video and pre-process the source audio and video to obtain audio and video or multiple frames of images. For example, the pre-processing performed by the pre-processing module 212 may include color format conversion (e.g., from RGB to YCbCr), octree structuring, audio and video splicing, audio track merging and deletion, or channel number adjustment.

[0074] The video and audio transmission adapter 213 is used to receive video and audio or images and encode the video and audio, images, or images to obtain coded data (e.g., using a PLC algorithm). In some optional situations, the coded stream (coded data) obtained by encoding can also be called a bit stream. If the coded data is obtained by encoding video and audio data, the bit stream refers to the video and audio stream.

[0075] The communication interface 214 in the source device 210 can be used to receive encoded data (such as a video stream or an audio / video stream) and send the encoded data (or a version of the encoded data after any other processing) to another device such as the host device 220 or any other device through a unified multimedia interconnection channel for storage, display, playback, or image reconstruction.

[0076] In the example of Figure 2, the audio and video sending adapter 213 and the communication interface 214 in the source device 210 can be called the controller of the source device 210 or the unified multimedia interconnection controller, and the audio and video receiving adapter 223 and the communication interface 223 in the sink device 220 can be called the controller of the sink device 220 or the unified multimedia interconnection controller.

[0077] Optionally, the source device 210 includes a bitstream buffer, which is used to store bitstreams corresponding to one or more coding units.

[0078] The sink device 220 can implement the function of video and audio decoding. As shown in FIG1 , the sink device 220 can be any one of the smart TV 120 or the video and audio playback device shown in FIG1 .

[0079] The sink device 220 may include a video and audio playback unit 221 , a post-processing module 222 , a video and audio receiving adapter 223 , and a communication interface 224 .

[0080] The communication interface 224 in the sink device 220 is configured to receive the encoded data (or a version of the encoded data after any other processing has been performed on the encoded data) from the source device 210 or any other source device such as a storage device.

[0081] Communication interface 214 and communication interface 224 can be used to connect the source device 210 and the sink device 220 through a direct communication link, such as a direct wired connection, as shown in the unified multimedia interconnection channel in Figure 2. For details about the unified multimedia interconnection channel, please refer to the description of Figure 1 and will not be repeated here.

[0082] The communication interface 224 corresponds to the communication interface 214, for example, and can be used to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded data (such as video stream or audio and video stream).

[0083] Both the communication interface 224 and the communication interface 214 can be configured as a unidirectional communication interface, as indicated by the arrow pointing from the source device 210 to the corresponding unified multimedia interconnection channel of the sink device 220 in Figure 2, or a bidirectional communication interface, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link, or data transmission such as encoded compressed data transmission, etc.

[0084] The video and audio receiving adapter 223 is used to receive encoded data and decode the encoded data to obtain decoded data (video or video and audio, etc.).

[0085] The post-processing module 222 is used to post-process the decoded data to obtain post-processed data (such as an image to be displayed or audio and video to be played). The post-processing performed by the post-processing module 222 may include, for example, color format conversion (such as from YCbCr to RGB), octree reconstruction, audio and video splitting and fusion, or any other processing for generating data for output by the audio and video playback unit 221.

[0086] The audio and video playback unit 221 is used to receive post-processed data for display or playback to a user or viewer, etc. The audio and video playback unit 221 can be or include any type of display for representing the reconstructed image, such as an integrated or external display screen or display. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display screen. The audio and video playback unit 221 can also include one or more audio and video playback modules, each of which can refer to a speaker, a smart speaker, an amplifier, etc.

[0087] As an optional implementation, the source device 210 and the sink device 220 can transmit encoded data through a data forwarding device. For example, the data forwarding device can be a router or a switch. It is worth noting that the data forwarding device needs to support a unified multimedia interconnection interface. It is understandable that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration and does not constitute a structural limitation on the source device and the sink device. In other embodiments of the present application, the structural limitation of the source device and the sink device can also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0088] The implementation of the video transmission method provided in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0089] Here, the video transmission method of an embodiment of the present application is executed by the source device 210 and the sink device 220 shown in Figure 2 as an example. Figure 3 is a flow chart of a video transmission method provided by the present application. Among them, the source device 31 is used to implement the functions of the source device 210, and the sink device 32 is used to implement the functions of the sink device 220. In this embodiment, the source device 31 can also be referred to as an audio and video sending device, an audio and video sending terminal, or a video sending device, and the sink device 32 can also be referred to as an audio and video receiving device, an audio and video playback device, or a video receiving device. The source device 31 and the sink device 32 are connected via a channel 33, which can refer to a unified multimedia interconnection channel. The specific implementation of the unified multimedia interconnection channel can be referred to in the description of Figure 2 above and will not be repeated here. The compressed video complies with the PLC standard, and therefore, the compressed video can be referred to as PLC video. In some examples, the channel 33 can also be referred to as a connection channel, a first channel, or other names.

[0090] In a first possible application scenario, the source device 31 may be the set-top box 110 in Figure 1, and the sink device 32 may be the smart TV 120 in Figure 1. For example, the set-top box pushes video and audio data to the smart TV.

[0091] In a second possible application scenario, the source device 31 may be the set-top box 110 in FIG1 , and the sink device 32 may be any video and audio playback device in FIG1 , such as any one of the video and audio playback devices 121 to 124. For example, the set-top box 110 pushes a video stream to the video and audio device 121.

[0092] In a third possible application scenario, the source device 31 may be the smart TV 120 in FIG1 , and the sink device 32 may be any video and audio playback device in FIG1 , such as any one of the video and audio playback devices 121 to 124 . For example, the smart TV 120 pushes a video stream to the video and audio playback device 121 .

[0093] The above three possible application scenarios are merely examples provided in this embodiment and should not be construed as limiting the present application. In other possible examples, the source device 31 may be any of the video and audio playback devices in FIG1 (e.g., the video and audio playback device 121 ), and the sink device 32 may be another video and audio playback device different from the aforementioned video and audio playback device (e.g., the video and audio playback device 122 ).

[0094] Referring to FIG. 3 , the video transmission method provided in this embodiment includes the following steps S310 to S330 .

[0095] S310 : The source device 31 queries the sink device 32 for its processing capability for compressed video.

[0096] The processing capability of the sink device 32 for the compressed video includes the decompression capability of the sink device 32 for the compressed video.

[0097] Exemplarily, the source device 31 queries the capability description information of the sink device 32 , where the capability description information indicates the processing capability of the sink device 32 for compressed video.

[0098] In some possible scenarios, the capability description information may also be referred to as a capability descriptor, which may include one or more bits of information.

[0099] In one possible example, the sink device 32 includes a memory that stores the capability description information. The memory may include, but is not limited to, random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, a hard disk, a removable hard disk, a CD-ROM, or any other storage medium known in the art.

[0100] In another possible example, sink device 32 includes a dedicated register whose state is used to indicate the capability description information. If the value indicated by the register is 1, the capability description information indicates that sink device 32 has the capability to decompress compressed video, i.e., supports decompressing compressed video. If the value indicated by the register is 0, the capability description information indicates that sink device 32 does not have the capability to decompress compressed video, i.e., does not support decompressing compressed video.

[0101] It is worth noting that since the capability description information indicates whether the host device 32 has the ability to decompress compressed video, in some cases, the capability description information is also called device comprehensive capability description information. The area in the host device 32 that stores the DCCD information is called the DCCD area or simply DCCD, which is not limited in this application.

[0102] It is worth noting that the DCCD may also store other content describing the audio and video processing capabilities of the host device, such as audio processing capabilities (whether the host device supports processing pure audio data packets), HDR display capabilities (whether the host device supports displaying specific types of HDR videos, such as Colorful HDR), or others.

[0103] In a possible implementation, the source device 31 inquiring about the processing capability of the sink device 32 for the compressed video includes: the source device 31 inquiring about capability description information of the sink device 32 through the auxiliary link.

[0104] When the channel 33 is a signal line, the auxiliary link may be a group of lines in the signal line. When the channel 33 is a network, such as Bluetooth or wireless fidelity (WIFI), the auxiliary link may be a designated frequency band in the network.

[0105] The secondary link is an auxiliary channel included in the channel 33 and is used to transmit additional information, such as capability description information of the sink device 32 or extended display identification data (EDID).

[0106] In one possible example, source device 31 sends a capability query instruction to sink device 32 via the auxiliary link. After receiving the instruction, sink device 32 feeds back its own capabilities to source device 31 via the auxiliary link. The capabilities include the aforementioned decompression capability of compressed video.

[0107] In one possible scenario, there are one or more auxiliary links on the channel 33, and the transmission rate of the auxiliary links is relatively low.

[0108] If the capability description information indicates that the sink device 32 has the ability to decompress compressed video, the source device 31 executes S320; if the capability description information includes that the sink device 32 does not have the ability to decompress compressed video, the source device 31 sends uncompressed video (such as multiple frame images) to the sink device 32.

[0109] In the present application, the source device 31 can directly query the processing capability of the host device 32 for the compressed video, and only execute S320 when the host device has the decompression capability of the compressed video. In this way, the invalid communication process between devices caused by the host device not having the decompression capability of the compressed video and the problem of the transmission resource bandwidth between devices being occupied are avoided.

[0110] S320 : The source device 31 transmits the first VBP, the compression parameter message, and the compressed video to the sink device 32 .

[0111] The compressed video mode field included in the first VBP is a first value, and the first value is used to indicate that the compressed video mode is compressed video. The compression parameter message is used to indicate: compression parameter information of the compressed video.

[0112] The first VBP, compression parameter message and compressed video are data in the video stream transmitted from the source device 31 to the sink device 32 , and specifically may be data in one or more video frames in the video stream.

[0113] Exemplarily, the source device 31 generates a corresponding video stream according to the video data to be transmitted (such as compressed video), and transmits the video stream carrying the compressed video to the sink device 32 .

[0114] The following description is made by taking the first VBP, the compression parameter message, and the compressed video as data in a video frame in the video stream as an example.

[0115] The format of a video frame in a video stream is described below in conjunction with FIG4 , which is a schematic diagram of the structure of a video frame provided by the present application. In FIG4 , a video frame includes: a multimedia message and a timing signal.

[0116] Multimedia packets include one or both of AVPs and audio sample packets (ASPs or ASP packets). AVPs are used to transmit valid video data, valid pixels, or valid pixel data (or valid video pixel data). In this embodiment, AVPs are used to transmit the compressed video, while ASPs are used to transmit audio data. As shown in Figure 4, multiple AVPs are arranged in rows within a video frame. Multiple AVPs belonging to the same row constitute a video row.

[0117] It is worth noting that the structure of the video frame in Figure 4 is only an example provided by this application and should not be understood as a limitation of this application. In some possible implementations, a single video frame may also include only one line of AVP, which is not limited by this application.

[0118] Optionally, when the link bandwidth is greater than the video bandwidth, an ASP can be inserted between two AVPs, but the ASP has a lower priority than the AVP.

[0119] Timing signals include the horizontal synchronization signal (Hsync), the vertical synchronization signal (Vsync), and the display enable (DE) signal. These timing signals describe the video timing, that is, when which messages or signals should be transmitted. Hsync indicates the start of scanning a row of pixels (e.g., when the row register in the sink device 32 receives a signal of 1), Vsync indicates the start of scanning a video frame, and the display enable (DE) signal indicates whether valid data can be received (e.g., DE = 1 indicates that valid data can be received, and DE = 0 indicates that valid data cannot be received).

[0120] In conjunction with Figure 4, the video timing of the video frame is briefly explained: During the blanking period (blank) of the total row pixels (Htotal) or the vertical blanking area (Vblank), the video frame cannot transmit AVP, but can transmit other messages, such as ASP, DIP, etc.; during the active period (Vactive) of the total row pixels (Htotal) or the vertical active area, the video frame can transmit AVP. The starting point of Vactive is marked by the horizontal blanking packet (HBP), which indicates the end of the row blanking area. Exemplarily, the audio and video transmission adapter in the source device 31 sends the HBP for each video line and immediately sends it at the end of Hblank.

[0121] As shown in FIG4 , the compression parameter message is sent in the vertical blanking area. The compression parameter message is one of the multiple types of messages included in the DIP, and the compression parameter message is used to transmit compression parameter information. For the content of the compression parameter information, please refer to the following example and will not be described in detail here.

[0122] In the video frame shown in FIG4 , the vertical blanking area of ​​the video frame includes a VBP, which is used to indicate the video line where the rising edge of Vsync (in terms of positive polarity) of a video frame is located. Exemplarily, the video and audio transmission adapter in the source device 31 sends one or more VBPs in each frame (such as three consecutive VBPs in FIG4 ), and ASP and DIP cannot be inserted between VBPs. For example, if Vsync is positive polarity, the video and audio transmission adapter replaces HBP with VBP on the video line where the rising edge of Vsync is located, and sends two more VBPs immediately thereafter; if Vsync is negative polarity, the video and audio transmission adapter replaces HBP with VBP on the video line where the falling edge of Vsync is located, and sends two more VBPs immediately thereafter. And DIP is transmitted after VBP.

[0123] For the above VBP, the VBP includes a tunnel message header and a payload portion. For example, the tunnel message header occupies 4 bytes, the payload portion occupies 28 bytes, and a single VBP occupies a total of 32 bytes.

[0124] In one possible scenario, the VBP includes a field indicating whether the videos carried by the multiple AVPs are compressed videos or uncompressed videos.

[0125] The above fields are carried in the payload of the VBP. The following is an exemplary description of the tunnel message header and payload of the VBP in conjunction with Table 1 and Table 2, respectively.

[0126] Table 1: VBP tunnel header

[0127] Among them, ShuttleID is used to identify a data transmission channel on the link (Link), and the value is 0 to 127. For audio and video adapters (audio and video transmitting adapters or audio and video receiving adapters), this field is the adapter ID set at the factory.

[0128] Table 2: Load section of VBP

[0129] The Compressed Video Mode (CompressVideoMode) field in the VBP is used to indicate whether the video carried by the AVP is compressed video. If the CompressVideoMode field in the VBP is the first value (such as 1b above), it means that the data carried by the AVP in the video frame is compressed video. If the CompressVideoMode field in the VBP is the third value (such as 0b above), it means that the data carried by the AVP in the video frame is uncompressed video.

[0130] Exemplarily, the data carried by the AVP is valid pixel data in the compressed video.

[0131] It is understandable that when the data carried by the AVP in the video stream is not compressed video, it can also be uncompressed video, or compressed video that complies with other standards except the PLC standard.

[0132] In this embodiment, the source device 31 is provided with an identifier indicating the format of the AVP in the VBP, and packages the AVP according to the format indicated by the identifier (i.e., when the video format is different, the corresponding AVP packages are different), so that the host device can parse these AVPs based on the format indicated by the identifier and output the video, thereby avoiding the problem of reduced picture optimization efficiency caused by the host device consuming more computing power.

[0133] The above compression parameter message will be described in conjunction with Figure 4. The tunnel message header and payload of DIP will be described in conjunction with Tables 3 and 4, respectively.

[0134] Table 3: DIP tunnel header

[0135] Table 4: DIP load section

[0136] In this embodiment, when the second value corresponding to the descriptive information type field in the DIP is 0x02, the DIP can be called a compression parameter message. As for the compression parameter information carried in the payload part of the compression parameter message, a possible example is given below and will not be repeated here.

[0137] In one possible scenario, when the data length of all compression parameters corresponding to a video frame is less than or equal to 32 bytes, there is only one compression parameter message in the video frame, and S=1 and E=1 in the Flags field of the compression parameter message, that is, the message start flag of the compression parameter message is 1 and the message end flag is 1.

[0138] In another possible scenario, when the data length of all compression parameters corresponding to a video frame is greater than 32 bytes, the video frame will include multiple compression parameter messages.

[0139] For example, when the data length of all compression parameters corresponding to a video frame is 64 bytes, the video frame will include two compression parameter messages. In the Flags field of the first compression parameter message, S=1 and E=0, meaning the message start flag of the first compression parameter message is 1 and the message end flag is 0. In the Flags field of the second compression parameter message, S=0 and E=1, meaning the message start flag of the second compression parameter message is 0 and the message end flag is 1.

[0140] In the present application, since the compression parameter message includes a message start flag and a message end flag, the message start flag and the message end flag can enable the host device 32 to clearly understand the start or end of the compression parameter message sent to it by the source device 31, which is beneficial to improving the host device 32's parsing of the compression parameter message, thereby improving the efficiency of the host device 32 in decoding the compressed video according to the compression message parameters carried by the compression parameter message.

[0141] In one possible scenario, when a video frame includes two compression parameter messages, the INDEX field in the first compression parameter message is 1, and the INDEX field in the second compression parameter message is 2. In other examples of the present application, if the video frame includes more compression parameter messages, the value corresponding to the INDEX field will increase sequentially, and the sink end 32 can then detect whether any message is lost in the middle by detecting the INDEX.

[0142] Continuing to refer to FIG. 3 , the video transmission method provided in this embodiment further includes the following step S330 .

[0143] S330 : The sink device 32 receives the first VBP, the compression parameter message, and the compressed video transmitted by the source device 31 .

[0144] In a possible example, the sink device 32 receives the video stream transmitted by the source device 31 through the channel 33. For the content of each video frame in the video stream, reference may be made to the description in S320 above, which will not be repeated here.

[0145] In one possible embodiment, the sink device 32 may decode and display the received video stream.

[0146] Since the VBP in each video frame carries a compressed video mode field, the sink device 32 enables the frame-level compression mode according to the compressed video mode field carried by the VBP in each video frame, that is, uses corresponding means to parse the compressed video.

[0147] For example, since the compressed videos in this embodiment all comply with the PLC standard, the sink device 32 may parse the compressed videos using a decoding method that complies with the PLC standard.

[0148] Compared with the DP protocol or HDMI protocol, which only transmits compressed videos obtained by specific compression methods (such as DSC), the present application can transmit compressed videos that comply with the PLC standard. The above-mentioned specific compression method does not include PLC. Therefore, it avoids the problem of being affected when transmitting compressed videos that comply with the PLC standard, and improves the robustness of data transmission and the efficiency of transmitting compressed videos that comply with the PLC standard.

[0149] In a possible example, the compression parameter information may include the following content.

[0150] The number of slices, chunk size, prediction algorithm parameters used in the encoding process, transmission delay CU, bit rate control buffer size (RC buffer size), etc.

[0151] Slices are the unit of division of a video frame during encoding. A video frame can be divided into multiple slices, each of which can be independently encoded and decoded.

[0152] Chunking refers to the technology of dividing the data stream into smaller blocks for transmission. Chunk size refers to the size of each chunk.

[0153] Transmission delay cu refers to the transmission delay between the reference pixel block and the current pixel block during the intra-frame prediction stage of the video encoding process.

[0154] rc buffer size is used to control the video bit rate and quality. rc buffer size specifies the buffer size used in the encoder to store the target bit rate, thereby controlling the output speed and quality of the bitstream.

[0155] In one possible embodiment, Figure 5 shows a data arrangement in an AVP. As shown in Figure 5, Figure 5 is a schematic diagram of the AVP carrier provided by this application. The first line in Figure 5 is the AVP message header, and the meaning of each field can be seen in Table 5 below.

[0156] Table 5: AVP header

[0157] The arrangement of data in the AVP shown in Figure 5 is the arrangement when the data belongs to compressed video. In this case, the payload part of the AVP carries multiple groups of data in the compressed video with an 8-bit granularity, that is, one AVP carries multiple groups of 8-bit data.

[0158] In a possible example, one AVP carries a maximum of 508 groups of data.

[0159] In one possible scenario, when the compressed video cannot fill a complete line (4 bytes), the source device 31 will fill with 0s to perform structure shaping to ensure that the payload part in the AVP is 4-byte aligned.

[0160] For example, when the amount of valid video pixel data that a video frame can carry is limited and the valid video pixel data to be transmitted is large, a single video frame will be filled with as much valid video pixel data as possible. However, the amount of pixel data is inconsistent with the maximum amount of pixel data that a video frame can carry, which may result in bits of unfilled pixel portions in the video frame. To address the above issue, if the compressed video cannot completely fill the last AVP in the Nth row of AVPs, the unfilled pixel portions in the last 32 bits of the last AVP are 0, and the Nth row is one of one or more rows.

[0161] It is worth noting that the "last AVP" here refers to the last AVP in each row, not the last AVP in the video frame. Furthermore, the last 32 bits of the last AVP do not set the unfilled pixels of the AVP to 0 if each AVP in the video frame cannot be completely filled. Instead, the unfilled pixels in the last 32 bits of the last AVP are set to 0 only if the last AVP in a row of AVPs in the video frame cannot be completely filled.

[0162] For example, when DE is 1, the video and audio transmission adapter obtains pixel data from the pixel data interface on the rising edge of the pixel clock, arranges the pixel data in sequence to generate AVPs, and then sends the generated AVPs in sequence. Except for the AVP packet at the end of the row, which needs to be padded with padding data (such as 0 or other specified symbols) for padding, the pixels in a row are arranged without padding data.

[0163] In a possible embodiment, as shown in Figure 6, which is a second flow chart of a video transmission method provided by this application, Figure 6 is shown based on Figure 3. After the source device 31 completes transmitting the compressed video, the video transmission method further includes steps S340 and S350.

[0164] S340 : The source device 31 transmits the second VBP and the uncompressed video to the sink device 32 .

[0165] The compressed video mode field included in the second VBP is a third value, and the third value is used to indicate that the compressed video mode is non-compressed video.

[0166] In a possible example, after the source device 31 completes transmission of the video stream carrying the compressed video, it transmits a video stream carrying the uncompressed video to the sink device 32 .

[0167] The video stream carrying the uncompressed video includes one or more video frames, and the video frame includes: one or more second VBPs and multiple AVPs carrying the uncompressed video.

[0168] For the content of the second VBP and AVP, reference may be made to the description of VBP and AVP in S320 above, which will not be repeated here.

[0169] In one possible example, the value of the CompressVideoMode field carried in VBP B is "0b." When the value of the CompressVideoMode field is 0b, the data carried by the multiple AVPs included in the video frame is uncompressed video. Based on the value of the CompressVideoMode field being 0b, the sink device 32 disables compression mode, i.e., parses the uncompressed video in a conventional manner.

[0170] For example, the above conventional method can parse the format information of the uncompressed video for the host device 32, such as resolution and color space format (such as luminance-chrominance YUV). If the format of the uncompressed video does not match the requirements of the device, pixel format conversion is required, and then the format-converted video is sent to the rendering engine for processing and playback.

[0171] Since the VBP in each video frame carries a compressed video mode field, the host device 32 can enable / disable the compression mode at the frame level according to the compressed video mode field carried by the VBP in each video frame, that is, use corresponding means to parse the compressed video / uncompressed video.

[0172] When uncompressed video is carried in AVP, the payload of AVP can carry multiple groups of data in the uncompressed video at a granularity of 8 bit / 10 bit / 12 bit / 16 bit.

[0173] S350: The sink device 32 receives the second VBP and the uncompressed video.

[0174] Exemplarily, the sink device 32 receives the video stream carrying the uncompressed video.

[0175] In a possible embodiment, if the sink device 32 does not have the ability to decompress the compressed video, the above-mentioned contents of S340 and S350 may be directly executed. For details, please refer to the contents shown in FIG. 6 , which will not be described in detail here.

[0176] In other embodiments of the present application, the above method can also be performed by the video and audio transmission adapter 213 and the video and audio reception adapter 223 in Figure 2. For example, the video and audio transmission adapter 213 can correspond to the above-mentioned source device 210 and source device 31; the video and audio reception adapter can correspond to the above-mentioned sink device 220 and sink device 32.

[0177] It is understood that in order to implement the functions in the above embodiments, the source device and the sink device include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily appreciate that, in combination with the units and method steps of the various examples described in the embodiments disclosed in this application, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a manner driven by computer software depends on the specific application scenario and design constraints of the technical solution.

[0178] The video transmission method provided according to the present embodiment is described in detail above with reference to FIG. 1 to FIG. 6 . The video device provided according to the present embodiment will be described below with reference to FIG. 7 .

[0179] FIG7 is a schematic structural diagram of a video device provided by the present application. The video device 700 can be used to implement the functions of any one of the devices in the above-mentioned method embodiments, and thus can also achieve the beneficial effects possessed by the above-mentioned method embodiments. In this embodiment, the video device 700 can be the set-top box 110, smart TV 120, or any host device as shown in FIG1 , or the source device 210 or host device 220 as shown in FIG2 , or the source device or host device provided in subsequent embodiments. It should be understood that the video device 700 can also be a module (such as a chip) applied to any of the aforementioned devices.

[0180] As shown in Figure 7, the video device 700 includes a transceiver module 710 and a processing module 720. The transceiver module 710 and the processing module 720 can collaborate to implement the various steps in the aforementioned method embodiment. A more detailed description of the transceiver module 710 and the processing module 720 can be directly obtained by referring to the relevant description of the device in the method embodiment shown in the aforementioned figures, and is not repeated here.

[0181] When a video device implements any of the video transmission methods shown in the aforementioned figures through software, the video device and its various units may also be software modules. The aforementioned video transmission method is implemented by a processor calling the software module. The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0182] It can be understood that the video device shown in FIG. 7 is only an example provided in this embodiment. The video device may include more or fewer units according to different video transmission processes, and this application does not limit this.

[0183] When the video device is implemented via hardware, the hardware may be implemented via a processor or a chip system. The chip system includes one or more chips, each of which includes a processor and a power supply circuit. The power supply circuit is used to power the processor, and the processor is used to implement any possible implementation method of the above embodiments through logic circuits or executing code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments and will not be repeated here.

[0184] It is understood that the processor in the embodiments of the present application may be a CPU, or other general-purpose processor, digital signal processor (DSP), ASIC, FPGA or other programmable logic device, transistor logic device, hardware component or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0185] In addition, the video device 700 shown in Figure 7 can also be processed by a video processing device, as shown in Figure 8. Figure 8 is a structural diagram of the video processing device provided in this application. The video processing device 800 includes: a memory 810 and at least one processor 820. The processor 820 can implement the video transmission method provided in the above embodiment, and the memory 810 is used to store software instructions corresponding to the above video transmission method.

[0186] As an optional implementation, in hardware implementation, the video processing device 800 may refer to a chip or chip system encapsulated with one or more processors 820. For example, when the video processing device 800 is used to implement the method steps in the above embodiment, the processor 820 included in the video processing device 800 executes the steps of the source device 31 in the above method and its possible sub-steps. In an optional scenario, the video processing device 800 may also include a communication interface 830, which can be used to send and receive data. For example, the communication interface 830 is used to receive audio data, or send audio streams, etc.; the communication interface 830 can be implemented by the interface circuit included in the video processing device 800. Therefore, in some examples, the communication interface 830 can also be called a transceiver of the video processing device. In this embodiment, the communication interface 830 supports wired connection using a unified multimedia interconnection interface.

[0187] In an embodiment of the present application, the communication interface 830, the processor 820, and the memory 810 may be connected via a bus 840, which may be divided into an address bus, a data bus, a control bus, etc. The bus 840 may be a PCIe bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), or other types of buses.

[0188] It is worth noting that the video processing device 800 can also perform the functions of the video apparatus 700 shown in FIG. 7 , which will not be described in detail here.

[0189] The video processing device 800 provided in this embodiment can be the above-mentioned set-top box 110, smart TV 120, source device 210 or source device 31, or other devices with video processing functions, and this application is not limited to this. For example, when the aforementioned sink device 32 also has video processing functions, the video processing device 800 can refer to any of the aforementioned sink devices 32.

[0190] In addition, the video device 700 shown in FIG7 can also be implemented by a sink device. When the video device 700 is implemented by a sink device, this embodiment provides a possible example, as shown in FIG9 , which is a schematic structural diagram of the sink device provided by this application. The sink device 32 includes: a processor 910, an external memory interface 920, an internal memory 921, a USB interface 930, a unified multimedia interconnection interface 931, an antenna 1, an antenna 2, a mobile communication module 950, a wireless communication module 960, an audio module 970, a speaker 970A, a receiver 970B, a microphone 970C, a sensor module 980, a button 990, an indicator 992, a camera 993, and a display screen 994.

[0191] Among them, the above-mentioned sensor module 980 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0192] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the sink device. In other embodiments, the sink device may include more or fewer components than illustrated, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0193] The processor 910 may include one or more processing units. For example, the processor 910 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0194] The controller can be the nerve center and command center of the sink device. The controller can generate operation control signals based on instruction opcodes and timing signals to complete the control of instruction fetching and execution.

[0195] Processor 910 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 910 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 910. If processor 910 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 910 latency, and thus improves system efficiency.

[0196] In some embodiments, the processor 910 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, a unified multimedia interconnect interface, etc.

[0197] It is understood that the interface connection relationship between the modules illustrated in this embodiment is only for illustrative purposes and does not constitute a structural limitation on the sink device. In other embodiments, the sink device may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0198] The wireless communication function of the sink device may be implemented using antenna 1, antenna 2, mobile communication module 950, wireless communication module 960, a modem processor, and a baseband processor. In some embodiments, antenna 1 of the sink device is coupled to mobile communication module 950, and antenna 2 is coupled to wireless communication module 960, enabling the sink device to communicate with a network and other devices using wireless communication technologies.

[0199] The wired communication function of the sink device can be implemented through the USB interface 930 or the unified multimedia interconnection interface 931. For example, the sink device receives or sends a video stream through a channel connected to the unified multimedia interconnection interface 931.

[0200] The sink device implements display functionality through a GPU, display screen 994, and an application processor. A GPU is a microprocessor for image processing that connects display screen 994 and the application processor. The GPU performs mathematical and geometric calculations for graphics rendering. Processor 910 may include one or more GPUs that execute program instructions to generate or modify display information.

[0201] The display screen 994 is used to display images, videos, etc. The display screen 994 includes a display panel.

[0202] The sink device can implement a camera function using an ISP, a camera 993, a video codec, a GPU, a display 994, and an application processor. The ISP processes data fed back by the camera 993. The camera 993 is used to capture still images or video. In some embodiments, the sink device may include one or N cameras 993, where N is a positive integer greater than 1.

[0203] In this embodiment, the above display screen 994, video codec, GPU, display screen 994 and application processor can also be collectively referred to as a display unit of the host device 32, which is used to process and display the received video stream.

[0204] External memory interface 920 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the host device. The external memory card communicates with processor 910 via external memory interface 920 to implement data storage. For example, files such as music and videos can be stored on the external memory card.

[0205] The internal memory 921 can be used to store computer executable program code, which includes instructions. The processor 910 executes various functional applications and data processing of the host device by running the instructions stored in the internal memory 921. For example, in an embodiment of the present application, the processor 910 can execute instructions stored in the internal memory 921, and the internal memory 921 can include a program storage area and a data storage area.

[0206] The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the host device (such as audio data, a phone book, etc.). In addition, the internal memory 921 can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, a flash memory device, or a universal flash storage (UFS).

[0207] The sink device can implement audio functions such as music playback and recording through the audio module 970, speaker 970A, receiver 970B, microphone 970C, and application processor.

[0208] Buttons 990 include a power button, a volume button, and the like. Buttons 990 may be mechanical buttons or touch buttons. Indicator 992 may be an indicator light that can be used to indicate charging status, battery level changes, messages, missed calls, notifications, and the like.

[0209] An embodiment of the present application also provides a chip system, which includes a control circuit and an interface circuit. The chip includes a control circuit and an interface circuit. The control circuit is used to query the host device's processing capability for compressed video. The control circuit and the interface circuit work together to implement the above-mentioned function of the source device, and / or the interface circuit is used to receive a first vertical blanking message VBP, a compression parameter message, and compressed video transmitted by the source device. The compressed video mode field included in the first VBP is a first value, and the first value is used to indicate that the compressed video mode is compressed video. The compression parameter message is used to indicate: compression parameter information of the compressed video. The control circuit and the interface circuit work together to implement the above-mentioned function of the host device.

[0210] The method steps in the embodiments of the present application can also be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and storage medium can also exist as discrete components in the video processing device and the host device.

[0211] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0212] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A video transmission method, characterized in that, the method is executed by a source device or a chip in the source device, and the method includes: querying the processing ability of a destination device for compressed video; when the destination device has the ability to decompress the compressed video, transmitting a first vertical blanking packet (VBP), a compression parameter packet, and the compressed video to the destination device, where the compression video mode field included in the first VBP is a first value, and the first value is used to indicate that the compression video mode is compressed video, and the compression parameter packet is used to indicate: the compression parameter information of the compressed video.

2. The method according to claim 1, characterized in that, the transmitting the compressed video to the destination device includes: transmitting an active video packet (AVP) to the destination device, and the AVP is used to carry the compressed video.

3. The method according to claim 2, characterized in that, one AVP carries multiple groups of 8-bit data.

4. The method according to any one of claims 1 to 3, characterized in that, the compressed video complies with the perceptual lossless compression (PLC) standard.

5. The method according to any one of claims 1 to 4, characterized in that, the descriptive information type byte included in the compression parameter packet is a second value.

6. The method according to any one of claims 1 to 5, characterized in that, when transmitting a compression parameter packet to the destination device, the packet start flag included in the compression parameter packet is 1, and the packet end flag included in the compression parameter packet is 1.

7. The method according to any one of claims 1 to 5, characterized in that, when transmitting multiple compression parameter packets to the destination device, the packet start flag included in the first compression parameter packet among the multiple compression parameter packets is 1, and the packet end flag included in the last compression parameter packet among the multiple compression parameter packets is 1.

8. The method according to any one of claims 1 to 7, characterized in that, the source device and the destination device are connected through an audio-video interface network.

9. The method according to any one of claims 1 to 8, characterized in that, when the transmission of the compressed video is completed, the method further includes: transmitting a second vertical blanking packet (VBP) and uncompressed video to the destination device, where the compression video mode field included in the second VBP is a third value, and the third value is used to indicate that the compression video mode is uncompressed video.

10. A video transmission method, characterized in that, the method is executed by a destination device or a chip in the destination device, the destination device provides the processing ability for compressed video that supports query by the source device, and when the destination device has the ability to decompress the compressed video, the method includes: receiving the first vertical blanking packet (VBP), the compression parameter packet, and the compressed video transmitted by the source device, where the compression video mode field included in the first VBP is a first value, and the first value is used to indicate that the compression video mode is compressed video, and the compression parameter packet is used to indicate: the compression parameter information of the compressed video.

11. The method according to claim 10, characterized in that, Receiving the compressed video transmitted by the source device includes: Receiving an Active Video Packet (AVP) transmitted by the source device, where the AVP is used to carry the compressed video.

12. The method according to claim 11, wherein, One AVP carries multiple groups of 8-bit data.

13. The method according to any one of claims 10 to 12, wherein, The compressed video complies with the Perceptual Lossless Compression (PLC) standard.

14. The method according to any one of claims 10 to 13, wherein, The type byte of the descriptive information included in the compression parameter packet is a second value.

15. The method according to any one of claims 10 to 14, wherein, If a compression parameter packet transmitted by the source device is received, the packet start flag included in the compression parameter packet is 1, and the packet end flag included in the compression parameter packet is 1.

16. The method according to any one of claims 10 to 14, wherein, If multiple compression parameter packets transmitted by the source device are received, the packet start flag included in the first compression parameter packet among the multiple compression parameter packets is 1, and the packet end flag included in the last compression parameter packet among the multiple compression parameter packets is 1.

17. The method according to any one of claims 10 to 16, wherein, The source device and the sink device are connected through an audio-video interface network.

18. The method according to any one of claims 10 to 17, wherein, The method further includes: Receiving a second Vertical Blanking Packet (VBP) and uncompressed video transmitted by the source device, where the compressed video mode field included in the second VBP is a third value, and the third value is used to indicate that the compressed video mode is uncompressed video.

19. A chip, wherein, it includes: A control circuit and an interface circuit; The control circuit is used to query the processing ability of the sink device for the compressed video, and the control circuit and the interface circuit cooperate to execute the method according to any one of claims 1 to 9; and / or, The interface circuit is used to receive a first Vertical Blanking Packet (VBP), a compression parameter packet, and compressed video transmitted by the source device. The compressed video mode field included in the first VBP is a first value, and the first value is used to indicate that the compressed video mode is compressed video. The compression parameter packet is used to indicate the compression parameter information of the compressed video. The control circuit and the interface circuit cooperate to execute the method according to any one of claims 10 to 18.

20. A video processing device, wherein, it includes: A memory and a transceiver; The memory is used to store the compressed video; The transceiver is used to transmit a first Vertical Blanking Packet (VBP), a compression parameter packet, and compressed video. The compressed video mode field included in the first VBP is a first value, and the first value is used to indicate that the compressed video mode is compressed video. The compression parameter packet is used to indicate the compression parameter information of the compressed video; The memory and the transceiver are used to cooperate to execute the method according to any one of claims 1 - 9. ​ 21. A display device, characterized in that, it includes a transceiver and a display unit; the transceiver is configured to receive a first vertical blanking packet (VBP), a compression parameter packet, and compressed video, the compression video mode field included in the first VBP is a first value, and the first value is used to indicate that the compression video mode is compressed video, and the compression parameter packet is used to indicate: the compression parameter information of the compressed video; the display unit is configured to process and display the first vertical blanking packet (VBP), the compression parameter packet, and the compressed video; the transceiver and the display unit are configured to cooperate to execute the method according to any one of claims 10-18.

22. A video transmission system, characterized in that, it includes a video processing device and a display device, and the video processing device and the display device are connected through an audio-visual interface network, wherein, the video processing device is configured to query the processing capability of the display device for compressed video, and when the display device has the decompression capability for the compressed video, transmit a first vertical blanking packet (VBP), a compression parameter packet, and compressed video to the display device, the compression video mode field included in the first VBP is a first value, and the first value is used to indicate that the compression video mode is compressed video, and the compression parameter packet is used to indicate: the compression parameter information of the compressed video; the display device is configured to receive a first vertical blanking packet (VBP), a compression parameter packet, and compressed video, the compression video mode field included in the first VBP is a first value, and the first value is used to indicate that the compression video mode is compressed video, and the compression parameter packet is used to indicate: the compression parameter information of the compressed video.

23. A computer-readable storage medium, characterized in that, the storage medium stores a computer program or instruction, and when the computer program or instruction is executed by a processing device, it implements the method according to any one of claims 1 to 9, or implements the method according to any one of claims 10-18.

24. A computer program product, including a computer program or instruction, characterized in that, when the computer program or instruction is executed by a processing device, it implements the method according to any one of claims 1 to 9, or implements the method according to any one of claims 10-18.

Images