Video transmission method and device

Abstract

The present invention relates to the technical field of media, and provides a video transmission method and a device, which can dynamically enable a fast video transmission mode for video data and reduce the transmission delay of video data. The method is applied to or a source device or a chip in a source device. The method comprises: querying the capability of a host device; and when the host device has a fast video transmission capability, transmitting a first vertical blanking packet (VBP) and a first video data to the host device, wherein the fast video transmission field comprised in the first VBP has a first value, and the first value is used for instructing to enable a fast video transmission mode for the first video data.

Description

Video transmission method and device Technical Field

[0001] The present application relates to the field of media technology, and in particular to a video transmission method and device. Background Art

[0002] With the development of big data, artificial intelligence (AI), and cloud computing, various types of multimedia data are constantly emerging, and the requirements for the transmission performance of multimedia data between different devices are also increasing. Taking video data as an example, in some scenarios, video data must be transmitted quickly between the transmitter and the display. Therefore, how to achieve this fast transmission presents many challenges.

[0003] Summary of the Invention

[0004] The present application provides a video transmission method and device, which can dynamically enable a fast video transmission mode for video data and reduce the transmission delay of video data.

[0005] This application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a video transmission method, which is applied to a source device or a chip in a source device, the method comprising: querying the capabilities of a host device; when the host device has fast video transmission capabilities, transmitting a first vertical blanking message VBP and first video data to the host device; wherein the fast video transmission field included in the first VBP is a first value, and the first value is used to indicate that a fast video transmission mode is enabled for the first video data.

[0007] In this application, when the source device queries that the host device has the ability to transmit fast video, the source device can carry a field in the VBP that instructs the host device to enable the fast video transmission mode, and send the VBP and video data to the host device, so that the host device enables the fast video transmission mode according to the instruction of the fast video transmission field. This method can dynamically enable the fast video transmission mode for the video data and reduce the transmission delay of the video data.

[0008] Furthermore, when the host device supports fast video transmission capability and needs to enable fast video transmission mode, the method can more conveniently and dynamically control the enabling of fast video transmission mode by carrying a field indicating enabling of fast video transmission mode in the VBP, without the need for manual enabling.

[0009] In one possible implementation, the video transmission method provided by the present application further includes: sending a fast video transmission magnification to a sink device to instruct the sink device to set a video processing mode of the sink device according to the fast video transmission magnification. The video transmission magnification is less than or equal to a maximum video transmission magnification supported by the sink device.

[0010] In one possible implementation, the video transmission method provided in the present application also includes: transmitting a second vertical blanking message VBP and second video data to the host device; wherein the fast video transmission field included in the second VBP is a second value, and the second value is used to indicate that the fast video transmission mode is turned off for the second video data.

[0011] In one possible implementation, the above-mentioned transmission of the first VBP and the first video data to the host device specifically includes: transmitting a first video frame to the host device, the first video frame including at least one first VBP and multiple first valid video packets AVP, and the multiple first AVPs are used to carry the first video data.

[0012] In one possible implementation, the above-mentioned transmission of the second VBP and the second video data to the host device specifically includes: transmitting a second video frame to the host device, the second video frame including at least one second VBP and multiple second valid video packets AVP, and the multiple second AVPs are used to carry the second video data.

[0013] In this application, VBP is sent in each video frame, and VBP includes a field for fast video transmission mode. In this way, frame-level control of video data can be achieved to enable fast video transmission mode for a certain video frame, which is highly flexible.

[0014] In a second aspect, the present application provides a video transmission method, which is applied to a host device or a chip in the host device, and the source device supports fast video transmission capability. The method includes: receiving a first vertical blanking message VBP and first video data sent by the source device; when the fast video transmission field included in the first VBP is a first value, starting a fast video transmission mode for the first video data.

[0015] In the present application, after the host device receives the first VBP and the first video data sent by the source device, it turns on the automatic low-latency mode according to the fast video transmission indication in the first VBP sent by the source device, and quickly receives the first video data in the fast video transmission mode, which can reduce the transmission delay of the video data.

[0016] Furthermore, when the host device supports fast video transmission capability and needs to enable fast video transmission mode, the method can more conveniently and dynamically control the enabling of fast video transmission mode by carrying a field indicating enabling of fast video transmission mode in the VBP, without the need for manual enabling.

[0017] In a possible implementation, the video transmission method provided in the present application further includes: receiving a fast video transmission rate sent by a source device; and setting a video processing mode of the sink device according to the fast video transmission rate.

[0018] In the present application, the sink device setting the video processing mode of the sink device according to the fast video transmission ratio (i.e., setting the sink device capability) may include: the sink device adjusting the transmission bandwidth between the various modules in the sink device based on the fast video transmission ratio, so that after the sink device receives the video data, it can quickly transmit the video data to the processing module for processing. Alternatively, the sink device setting the video processing mode of the sink device may also include the sink device setting one or more processing modules for video quality enhancement from a closed state to an open state, so that after the sink device quickly receives the video data, the sink device performs quality enhancement processing on the video data, thereby improving the video quality.

[0019] In one possible implementation, the video transmission method provided in the present application also includes: receiving a second vertical blanking message VBP and second video data sent by a source device; when the fast video transmission field included in the second VBP is a second value, turning off the fast video transmission mode for the second video data.

[0020] In one possible implementation, the above-mentioned receiving the first vertical blanking message VBP and the first video data sent by the source device specifically includes: receiving the first video frame sent by the source device, the first video frame includes at least one first VBP and multiple first valid video messages AVP, and the multiple first AVPs are used to carry the first video data.

[0021] In one possible implementation, the second vertical blanking message VBP and the second video data sent by the receiving source device specifically include: receiving the second video frame sent by the source device, the second video frame includes at least one second VBP and multiple second valid video messages AVP, and the multiple second AVPs are used to carry the second video data.

[0022] In conjunction with the first and second aspects above, in one possible implementation, the transmission bandwidth of the first video data is a second bandwidth, the second bandwidth being the bandwidth of the source device after adjusting the first bandwidth of the currently transmitted video data, and the second bandwidth being greater than the first bandwidth. The increased transmission bandwidth of the first video data allows for faster transmission of the first video data from the source device to the sink device, thereby reducing video transmission latency.

[0023] In combination with the above-mentioned first aspect and second aspect, in a possible implementation manner, the second bandwidth is N+1 times the first bandwidth; wherein N represents a fast video transmission rate, and N>0.

[0024] In combination with the above-mentioned first and second aspects, in one possible implementation, when the fast video transmission mode is turned on, the length of the effective video area of ​​the first video frame is reduced, so that the first video frame can be quickly sent to the host device, and the host device will receive the first video frame faster.

[0025] In combination with the above-mentioned first and second aspects, in a possible implementation, the transmission bandwidth of the second video data is a third bandwidth, the third bandwidth is the bandwidth after the source device adjusts the second bandwidth of the currently transmitted video data, and the third bandwidth is smaller than the second bandwidth.

[0026] Optionally, the source device can reduce the video transmission bandwidth to the bandwidth before the fast video transmission mode is turned on, such as the first bandwidth mentioned above, then the first bandwidth is equal to the third bandwidth. The video transmission bandwidth can also be reduced to other values, which are determined according to actual needs and are not limited in this application.

[0027] In a third aspect, the present application provides a video processing device, which includes a module for executing the method of any one of the implementations of the first aspect or the second aspect.

[0028] In a fourth aspect, the present application provides a source device comprising: a memory, a transceiver, and a processor; the memory, the transceiver, and the processor are configured to collaboratively execute the method described in the first aspect and any one of its possible implementations.

[0029] In a fifth aspect, the present application provides a host device, comprising: a transceiver and a processor; the transceiver and the processor are configured to collaboratively execute the method described in the second aspect and any one of its possible implementations.

[0030] In a sixth aspect, the present application provides a video transmission system. The video transmission system includes the source device provided in the fourth aspect and the sink device provided in the fifth aspect. The source device can be used to implement the functions of the source device in the first or second aspect, and the sink device can be used to implement the functions of the sink device in the first or second aspect. Therefore, the video transmission system can also achieve the beneficial effects of the methods described in the first and second aspects, which will not be elaborated here.

[0031] In a seventh aspect, the present application provides a computer-readable storage medium storing computer instructions, which, when executed on a computing device, execute the method of any one of the first aspect and its possible implementations, or the second aspect and its possible implementations. For example, the computing device is the aforementioned source device or sink device.

[0032] In an eighth aspect, the present application provides a computer program product comprising computer instructions that, when executed on a computing device, execute the method of the first aspect and any one of its possible implementations. For example, the computing device is the aforementioned source device or sink device.

[0033] In a ninth aspect, the present application provides a chip system, comprising: a processor configured to call and execute a computer program from a memory, so that a computing device equipped with the chip system executes the method described in any one of the first aspect and its possible implementations, or the second aspect and its possible implementations. The computing device may be the aforementioned source device or sink device.

[0034] It should be understood that the beneficial effects achieved by the technical solutions of the second to ninth aspects of this application and the corresponding possible implementation methods can be referred to the technical effects of the first aspect and its corresponding possible implementation methods mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG1 is a schematic diagram of a video transmission system provided in an embodiment of the present application;

[0036] FIG2 is a schematic diagram of an audio and video encoding and decoding system provided in an embodiment of the present application;

[0037] FIG3 is a block diagram of an audio and video transmission adapter provided in an embodiment of the present application;

[0038] FIG4 is a block diagram of an audio / video receiving adapter provided in an embodiment of the present application;

[0039] FIG5 is a flow chart of a video transmission method according to an embodiment of the present application;

[0040] FIG6 is a schematic diagram of the structure of a vertical blanking message VBP provided in an embodiment of the present application;

[0041] FIG7 is a schematic diagram of the structure of a video frame provided in an embodiment of the present application;

[0042] FIG8 is a schematic diagram of the structure of an effective video packet AVP provided in an embodiment of the present application;

[0043] FIG9 is a schematic diagram of a timing sequence of a video frame according to an embodiment of the present application;

[0044] FIG10 is a second timing diagram of a video frame provided in an embodiment of the present application;

[0045] FIG11 is a second flow chart of a video transmission method provided in an embodiment of the present application;

[0046] FIG12 is a third flow chart of a video transmission method provided in an embodiment of the present application;

[0047] FIG13 is a schematic structural diagram of a video processing device provided by the present application;

[0048] FIG14 is a schematic diagram of the structure of a video processing device provided by this application;

[0049] FIG15 is a schematic structural diagram of the display device provided in this application. DETAILED DESCRIPTION

[0050] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0051] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects rather than to describe a specific order of the objects. For example, "first video data" and "second video data" are used to distinguish different video data rather than to describe a specific order of the video data.

[0052] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0053] In the description of the embodiments of the present application, unless otherwise specified, “plurality” means two or more.

[0054] It is understandable that different scenarios have different requirements for the transmission delay of video data. For example, in a gaming scenario, the transmission delay of video data is required to be as low as possible, so that the user's gaming experience can be higher. For other scenarios, there are no strict requirements for the transmission delay of video data. To meet the needs of different scenarios, an embodiment of the present application provides a video transmission method for transmitting video between a source device and a sink device. The method supports a quick video transport (QVT) mode. When the source device queries that the sink device has a quick video transmission capability, the source device can control the sink device to turn on or off the quick video transmission mode. When the main link bandwidth between the source device and the sink device is sufficient, the source device can carry a field in the vertical blanking packet indicating that the sink device has turned on the quick video transmission mode, and send the vertical blanking packet (VBP) and the video data to the sink device, so that the video data is quickly transmitted to the sink device to reduce the transmission delay of the video data.

[0055] The technical solutions involved in the embodiments of this application can be applied not only to current audio and video transmission technologies or audio and video standards, but may also be applied to future audio and video transmission technologies or audio and video standards. The terms used in the implementation methods of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0056] Video streaming refers to the transmission of video data. For example, a video stream can be processed as a stable and continuous flow across a network. Video (or video data) captured by a video capture device consists of multiple frames. This video data is packaged and encapsulated to create a video stream, which consists of multiple frames, each corresponding to a single image.

[0057] In this embodiment, video is a general term that includes a sequence of multiple consecutive frames, with one frame corresponding to one image. Video and audio are information application technology terms that refer to video, audio, or multimedia content including video and audio.

[0058] Fast video transmission: refers to the rapid transmission of video data from the source device to the sink device. The essence of fast video transmission is to reduce the transmission time of the valid video data of the video frame (valid video data refers to the valid pixel data in the video frame), so that the sink device can receive the video data more quickly.

[0059] The video formats supported by the video transmission method provided in the embodiments of the present application include but are not limited to RGB, YCbCr4:4:4, YCbCr4:2:2, YCbCr4:2:0, ARGB, Y-only, RAW, etc.

[0060] The video transmission method provided in the embodiments of the present application supports video data with component bit widths (or color depths) including, but not limited to, one or more of the following: 8-bit, 10-bit, 12-bit, or 16-bit. For example, for video data in RGB format, if the component bit width is 8-bit, the R component of a pixel occupies 8 bits, the G component occupies 8 bits, and the B component occupies 8 bits. This can also be described as supporting video data with 8 bits per component (bpc), 10 bits per component, 12 bits per component, or 16 bits per component.

[0061] In order to make the description of the following embodiments clear and concise, an introduction to related technologies is first given.

[0062] Figure 1 is a schematic diagram of a video transmission system provided by an embodiment of the present application. The video processing process may include but is not limited to: video acquisition, video encoding, video transmission, video decoding and playback processes.

[0063] The audio and video transmission system in Figure 1 includes a set-top box 110, a smart TV 120, multiple audio and video playback devices, and a server 130. The set-top box 110 is connected to the operator's network via a network cable and can receive audio and video streams from the server 130. The network can implement audio and video transmission and can include one or more network devices, such as a router or switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate via wireless communication, which is not limited in this embodiment of the present application.

[0064] The set-top box 110 is a video and audio processing device that is used to implement functions such as receiving, processing, and pushing video streams or video and audio streams. In some possible scenarios, the set-top box 110 may also be called an Internet TV set-top box, a network high-definition player, or other similar devices. For example, the set-top box 110 may refer to a TV box provided by a network operator or a TV box purchased by a user. For the hardware implementation of the set-top box 110, reference may be made to the description of FIG. 14 below, which will not be described in detail here.

[0065] Smart TV 120 is a display device with audio and video processing capabilities that implements functions such as receiving, processing, pushing, and playing video or audio streams. In some possible scenarios, Smart TV 120 may refer to an audio and video device such as a conference tablet, smart TV, or projector, but this embodiment of the application is not limited to this. The hardware implementation of Smart TV 120 can be referred to the description of Figures 14 or 15 below and will not be repeated here.

[0066] The multiple audio and video playback devices include audio and video playback devices 121 to 124. For example, these audio and video playback devices may include, but are not limited to, multimedia control platforms or other devices that support audio and video playback functions, such as virtual reality (VR) terminal devices or augmented reality (AR) terminal devices. The hardware implementation of the audio and video playback devices can be found in the description of FIG. 15 below and is not detailed here.

[0067] In this embodiment, the set-top box 110 and the smart TV 120 are connected via a network, for example, via an audio and video interface network. The set-top box 110 and each audio and video playback device may also be connected via an audio and video interface network. The smart TV 120 and each audio and video playback device may also be connected via an audio and video interface network. Exemplarily, the audio and video interface network may be a wired network that supports video and audio and video transmission. Optionally, the audio and video interface network may be referred to as a unified multimedia internet network. In some implementations, the unified multimedia internet network may also be referred to by other names.

[0068] The unified multimedia internet network supports both uncompressed and compressed video transmission, as well as advanced features such as quick video transport (QVT), auto low latency mode (ALLM), and dynamic frame rate refresh (DFR). It also supports LPCM audio and video formats defined in IEC 60958, as well as various HDR protocols, such as HDR Vivid, as specified in T / UWA 005.1-2022. The unified multimedia internet network also supports encryption control and protection for audio and video data transmission.

[0069] The server 130 may be an application server or an authentication and authorization server. The server 130 may provide video services, game services, message services, music services, authentication and authorization services, and the like. In one example, the functions of multiple services may be integrated on the server 130. For example, a game service and a music service may be deployed on the server 130. In another example, the functions of some services may be integrated on the server 130. For example, some services of the game service and some services of the video service may be deployed on the server 130. The server 130 may also utilize virtualization technology to provide multiple virtual machines, which provide various services. The embodiments of the present application do not limit the deployment form of the server. The network device 131 is connected to the server 130 via a wireless or wired mode. The schematic diagram of FIG1 is merely an example. Other devices may also be included in the network, which are not shown in FIG1.

[0070] Figure 1 is only a schematic diagram, and the video and audio transmission system may also include other devices, which are not shown in Figure 1. The embodiment of the present application does not limit the number and type of each device included in the system.

[0071] Based on the audio and video transmission system shown in Figure 1, Figure 2 is a schematic diagram of an audio and video encoding and decoding system provided in an embodiment of the present application. The audio and video encoding and decoding system includes a source device 210 and a host device 220. The source device 210 establishes a communication connection with the host device 220 through a unified multimedia Internet network.

[0072] The above-mentioned source device 210 can realize the function of audio and video encoding. As shown in Figure 1, the source device 210 can be the set-top box 110 or the smart TV 120 shown in Figure 1. The source device 210 can also be an audio and video control center with audio and video encoding capabilities. For example, the audio and video control center includes one or more servers.

[0073] The source device 210 may include a data source 211 , a pre-processing module 212 , an audio and video transmission adapter 213 , and a communication interface 214 .

[0074] The data source 211 may include or may be any type of electronic device for capturing audio and video, and / or any type of source audio and video generating device, such as a computer graphics processor for generating computer animation scenes or any type of device for acquiring and / or providing source audio and video, or computer-generated source audio and video. The data source 211 may be any type of memory or storage for storing the above-mentioned source audio and video. The above-mentioned source audio and video may include multiple audio and video streams or images captured by multiple audio and video capturing devices (such as cameras), such as ultra high definition (UHD) video, high definition (HD) video, 4K video, 8K video, etc.

[0075] The pre-processing module 212 is configured to receive source audio and video and pre-process the source audio and video to obtain audio and video or multiple frames of images. For example, the pre-processing performed by the pre-processing module 212 may include color format conversion (e.g., from RGB to YCbCr), octree structuring, audio and video splicing, audio track merging and deletion, or channel number adjustment.

[0076] The audio / video transmission adapter 213 is used to receive audio / video or images and encode the audio / video or images to obtain encoded data. In some optional situations, the encoded code stream (encoded data) may also be referred to as a bit stream. If the encoded data is obtained by encoding audio / video data, the bit stream refers to the audio / video stream; if the encoded data is obtained by encoding video data, the bit stream refers to the video stream.

[0077] The communication interface 214 in the source device 210 can be used to receive encoded data (such as a video stream or an audio / video stream) and send the encoded data (or a version of the encoded data after any other processing) to another device such as the host device 220 or any other device via the unified multimedia internet network for storage, display, playback, or image reconstruction.

[0078] Optionally, the source device 210 includes a bitstream buffer, which is used to store bitstreams corresponding to one or more coding units.

[0079] The above-mentioned sink device 220 can implement the function of audio and video decoding. As shown in Figure 1, when the source device 210 is a set-top box 110, the sink device 220 can be any of the smart TVs 120 or audio and video playback devices shown in Figure 1. When the source device 210 is a smart TV 120, the sink device 220 can be any of the audio and video playback devices.

[0080] The sink device 220 may include a video and audio playback unit 221 , a post-processing module 222 , a video and audio receiving adapter 223 , and a communication interface 224 .

[0081] The communication interface 224 in the sink device 220 is configured to receive the encoded data (or a version of the encoded data after any other processing has been performed on the encoded data) from the source device 210 or any other source device such as a storage device.

[0082] Communication interface 214 and communication interface 224 can be used to connect source device 210 and sink device 220 through a direct communication link, such as a direct wired connection, as shown in the unified multimedia internet network in FIG2 . For details about the unified multimedia internet network, please refer to the description of FIG1 and will not be repeated here.

[0083] The communication interface 224 corresponds to the communication interface 214 and can be used, for example, to transmit data and process the data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain audio and video data.

[0084] Both the communication interface 224 and the communication interface 214 can be configured as a unidirectional communication interface or a bidirectional communication interface as indicated by the arrow pointing from the source device 210 to the corresponding unified multimedia internet network of the sink device 220 in FIG2 , and can be used to send and receive messages, etc. to establish a connection, and transmit other information, etc., such as transmitting information related to the communication link, or, for example, transmitting information related to audio and video data (such as descriptive information DIP of audio and video).

[0085] The video and audio receiving adapter 223 is used to receive encoded data and decode the encoded data to obtain decoded data (video or video and audio, etc.).

[0086] The post-processing module 222 is used to post-process the decoded data to obtain post-processed data (such as an image to be displayed or audio and video to be played). The post-processing performed by the post-processing module 222 may include, for example, color format conversion (such as from YCbCr to RGB), octree reconstruction, audio and video splitting and fusion, or any other processing for generating data for output by the audio and video playback unit 221.

[0087] The audio and video playback unit 221 is used to receive post-processed data for display or playback to a user or viewer, etc. The audio and video playback unit 221 can be or include any type of display for representing the reconstructed image, such as an integrated or external display screen or display. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display screen. The audio and video playback unit 221 can also include one or more audio and video playback modules, each of which can refer to a speaker, a smart speaker, an amplifier, etc.

[0088] As an optional implementation, the source device 210 and the sink device 220 can transmit the encoded data through a data forwarding device. For example, the data forwarding device can be a router or a switch. It is worth noting that the data forwarding device needs to support the unified multimedia internet network.

[0089] Figure 3 shows a logical block diagram of an audio / video transmitter adapter. The adapter receives video and audio signals, including vertical synchronization (Vsync), horizontal synchronization (Hsync), pixel clock, video data, and display enable (DE), as well as descriptive information. These signals are then encapsulated into corresponding packets, such as the vertical blanking packet (VBP), horizontal blanking packet (HBP), active video packet (AVP), audio sample packet (ASP), and descriptive information packet (DIP). When content protection is enabled, the AVP and ASP are sent to the content protection encryption module for encryption, generating an encryption description packet (EDP) and a key distribution packet (KDP). The encrypted AVP is AVP', and the encrypted ASP is ASP'. These packets are combined into a single audio / video stream by the audio / video stream multiplexer and then sent to the audio / video receiver adapter of the sink device via the transport and electrical layers of the unified multimedia internetwork.

[0090] Figure 4 is a logic block diagram of an audio and video receiving adapter. The audio and video receiving adapter is responsible for receiving the audio and video stream, parsing it into various messages (VBP, HBP, DIP, AVP (AVP' in the case of encryption), ASP (ASP' in the case of encryption)), and restoring vertical synchronization (Vsync) / horizontal synchronization (Hsync) / pixel clock / pixel data / display enable (DE) / audio clock / audio data based on the messages.

[0091] The implementation of the video transmission method provided in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0092] Here, the video transmission method of an embodiment of the present application is described as being executed by the source device 210 or a chip in the source device 210 and the sink device 220 or a chip in the sink device 220 shown in FIG2 . FIG5 is a flow chart of a video transmission method provided by the present application. In this embodiment, the source device may also be referred to as an audio and video transmitting device, an audio and video transmitting terminal, etc., and the sink device may also be referred to as an audio and video receiving device, an audio and video playing device, etc. In this embodiment, the source device and the sink device are connected via a unified multimedia internet network.

[0093] In a first possible application scenario, the source device may be the set-top box 110 in Figure 1, and the sink device may be the smart TV 120 in Figure 1. For example, the set-top box pushes video and audio data to the smart TV.

[0094] In a second possible application scenario, the source device may be the set-top box 110 in FIG1 , and the sink device may be any video and audio playback device in FIG1 , such as any one of the video and audio playback devices 121 to 124. For example, the set-top box pushes video and audio data to the video and audio playback device.

[0095] In a third possible application scenario, the source device may be the smart TV 120 in FIG1 , and the sink device may be any of the video and audio playback devices in FIG1 , such as any of the video and audio playback devices 121 to 124. For example, the smart TV pushes video and audio data to the video and audio playback device.

[0096] The above three possible application scenarios are merely examples provided in this embodiment and should not be construed as limiting the present application. In other possible examples, the source device may be any of the video and audio playback devices in FIG1 (e.g., video and audio playback device 121), and the sink device may be another video and audio playback device different from the aforementioned video and audio playback device (e.g., video and audio playback device 122).

[0097] 5 , the video transmission method provided in the embodiment of the present application includes S501 - S503 .

[0098] S501: The source device queries the capability of the sink device.

[0099] In an embodiment of the present application, the sink device provides information indicating the capabilities of the sink device. The source device queries the capabilities of the sink device, including the source device obtaining capability description information of the sink device. The process of the source device obtaining the capability information of the sink device includes: the source device sends a query message to the sink device, where the query message is used to query the capabilities of the sink device; and then, the sink device sends (feeds back) the capability description information of the sink device to the source device, and the source device receives the capability description information sent by the sink device, where the capability description information may indicate the capabilities of the sink device.

[0100] The capability description information of the sink device may include information indicating the fast video transmission capability of the sink device, that is, including information indicating whether the sink device has the fast video transmission capability. In some possible situations, the capability description information may also be called a capability descriptor, which may include one or more bits of information.

[0101] In some cases, the sink device's capability description information may also be referred to as device comprehensive capability description information. The area in the sink device that stores this DCCD information is referred to as a DCCD area or simply DCCD, though this application is not limited to this. In the DCCD, the sink device declares whether it supports fast video transmission capabilities, such as fast video transmission capabilities. The source device queries the sink device's capabilities through the DCCD, and the sink device then provides feedback on its capabilities to the source device.

[0102] Optionally, the DCCD may also store other content describing the audio and video processing capabilities of the host device, such as audio processing capabilities (whether the host device supports processing pure audio data packets), HDR display capabilities (whether the host device supports displaying specific types of HDR videos, such as Colorful HDR), or others.

[0103] In an optional example, the sink device includes a memory storing the capability description information. The memory may include, but is not limited to, random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, a hard disk, a removable hard disk, a CD-ROM, or any other form of storage medium known in the art.

[0104] In another optional example, the sink device includes a dedicated register whose state is used to indicate the capability description information. If the value indicated by the register is 1, the capability description information indicates that the sink device has fast video transmission capability; if the value indicated by the register is 0, the capability description information indicates that the sink device does not have fast video transmission capability.

[0105] S502: If the sink device has a capability for fast video transmission, the source device transmits a first VBP and first video data to the sink device, wherein the first VBP includes a Quick Video Transmission (QVT) field with a first value, which indicates that a quick video transmission mode is enabled for the first video data. Accordingly, the sink device receives the first VBP and first video data sent by the source device.

[0106] The first video data is a frame of data in the video data to be sent by the source device, that is, the first video data corresponds to valid data (or valid pixels, or valid pixel data) of a frame image in the video.

[0107] It will be appreciated that when the source device determines that the sink device has fast video transmission capabilities, the source device can choose whether to instruct the sink device to enable fast video transmission mode based on actual needs. For example, the source device can choose whether to enable fast video transmission mode based on the type of video content. Alternatively, different types of video content may include, but are not limited to, graphics, images, videos, or games. The source device can also choose whether to enable fast video transmission mode for all types of video content.

[0108] In the embodiment of the present application, the vertical blanking message is used to transmit the VBS (Vblank Start) signal, i.e., the vertical blanking start signal. The vertical blanking message VBP includes a 4-byte message header (also called a tunnel message header) and a 28-byte message payload (also called a payload, load), totaling 32 bytes.

[0109] For example, referring to the structural diagram of VBP shown in Figure 6, the tunnel message header occupies 4 bytes, the message payload occupies 28 bytes, and a single VBP occupies a total of 32 bytes. The following is an exemplary description of the tunnel message header and message payload of VBP in combination with Table 1 and Table 2.

[0110] Table 1: VBP tunnel header

[0111] The message payload of the VBP includes frame-level control information (VFC), which is used to describe information related to the video of this frame.

[0112] Table 2: VBP message payload

[0113] In conjunction with Table 2, the QVT field in Table 2 stands for Quick Video Transport. The QVT field occupies 1 bit. A value of 1 in the QVT field indicates that the Quick Video Transport mode is enabled, while a value of 0 in the QVT field indicates that the Quick Video Transport mode is not enabled. For example, if the source device determines that the Quick Video Transport mode needs to be enabled, the source device sets the 1st bit of the QVT field in the VBP to 1 to instruct the sink device to enable the QVT mode.

[0114] In one possible implementation, the process of the source device transmitting the first VBP and the first video data to the host device specifically includes: the source device transmits a first video frame to the host device, the first video frame includes at least one first VBP and multiple first valid video packets AVP, and the multiple first AVPs are used to carry the first video data.

[0115] In the embodiment of the present application, after the source device (the video and audio transmission adapter in the source device) obtains the video data, it encapsulates the video data into an AVP message. It will be appreciated that for a frame of video data, the video data is encapsulated into multiple AVP messages. Optionally, the source device can send three VBPs to the sink device to ensure the success rate of VBP transmission.

[0116] As can be understood, referring to the block diagram of the audio and video transmission adapter shown in Figure 3 above, after the audio and video transmission adapter in the source device encapsulates the video data into AVP packets, the AVP packets in the source device are combined with other packets through the audio and video stream multiplexer to form a video stream. Other packets include vertical blanking packets (VBP), horizontal blanking packets (HBP), active video packets (AVP), audio sample packets (ASP), and descriptive information packets (DIP). The video stream includes multiple video frames, each of which corresponds to an image of the video.

[0117] Based on the above, it can be seen that in the embodiment of the present application, a video frame includes an AVP that carries video data and also includes a VBP that carries frame-level control information.

[0118] In an embodiment of the present application, VBP is sent in each video frame, and VBP includes a field for fast video transmission mode. In this way, frame-level control of video data can be achieved to enable fast video transmission mode for a certain video frame, which is highly flexible.

[0119] The format of a video frame in a video stream is described below in conjunction with FIG7 , which is a schematic diagram of the structure of a video frame provided by the present application. In FIG7 , a video frame includes: a multimedia message and a timing signal.

[0120] Multimedia packets include one or both of Active Video Packets (AVPs) and Audio Sample Packets (ASPs). AVPs are used to transmit video data, while ASPs are used to transmit audio data. As shown in Figure 7, multiple AVPs are arranged in rows within a video frame. Multiple AVPs belonging to the same row constitute a video row.

[0121] It is worth noting that the structure of the video frame in Figure 7 is only an example provided by this application and should not be understood as a limitation of this application. In some possible implementations, a single video frame may also include only one line of AVP message, which is not limited by this application.

[0122] Optionally, when the link bandwidth is greater than the video bandwidth, an ASP can be inserted between two AVPs, but the ASP has a lower priority than the AVP.

[0123] Timing signals include the horizontal synchronization signal (Hsync), the vertical synchronization signal (Vsync), and the display enable (DE) signal. These signals describe the video timing, that is, when which messages or signals should be transmitted. Hsync indicates the start of scanning a row of pixels (e.g., when the row register in the sink device receives a signal of 1), Vsync indicates the start of scanning a video frame, and display enable (DE) indicates whether valid data can be received (e.g., DE = 1 indicates that valid data can be received, and DE = 0 indicates that valid data cannot be received).

[0124] In conjunction with Figure 7, the video timing of the video frame is briefly explained: a video frame includes a vertical blanking area, a horizontal blanking area, and an active video area. In the blanking area, the video frame cannot transmit AVP messages, but can transmit other messages, such as ASP, DIP messages, etc.; in the video active area, the video frame can transmit AVP messages carrying audio and video data. Among them, the starting point of Hactive is marked by the horizontal blanking packet (HBP), which indicates the end of the line blanking area. Exemplarily, the audio and video sending adapter in the source device sends HBP for each video line and sends it immediately at the end of Hblank.

[0125] In the video frame shown in FIG7 , the line blanking area of ​​the video frame includes a VBP, which is used to indicate the video line where the rising edge of Vsync (in terms of positive polarity) of a video frame is located. Exemplarily, the audio and video transmission adapter in the source device sends one or more VBPs in each frame (such as three consecutive VBPs in FIG7 ), and ASP and DIP cannot be inserted between VBPs. For example, if Vsync is positive polarity, the audio and video transmission adapter replaces HBP with VBP on the video line where the rising edge of Vsync is located, and sends two more VBPs immediately thereafter; if Vsync is negative polarity, the audio and video transmission adapter replaces HBP with VBP on the video line where the falling edge of Vsync is located, and sends two more VBPs immediately thereafter.

[0126] Referring to the structural diagram of the valid video packet AVP shown in Figure 8, the AVP consists of two parts: a header and a payload (a total of 512 bytes). The length of the header part is 32 bits, and the payload part is used to transmit valid video pixel data (or called video data, or valid data of video, or valid pixels, or valid pixel data, or pixel data of valid video), and its length does not exceed 508 bytes.

[0127] The structure of the header of the valid video packet AVP is shown in Table 3 below.

[0128] Table 3: AVP header

[0129] As shown in Figure 8, each AVP is encapsulated as a 32-bit line, that is, in the payload of the AVP message, each line includes 32 bits of data. If 32 bits (i.e., 4 bytes) is defined as a symbol, then the payload of an AVP includes multiple symbols.

[0130] In an embodiment of the present application, when the fast video transmission mode (i.e., QVT mode) is enabled, the source device increases the transmission bandwidth occupied by each frame of video (i.e., the bandwidth of the virtual path used to transmit the video). After the bandwidth is adjusted, the transmission bandwidth of the first video data is the second bandwidth, which is the bandwidth adjusted by the source device to the first bandwidth of the currently transmitted video data, and the second bandwidth is greater than the first bandwidth. After the video transmission bandwidth is increased, the transmission time of the effective video data of each video frame (such as the above-mentioned first video data) is reduced, that is, the duration of the effective video area of ​​the first video frame is reduced, thereby quickly sending the first video frame (including the above-mentioned first VBP and the first video data) to the sink device, and the sink device will receive the content of each video frame more quickly.

[0131] In this embodiment of the present application, the relationship between the second bandwidth and the first bandwidth is: the second bandwidth is N+1 times the first bandwidth, where N represents the fast video transmission magnification, also known as the QVT multiplier, and N>0. Specifically, when the video transmission bandwidth is increased, the second bandwidth is determined to be the bandwidth obtained by multiplying the original first bandwidth by N times. The fast video transmission magnification N can also be understood as the multiplier by which the video transmission bandwidth is increased when QVT mode is enabled.

[0132] For example, Figure 9 illustrates the changes in the duration of the active video area and blanking area in a video frame in two scenarios: normal video data transmission (i.e., QVT mode is not enabled) and fast transmission (i.e., QVT mode is enabled). It is understood that a video frame includes a blanking area (including a horizontal blanking area and a vertical blanking area) and an active video area. When the fast video transmission mode is enabled, the video transmission bandwidth increases while the frame rate of the video data remains unchanged, which reduces the duration of the active video area of ​​the video frame. Since the frame rate of the video data remains unchanged, the duration of each video frame transmission remains unchanged. Therefore, the duration of the blanking area of ​​a video frame will increase.

[0133] Referring to Figure 9, during normal video transmission, the active video area is Tactive, and the blanking area is Tblank. In fast video transmission mode, the active video area, Tactive, is shortened to T'active, where T'active = Tactive / (1+N), where N is the fast video transmission factor. The blanking area, Tblank, is lengthened to T'blank. When the sink device receives valid video data from a video frame, its internal control devices (e.g., processing modules) need to be quickly awakened to process the video data in a timely manner.

[0134] Referring to Figure 10, if the fast video transmission rate in fast video transmission mode is N, the transmission bandwidth occupied by each video frame should be (1+N) times that in normal video transmission mode, and the required transmission time should be 1 / (1+N). During normal transmission, in a row of a video frame, the duration of Hactive is THactive, and the duration of the horizontal blanking area (Hblank) is THblank. After the video transmission bandwidth is increased (i.e., after the speed is increased), THactive and THblank are shortened to THactive' and THblank', respectively, and THactive' and THblank' satisfy:

[0135] In one line of the above-mentioned video frame, the duration of Hactive is shortened, and the duration of the horizontal blanking area (Hblank) is also shortened. It should be understood that when the number of active video lines in a video frame remains unchanged, the duration TVactive of Vactive is shortened to TVactive'. Since the frame rate of the video frame remains unchanged (that is, the duration of a video frame remains unchanged), the duration TVblank of the vertical blanking area increases to TVblank'.

[0136] It should be noted that the extended portion of the blanking area does not transmit HBP (horizontal blanking message). Optionally, when audio data needs to be transmitted, the extended portion of the blanking area can transmit audio sampling message ASP. When audio switching, audio control information (such as DIP) can be sent. If there is no audio data transmission, no message is transmitted in the extended portion of the blanking area.

[0137] It should be noted that after the fast video transmission mode is enabled, although the duration of the blanking area and the active video area of ​​the video frame has changed, the timing parameters in the frame-level control information of the vertical blanking message VBP (i.e., the message payload of the VBP) except the QVT parameters should remain unchanged.

[0138] Based on the above content, it can be seen that when the main link bandwidth between the source device and the host device is sufficient, when the fast video transmission mode is turned on, for the source device, on the one hand, the source device can quickly transmit the video data to the host device, thereby reducing the transmission delay of the video data between the source device and the host device; on the other hand, after the source device quickly completes the transmission of the video data, the source device can choose to turn off certain modules in the source device (such as the compression module), thereby reducing the power consumption of the source device.

[0139] 5 , as shown in FIG11 , to implement the QVT function, after the source device determines that the sink device has the capability of fast video transmission, before the source device sends the first VBP and the first video data to the sink device (i.e., before turning on the QVT mode), the source device should first execute S504-S506.

[0140] S504: The source device adjusts the video transmission bandwidth.

[0141] The above-mentioned video transmission bandwidth refers to the bandwidth of the source device sending video data.

[0142] In one case, when the source device and the sink device are directly connected (i.e., the source device and the sink device communicate directly, and there is no other intermediate device between the source device and the sink device), the source device adjusts the video transmission bandwidth according to the available bandwidth of the source device itself (i.e., the bandwidth capacity of the source device itself), increases the video transmission bandwidth, that is, adjusts the first bandwidth to the second bandwidth.

[0143] In another case, when there are one or more intermediate devices (such as switches) between the source device and the host device, the source device needs to interact with the intermediate devices to obtain the available bandwidth of all intermediate devices, and adjust the video transmission bandwidth according to the minimum value of the available bandwidth of the source device itself and the available bandwidth of each intermediate device (that is, the minimum available bandwidth), and adjust the first bandwidth to the second bandwidth. By adjusting the bandwidth according to the minimum value of the available bandwidth of the source device and all intermediate devices, it can be ensured that all devices transmitting video data support speeding up to the second bandwidth, thereby ensuring smooth transmission of video data.

[0144] It should be noted that when the source device adjusts bandwidth based on the available bandwidth, it also needs to consider the maximum video transmission rate supported by the sink device. The aforementioned video transmission rate N should be less than or equal to the maximum video transmission rate supported by the sink device. Optionally, the source device can also obtain the maximum video transmission rate supported by the sink device during the process of querying the sink device's capabilities (S501 above).

[0145] S505: Send the fast video transmission magnification to the sink device. Correspondingly, the sink device receives the fast video transmission magnification sent by the source device.

[0146] One possible implementation of the source device sending the fast video transmission magnification to the sink device is for the source device to write the actual transmission magnification (i.e., the fast video transmission magnification N) into the QVT multiplier register of the sink device through a configuration data write operation. The source device sends the video transmission magnification to the sink device to instruct the sink device to set the sink device's video processing mode according to the fast video transmission magnification.

[0147] Optionally, the source device may also send the fast video transmission rate to the sink device in other ways, which is not limited in the embodiment of the present application.

[0148] S506: The sink device sets a video processing mode of the sink device according to the fast video transmission magnification.

[0149] The sink device sets a video processing mode of the sink device so that the sink device can quickly process received video data.

[0150] In one possible implementation, the sink device sets the video processing mode of the sink device according to the fast video transmission ratio (i.e., sets the sink device capability), which may include: the sink device adjusts the transmission bandwidth between the various modules in the sink device based on the fast video transmission ratio. For example, the sink device increases the transmission bandwidth between the sink device's interface (used to receive video data, the interface including an audio and video receiving adapter) and the processing module (e.g., a graphics processing unit (GPU)). In this way, after receiving the video data, the sink device can quickly transmit the video data to the processing module for processing.

[0151] In one possible implementation, the sink device setting the video processing mode of the sink device may further include the sink device setting one or more processing modules for video quality enhancement from an off state to an on state. In this way, after the sink device quickly receives the video data, the sink device performs quality enhancement processing on the video data, thereby improving the video quality. It is understandable that the sink device turning off the processing module for video quality enhancement can be understood as the sink device entering a low-power mode (turning off the processing module for video quality enhancement can save power), and the sink device turning on the processing module for video quality enhancement can be understood as the sink device exiting the low-power mode.

[0152] In some examples, the processing for enhancing the image quality of the video includes but is not limited to at least one of resolution conversion, noise suppression, contrast enhancement (or dynamic contrast range enhancement), color correction, motion compensation, interpolation, audio and video synchronization, and super-resolution processing.

[0153] Of course, the method for the source device to set the video processing mode may also be other methods that enable the sink device to quickly process the received video data, and the embodiments of the present application are not limited to this.

[0154] It is understood that after the audio / video receiving adapter of the sink device receives the video stream, the sink device parses the first video frame (including the first VBP and the first video data) in the video stream to obtain the value of the QVT field in the VBP and determines whether to enable the fast video transfer mode. Specifically, if the fast video transfer field included in the first VBP has the first value (i.e., the value of the QVT field is 1), the sink device executes S503 below.

[0155] S503: Enable a fast video transmission mode for the first video data.

[0156] Corresponding to the VBP sent by the video sending adapter in the source device, the video receiving adapter in the sink device first performs a CRC check after receiving the VBP. If the check fails, the next VBP is received until the correct VBP is received. That is, the receiving end takes the first accurate VBP received as the basis.

[0157] In the embodiment of the present application, when the fast video transmission mode is enabled, the transmission time of the effective video data of each video frame is reduced, so the host device can receive the content of each video frame (eg, the first video data) more quickly.

[0158] In one implementation, after the sink device receives the video frame, it can immediately start and complete subsequent processing in advance, thereby reducing display delay.

[0159] In another implementation, after the sink device receives the video frame, if the video picture enhancement processing module is in a disabled state, the sink device may activate one or more of the above-mentioned video picture enhancement processing modules to improve the video quality.

[0160] In summary, in the video transmission method provided in the embodiment of the present application, when the source device queries that the sink device has fast video transmission capabilities, the source device can carry a field instructing the sink device to enable fast video transmission mode in the vertical blanking message, and send the vertical blanking packet (VBP) and video data to the sink device, so that the sink device enables the fast video transmission mode according to the instruction of the fast video transmission field. This method can dynamically enable the fast video transmission mode for the video data and reduce the transmission delay of the video data.

[0161] Furthermore, when the host device supports fast video transmission capability and needs to enable fast video transmission mode, the method can more conveniently and dynamically control the enabling of fast video transmission mode by carrying a field indicating enabling of fast video transmission mode in the VBP, without the need for manual enabling.

[0162] In one possible implementation, in the video transmission method provided in the embodiment of the present application, the fast video transmission mode can also be turned off during the video data transmission process. In conjunction with Figure 11, as shown in Figure 12, after the above S503, the video transmission method provided in the embodiment of the present application further includes S507-S508.

[0163] S507: The source device transmits a second vertical blanking packet (VBP) and the second video data to the sink device. The fast video transfer field included in the second VBP is a second value, which indicates that the fast video transfer mode is disabled for the second video data. Accordingly, the sink device receives the second VBP and the second video data sent by the source device.

[0164] In one possible implementation, the process of the source device transmitting the second VBP and the second video data to the host device specifically includes: the source device transmits a second video frame to the host device, the second video frame includes at least one second VBP and multiple second valid video packets AVP, and the multiple second AVPs are used to carry the second video data.

[0165] Optionally, when the source device determines that the fast video transmission mode needs to be turned off, the source device can adjust the second bandwidth of the currently transmitted video (that is, the bandwidth increased when the fast video transmission mode is turned on) to reduce the video transmission bandwidth, for example, to a third bandwidth, that is, the transmission bandwidth of the second video data is the third bandwidth, and the third bandwidth is the bandwidth after the source device adjusts the second bandwidth of the currently transmitted video data, and the third bandwidth is smaller than the second bandwidth.

[0166] In some examples, the video transmission bandwidth can be reduced to the bandwidth before the fast video transmission mode is turned on, such as the first bandwidth mentioned above, then the first bandwidth is equal to the third bandwidth. The video transmission bandwidth can also be reduced to other values, which is determined according to actual needs and is not limited in the embodiments of this application.

[0167] After the audio and video receiving adapter of the host device receives the video stream, for the second video frame in the video stream (including the second VBP and the second video data), when the fast video transport field included in the second VBP is the second value (that is, the value of the QVT field is 0), the host device executes S508.

[0168] S508: The sink device disables the fast video transmission mode for the second video data.

[0169] In conjunction with the description of the process of enabling fast video transmission mode on the sink device in the above embodiment, and with reference to the setting of the sink device's video processing mode described in S506 above, the sink device will perform the opposite process. For example, in one possible implementation, when the sink device disables fast video transmission mode for the second video data, the sink device may reduce the bandwidth between the sink device's interface and the sink device's processing module. In another possible implementation, if the sink device enables one or more processing modules for video quality enhancement when enabling fast video transmission mode, then when the sink device disables fast video transmission mode for the second video data, the sink device sets the one or more processing modules for video quality enhancement to a disabled state.

[0170] Based on the above content, it can be seen that the source device sends VBP in each video frame, and the VBP includes a field for fast video transfer mode. In this way, frame-level control of video data can be achieved to dynamically turn on or off the fast video transfer mode for video frames.

[0171] It is understood that in order to implement the functions in the above embodiments, the source device and the sink device include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily appreciate that, in combination with the units and method steps of the various examples described in the embodiments disclosed in this application, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a manner driven by computer software depends on the specific application scenario and design constraints of the technical solution.

[0172] The video transmission method provided according to this embodiment is described in detail above with reference to FIG. 1 to FIG. 12 . The video processing device provided according to this embodiment will be described below with reference to FIG. 13 .

[0173] Figure 13 is a structural diagram of a video processing device provided in an embodiment of the present application. The video device can be used to implement the functions of any one of the devices in the above method embodiments, and therefore can also achieve the beneficial effects possessed by the above method embodiments. In this embodiment, the video processing device can be a set-top box 110, a smart TV 120, or any display device as shown in Figure 1, or a source device 210 or a sink device 220 as shown in Figure 2, or a source device or sink device provided in subsequent embodiments. It should be understood that the video processing device can also be a module (such as a chip) applied to any of the aforementioned devices.

[0174] As shown in Figure 13, the video processing device includes a transceiver module 1301 and a processing module 1302. Transceiver module 1301 and processing module 1302 can collaboratively implement the various steps in the aforementioned method embodiment. A more detailed description of transceiver module 1301 and processing module 1302 can be directly obtained by referring to the relevant description of the device in the method embodiment shown in the aforementioned figures, and is not repeated here.

[0175] When a video processing device implements any of the video transmission methods shown in the aforementioned figures through software, the video processing device and its various units may also be software modules. The aforementioned video transmission method is implemented by a processor calling the software module. The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0176] It can be understood that the video processing device shown in FIG13 is only an example provided in this embodiment. The video processing device may include more or fewer units according to different video transmission processes, and this application does not limit this.

[0177] When the video processing device is implemented via hardware, the hardware may be implemented via a processor or a chip system. The chip system includes one or more chips, each of which includes an interface circuit and a control circuit. The interface circuit is used to receive data from other devices outside the chip and transmit it to the control circuit, or to send data from the control circuit to other devices outside the chip. The control circuit and the interface circuit are used to implement any possible implementation method of the above embodiments through logic circuits or execution code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments and will not be repeated here.

[0178] It is understood that the processor in the embodiments of the present application may be a CPU, or other general-purpose processor, digital signal processor (DSP), ASIC, FPGA or other programmable logic device, transistor logic device, hardware component or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0179] The video processing apparatus shown in FIG13 may also be implemented by a video processing device. FIG14 is a schematic structural diagram of the video processing device provided in this application. The video processing device includes: a memory 1401 and at least one processor 1402. The processor 1402 may implement the video transmission method provided in the above embodiment. The memory 1401 is used to store software instructions corresponding to the above video transmission method.

[0180] As an optional implementation, in hardware implementation, the video processing device may refer to a chip or chip system encapsulated with one or more processors 1402. For example, when the video processing device is used to implement the method steps in the above embodiment, the processor 1402 included in the video processing device executes the steps of the source device in the above method and its possible sub-steps. In an optional scenario, the video processing device may also include a communication interface 1403, which can be used to send and receive data. For example, the communication interface 1403 is used to receive audio and video data, or to send audio and video streams, etc.; the communication interface 1403 can be implemented by the interface circuit included in the video processing device. Therefore, in some examples, the communication interface 1403 can also be called a transceiver of the video processing device. In this embodiment, the communication interface 1403 supports the use of a unified multimedia internet network.

[0181] In an embodiment of the present application, the communication interface 1403, the processor 1402, and the memory 1401 may be connected via a bus 1404, which may be divided into an address bus, a data bus, a control bus, etc. The bus 1404 may be a Peripheral Component Interconnect Express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), or other types of buses.

[0182] It is worth noting that the video processing device can also perform the functions of the video processing apparatus shown in FIG13 , which will not be described in detail here.

[0183] The video processing device provided in this embodiment may be the set-top box 110, smart TV 120, source device 210, or other devices with video processing capabilities, and this application is not limited thereto. For example, when the aforementioned display device also has a video processing function, the video processing device may refer to any of the aforementioned display devices.

[0184] In addition, the video processing device shown in FIG13 can also be implemented by a display device. When the video processing device is implemented by a display device, this embodiment provides a possible example, as shown in FIG15 , which is a schematic structural diagram of the display device provided by the present application. The display device includes: a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a unified multimedia interconnection interface 331, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, a sensor module 380, a button 390, an indicator 392, a camera 393, a display screen 394, a subscriber identification module (SIM) card interface 1-N 395, etc.

[0185] Among them, the above-mentioned sensor module 380 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0186] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the display device. In other embodiments, the display device may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0187] The processor 310 may include one or more processing units. For example, the processor 310 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0188] The controller can be the nerve center and command center of the display device. The controller can generate operation control signals based on instruction opcodes and timing signals to complete the control of instruction fetching and execution.

[0189] Processor 310 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 310 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 310. If processor 310 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 310 latency, and thus improves system efficiency.

[0190] In some embodiments, the processor 310 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, a unified multimedia interconnect interface, etc.

[0191] It is understood that the interface connection relationship between the modules illustrated in this embodiment is only for illustrative purposes and does not constitute a structural limitation on the display device. In other embodiments, the display device may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0192] The wireless communication function of the display device can be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, a modem processor, and a baseband processor. In some embodiments, antenna 1 of the display device is coupled to mobile communication module 350, and antenna 2 is coupled to wireless communication module 360, so that the display device can communicate with the network and other devices through wireless communication technology.

[0193] The wired communication function of the display device can be implemented through the USB interface 330 or the unified multimedia interconnection interface 331. For example, the display device receives or sends video streams and AVP messages through the bus connected to the unified multimedia interconnection interface 331.

[0194] The display device implements display functions through a GPU, display screen 394, and an application processor. A GPU is a microprocessor for image processing that connects display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 310 may include one or more GPUs that execute program instructions to generate or modify display information.

[0195] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel.

[0196] The display device can implement a camera function through an ISP, a camera 393, a video codec, a GPU, a display 394, and an application processor. The ISP processes data fed back by the camera 393. The camera 393 is used to capture still images or videos. In some embodiments, the display device may include one or N cameras 393, where N is a positive integer greater than 1.

[0197] In this embodiment, the above display screen 394, video codec, GPU, display screen 394 and application processor can also be collectively referred to as a display unit of the host device, which is used to process and display the received video stream.

[0198] External memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the display device. The external memory card communicates with processor 310 via external memory interface 320 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0199] The internal memory 321 can be used to store computer executable program code, which includes instructions. The processor 310 executes various functional applications and data processing of the display device by running the instructions stored in the internal memory 321. For example, in an embodiment of the present application, the processor 310 can execute instructions stored in the internal memory 321, and the internal memory 321 can include a program storage area and a data storage area.

[0200] The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the display device (such as audio and video data, a phone book, etc.). In addition, the internal memory 321 can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, a flash memory device, or a universal flash storage (UFS).

[0201] The display device can implement audio functions such as music playback and recording through the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, and the application processor.

[0202] Buttons 390 include a power button, a volume button, and the like. Buttons 390 may be mechanical buttons or touch buttons. Indicator 392 may be an indicator light that can indicate charging status, battery level changes, messages, missed calls, notifications, and the like.

[0203] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using a software program, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions in accordance with the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a magnetic disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state drive (SSD)).

[0204] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0205] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0206] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0207] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0208] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

[0209] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A video transmission method, characterized in that: Applied to a source device or a chip in the source device, the method includes: Query the capabilities of the sink device; In a case where the sink device has a fast video transmission capability, transmitting a first vertical blanking packet VBP and the first video data to the sink device; The fast video transmission field included in the first VBP is a first value, and the first value is used to indicate that a fast video transmission mode is enabled for the first video data.

2. The method according to claim 1, characterized in that The transmission bandwidth of the first video data is a second bandwidth, the second bandwidth is a bandwidth obtained by adjusting the first bandwidth of the currently transmitted video data by the source device, and the second bandwidth is greater than the first bandwidth.

3. The method according to claim 2, characterized in that The second bandwidth is N+1 times the first bandwidth; wherein N represents a fast video transmission rate, and N>0.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: A fast video transmission magnification is sent to the sink device to instruct the sink device to set a video processing mode of the sink device according to the fast video transmission magnification.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Transmitting a second vertical blanking message VBP and second video data to the sink device; The fast video transmission field included in the second VBP is a second value, and the second value is used to indicate that the fast video transmission mode is turned off for the second video data.

6. The method according to any one of claims 1 to 5, characterized in that Transmitting a first VBP and first video data to the sink device includes: A first video frame is transmitted to the sink device, where the first video frame includes at least one first VBP and multiple first valid video packets AVP, where the multiple first AVPs are used to carry the first video data.

7. The method according to claim 6, characterized in that When the fast video transmission mode is enabled, the duration of the active video area of the first video frame is reduced.

8. The method according to claim 5, characterized in that Transmitting the second VBP and the second video data to the sink device includes: A second video frame is transmitted to the sink device, where the second video frame includes at least one second VBP and multiple second valid video packets AVPs, where the multiple second AVPs are used to carry the second video data.

9. A video transmission method, characterized in that: Applied to a sink device or a chip in the sink device, the method includes: receiving a first vertical blanking message VBP and first video data sent by a source device; When the fast video transmission field included in the first VBP is a first value, a fast video transmission mode is enabled for the first video data.

10. The method according to claim 9, characterized in that The transmission bandwidth of the first video data is a second bandwidth, the second bandwidth is a bandwidth obtained by adjusting the first bandwidth of the currently transmitted video data by the source device, and the second bandwidth is greater than the first bandwidth.

11. The method according to claim 10, characterized in that The second bandwidth is N+1 times the first bandwidth; wherein N represents a fast video transmission rate, and N>0.

12. The method according to any one of claims 9 to 11, characterized in that The method further comprises: receiving a fast video transmission rate sent by the source device; The sink device sets a video processing mode of the sink device according to the fast video transmission magnification.

13. The method according to any one of claims 9 to 12, characterized in that The method further comprises: receiving a second vertical blanking message VBP and second video data sent by a source device; When the fast video transfer field included in the second VBP is a second value, the fast video transfer mode is turned off for the second video data.

14. The method according to any one of claims 9 to 13, characterized in that Receiving a first vertical blanking message VBP and first video data sent by a source device, including: A first video frame sent by the source device is received, where the first video frame includes at least one of the first VBPs and multiple first valid video packets AVPs, where the multiple first AVPs are used to carry the first video data.

15. The method according to claim 14, characterized in that When the fast video transmission mode is enabled, the duration of the active video area of the first video frame is reduced.

16. The method according to claim 13, characterized in that Receiving a second vertical blanking message VBP and second video data sent by a source device, including: A second video frame sent by the source device is received, where the second video frame includes at least one second VBP and multiple second valid video packets AVP, where the multiple second AVPs are used to carry the second video data.

17. A video processing device, characterized in that: include: A memory, a transceiver, and a processor; the memory, the transceiver, and the processor are configured to collaboratively execute the method according to any one of claims 1 to 8.

18. A sink device, characterized in that: include: A transceiver and a processor; the transceiver and the processor are configured to collaboratively execute the method according to any one of claims 9 to 16.

19. A computer-readable storage medium, characterized in that Computer instructions are stored, which, when executed on a computing device, perform the method as claimed in any one of claims 1 to 8 or any one of claims 9 to 16.

Images