A video data transmission method, device, equipment, medium and product

Abstract

The present invention discloses a method, device, equipment, medium and product for transmitting video data, and relates to the field of server technology. Based on the first moment data and the second moment data, the first delay moment data of each frame data can be determined, and the second delay moment data is obtained by averaging based on a continuous preset number of first delay moment data. And the corresponding transmission mode is determined according to the relationship between the second delay moment data and the preset value. So that the control system at the local end enters the low-latency mode for transmission, and its current frame data to be transmitted skips the writing and reading process of DDR, and is directly transmitted from the local end to the remote end, thereby reducing the transmission delay of the video data and improving the transmission efficiency. At the same time, by switching between different transmission modes, the transmission method of the video data of the SoC chip is diversified and flexible.

Description

Technical Field

[0001] The present invention relates to the field of server technology, and in particular to a video data transmission method, device, equipment, medium and product. Background Art

[0002] System on Chip (SoC) is a highly integrated integrated circuit that integrates multiple functions and various peripheral interfaces in traditional electronic systems onto a single chip, which not only improves processing efficiency but also greatly reduces power consumption and cost.

[0003] The Double Data Rate Synchronous Dynamic Random-Access Memory (DDR) bus of traditional SoC chips is not always occupied by video and network functions, but also runs other program functions or processes. In addition, the video data in the SoC chip will first be written to the DDR through the DDR bus, and then the network module will read from the DDR. The write and read operations will occupy more system bus and memory bandwidth, which will lead to transmission delays of video data.

[0004] Therefore, how to improve the transmission efficiency of video data is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0005] The purpose of the present invention is to provide a video data transmission method, device, equipment, medium and product to solve the problem of data transmission delay caused by the DDR bus not being always occupied by the video function.

[0006] In order to solve the above technical problems, the present invention provides a method for transmitting video data, comprising:

[0007] Recording first moment data written into a double data rate synchronous dynamic random access memory with a preset number of consecutive frame data of a current frame and second moment data read from the double data rate synchronous dynamic random access memory within a preset time;

[0008] Perform difference processing on each of the second time data and each of the first time data to obtain corresponding first delayed time data; and perform average processing on each of the first delayed time data to obtain second delayed time data;

[0009] Determine the transmission mode of the frame data to be transmitted according to the relationship between the second delay time data and the preset value; and transmit the frame data to be transmitted to the remote end according to the transmission mode; wherein the transmission mode includes a normal mode and a low-delay mode; in the low-delay mode, the local end skips the storage process of the double data rate synchronous dynamic random access memory and directly transmits it to the remote end.

[0010] On the one hand, before writing a preset number of consecutive frame data into the double data rate synchronous dynamic random access memory, frame header attribute information is pre-added to the frame header of each frame data;

[0011] Before a preset number of consecutive frame data are read out from the double data rate synchronous dynamic random access memory, frame tail attribute information is pre-added to the frame tail of each frame data.

[0012] On the other hand, both the frame header attribute information and the frame tail attribute information include a frame sequence number, and the frame header attribute information and the frame tail attribute information of the same frame data have the same frame sequence number.

[0013] On the other hand, the bit width data of the frame header attribute information and the frame tail attribute information of each frame data is the same.

[0014] On the other hand, frame header attribute information is added to the frame header of each frame data in advance, including:

[0015] Before the system-level chip outputs the current frame data, the frame header attribute information is filled into the advanced expansion interface bus transmitted between the system-level chip and the double data rate synchronous dynamic random access memory;

[0016] Correspondingly, the frame tail attribute information is pre-added to the frame tail of each frame data, including:

[0017] When the filled current frame data is outputted by the system-level chip, the frame tail attribute information is filled into the advanced expansion interface bus transmitted between the system-level chip and the double data rate synchronous dynamic random access memory so as to be written into the double data rate synchronous dynamic random access memory.

[0018] On the other hand, determining the transmission mode of the frame data to be transmitted currently according to the relationship between the second delay time data and the preset value includes:

[0019] When the second delay time data exceeds the preset value and the current mode is the normal mode, the normal mode is switched to the low delay mode;

[0020] When the second delay time data does not exceed the preset value and the current mode is the normal mode, maintaining the normal mode;

[0021] When the data at the second delay time exceeds the preset value and the current mode is the low delay mode, maintaining the low delay mode;

[0022] When the data at the second delay moment does not exceed the preset value and the current mode is the low-delay mode, exit the low-delay mode and switch to the normal mode.

[0023] On the other hand, after determining the transmission mode of the frame data to be transmitted according to the relationship between the second delay time data and the preset value, and before transmitting the frame data to be transmitted to the remote end according to the transmission mode, the method further includes:

[0024] Configure a corresponding first register value according to the determined transmission mode;

[0025] The corresponding switching instruction is started according to the corresponding first register value so as to switch to the determined transmission mode to transmit the frame data currently to be transmitted.

[0026] On the other hand, transmitting the frame data to be transmitted to the remote end according to the low-delay mode includes:

[0027] Setting the write address information output by the master device of the advanced expansion interface bus transmitted between the system-level chip and the double data rate synchronous dynamic random access memory as the receiving first-in-first-out address of the driver of the embedded media access controller;

[0028] The frame data to be transmitted currently is transmitted to the remote end according to the burst transmission mode and the receiving first-in-first-out address of the driver of the embedded media access controller.

[0029] On the other hand, transmitting the frame data to be transmitted to the remote end according to the burst transmission mode and the receiving first-in-first-out address of the driver of the embedded media access controller includes:

[0030] Acquire the remaining capacity of the first-in-first-out buffer corresponding to the first-in-first-out address of the driver of the embedded media access controller and the output data bit width of the system-level chip;

[0031] Determining the burst length of the frame data to be transmitted currently according to the remaining capacity of the FIFO buffer and the output data bit width;

[0032] The frame data to be transmitted currently is transmitted to the remote end according to the burst length and a receiving first-in-first-out address of a driver of the embedded media access controller.

[0033] On the other hand, transmitting the frame data to be transmitted to the remote end according to the low-delay mode includes:

[0034] When the first register value is the register value corresponding to the low latency mode, a first interface conversion instruction is triggered; according to the first interface conversion instruction, a master device of the advanced extension interface bus is connected to a slave device of the advanced extension interface bus, and an indication signal of the remaining capacity of the first-in-first-out buffer corresponding to the first-in-first-out address of the driver of the embedded media access controller is received to facilitate direct connection to the remote end.

[0035] On the other hand, after receiving the indication signal of the remaining capacity of the first-in-first-out buffer corresponding to the first-in-first-out address of the driver of the embedded media access controller, the method further includes:

[0036] Receiving the frame data to be transmitted currently through a slave device of the advanced expansion interface bus;

[0037] The frame data to be transmitted is processed according to an Ethernet packetization method to send the data to the remote end.

[0038] On the other hand, transmitting the frame data currently to be transmitted to the remote end according to the conventional mode includes:

[0039] When the first register value is a register value corresponding to the normal mode, a second interface conversion instruction is triggered; according to the second interface conversion instruction, a master device of the advanced extension interface bus is connected to a system bus of the system-level chip, so as to output the video data currently to be transmitted to the double data rate synchronous dynamic random access memory, and transmit it to the remote end;

[0040] The slave device of the advanced expansion interface bus is turned off so as not to receive the video data currently to be transmitted.

[0041] On the other hand, before transmitting the frame data to be transmitted to the remote end according to the low-delay mode, the method further includes:

[0042] When the first register value is a register value corresponding to the low-latency mode, controlling the embedded media access controller to receive the frame data currently to be transmitted and the corresponding data capacity;

[0043] Obtaining the remaining space capacity of a receiving first-in-first-out buffer in the embedded media access controller;

[0044] Determine the remaining space capacity of the current receiving FIFO buffer according to the relationship between the remaining space capacity of the receiving FIFO buffer in the embedded media access controller and the data capacity of the frame data to be currently transmitted;

[0045] If the remaining space capacity of the current receiving FIFO buffer exceeds a preset range, the frame data to be transmitted is transmitted.

[0046] On the other hand, before transmitting the frame data to be transmitted to the remote end according to the conventional mode, the method further includes:

[0047] When the first register value is a register value corresponding to the normal mode, the embedded media access controller is controlled to use the master device and descriptor mechanism of the advanced expansion interface bus to obtain the current frame data to be transmitted from the double data rate synchronous dynamic random access memory for transmission to the remote end.

[0048] On the other hand, the process of establishing the receiving FIFO buffer in the embedded media access controller includes:

[0049] When the first register value is a register value corresponding to the low latency mode, writing the write data of the slave device of the advanced expansion interface bus into the receiving first-in-first-out buffer in the embedded media access controller;

[0050] When the first register value is a register value corresponding to the normal mode, the read return data of the master device of the advanced expansion interface bus is written into the receiving first-in-first-out buffer in the embedded media access controller.

[0051] On the other hand, after transmitting the frame data to be transmitted to the remote end according to the transmission mode, the method further comprises:

[0052] Receiving request information corresponding to the display status of the video data at the remote end;

[0053] Configure the corresponding video function register value according to the request information;

[0054] Correspondingly, configuring the corresponding video function register value according to the request information includes:

[0055] When the video function is normal, configuring the first video function normal register value according to the request information of the video function being normal;

[0056] In case of video function abnormality, the corresponding second video function abnormality register value is configured according to the request information of the video function abnormality.

[0057] On the other hand, after outputting the second video function abnormality register value, the method further includes:

[0058] Acquire the frame data to be transmitted and the corresponding line and field synchronization signal;

[0059] Calculating the video resolution information corresponding to the frame data to be transmitted currently according to the line field synchronization signal;

[0060] transmitting the frame data to be transmitted currently to the double data rate synchronous dynamic random access memory according to the video resolution information;

[0061] If frame loss occurs when the frame data currently to be transmitted is transmitted to the double data rate synchronous dynamic random access memory, the frame data currently to be transmitted is discarded according to the video resolution information of the frame data currently to be transmitted.

[0062] On the other hand, after transmitting the frame data to be transmitted to the remote end according to the transmission mode, the method further comprises:

[0063] receiving feedback data sent by the remote end;

[0064] The frame data of the next frame is obtained according to the flag information of the feedback data; and the frame data of the next frame is returned to the step of recording the first moment data written into the double data rate synchronous dynamic random access memory with a continuous preset number of frame data of the current frame within a preset time, and the second moment data read from the double data rate synchronous dynamic random access memory to transmit the frame data of the next frame.

[0065] In order to solve the above technical problems, the present invention further provides a video data transmission device, comprising:

[0066] A recording module, used to record first moment data written into a double data rate synchronous dynamic random access memory with a preset number of consecutive frame data of a current frame and second moment data read from the double data rate synchronous dynamic random access memory within a preset time;

[0067] A processing module, used for performing difference processing on each of the second moment data and each of the first moment data to obtain corresponding first delayed moment data; and performing average processing on each of the first delayed moment data to obtain second delayed moment data;

[0068] A transmission module, used to determine the transmission mode of the frame data to be transmitted currently according to the relationship between the second delay time data and the preset value; and transmit the frame data to be transmitted currently to the remote end according to the transmission mode; wherein the transmission mode includes a normal mode and a low-delay mode; the low-delay mode is skipped by the local end to skip the storage process of the double data rate synchronous dynamic random access memory and directly transmit to the remote end.

[0069] In order to solve the above technical problems, the present invention further provides a video data transmission device, comprising:

[0070] Memory for storing computer programs;

[0071] A processor is used to implement the steps of the video data transmission method when executing the computer program.

[0072] In order to solve the above technical problem, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the video data transmission method as described above are implemented.

[0073] In order to solve the above technical problem, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the video data transmission method when executed by a processor.

[0074] The present invention provides a video data transmission method, which configures a local control system, writes a preset number of frame data to DDR continuously, and the time taken to completely read the preset number of frame data back from DDR, that is, each frame data of the preset number continuously records the first moment data corresponding to writing into DDR, and the second moment data used to completely read the frame data back from DDR, and the first delay moment data of each frame data can be determined based on the first moment data and the second moment data, and the second delay moment data is obtained by performing mean processing based on the first delay moment data of the preset number continuously. And the corresponding transmission mode is determined according to the relationship between the second delay moment data and the preset value.

[0075] The beneficial effect of the present invention is that the corresponding transmission mode is determined according to the relationship between the second delay time data and the preset value, so that the control system of the local end enters the low-delay mode for transmission, and its current frame data to be transmitted skips the DDR writing and reading process, and is directly transmitted from the local end to the remote end, thereby reducing the transmission delay of the video data and improving the transmission efficiency. At the same time, by switching between different transmission modes, the transmission mode of the video data of the SoC chip is diversified and flexible, and the occupation of the system bus and memory bandwidth by the video function is reduced without affecting the normal function of the chip, thereby improving the overall performance of the chip, and improving the display quality and human-computer interaction experience of the remote end.

[0076] Secondly, the bit width data of the frame header attribute information and the frame tail attribute information of each frame data are the same, which ensures the accuracy of the data recorded at the source and the accuracy of the data transmission mode. Based on the recording process of the frame header and the frame tail, the statistical process of the first moment data and the second moment data is obtained, which ensures the accuracy of data recording at the source and improves the accuracy of the subsequent data transmission mode. The flexible switching process of the two transmission modes reduces the occupation of the system bus and memory bandwidth by the video function without affecting the normal function of the chip, improves the overall performance of the chip, and improves the display quality and human-computer interaction experience of the remote end. Through the conversion of the switching instruction of the hardware flag bit of the first register value, the switching process of different output modes is realized, ensuring that data is transmitted in the current correct transmission mode and improving the data transmission efficiency. Data is transmitted through burst transmission mode to improve transmission efficiency and ensure data integrity transmission. The burst length determination process improves data transmission efficiency through the balance processing of the remaining capacity of the FIFO and the output data bit width. The port connection control and conversion process in the low-latency mode cooperates with the trigger instruction of the first register value to ensure data transmission in the low-latency mode.

[0077] In addition, the present invention also provides a video data transmission device, equipment, medium and product, which have the same beneficial effects as the above-mentioned video data transmission method. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0079] Figure 1 A schematic diagram of the video function of the traditional server management control chip;

[0080] Figure 2 A flowchart of a video data transmission method provided by an embodiment of the present invention;

[0081] Figure 3 A schematic diagram of a server management and control chip video function provided by an embodiment of the present invention;

[0082] Figure 4 A logical schematic diagram of a network mode provided by an embodiment of the present invention;

[0083] Figure 5 A flowchart of a network module processing data to be sent provided by an embodiment of the present invention;

[0084] Figure 6A flowchart of EMAC data transmission in a conventional mode provided by an embodiment of the present invention;

[0085] Figure 7 A structural diagram of a video data transmission device provided by an embodiment of the present invention;

[0086] Figure 8 A structural diagram of a video data transmission device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0087] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0088] The core of the present invention is to provide a video data transmission method, device, equipment, medium and product to solve the problem of data transmission delay caused by the DDR bus not being occupied by the video function all the time.

[0089] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0090] The SoC chip integrates video compression and network functions. Taking the server management control chip as an example, the server management control chip refers to the server field. This chip is used to monitor the status of the server (temperature, fan, main central processing unit (CPU) operation status, etc.). It also has an important function, which is to transmit local video information to the remote end through the network for remote display and monitoring. Figure 1 A schematic diagram of the video function of the traditional server management control chip, such as Figure 1As shown, the original video data of the host is transmitted to the video graphics array (VGA) module inside the baseboard management control chip through the peripheral component interconnect express (PCIe). VGA generates original video data in the original optical three primary colors (Red-Green-Blue, RGB) format, and then after color space conversion, generates YUV format video data formed by three components of pixel format brightness (Y), blue color difference (U) and red color difference (V). The YUV format data has two channels. One channel is through video compression interleaved picture (IP) format (such as H.264 format, Joint Photographic Experts Group (JPEG) format, etc.), to obtain compressed video data, and the compressed data is written to the external DDR through the output control module. The other channel is the video in YUV format directly input to the output control module, and then written to the external DDR. The network driver then obtains compressed video data or YUV format data from the DDR, and sends it to the remote end through the Ethernet Media Access Controller (EMAC) of the network module for display by the remote end.

[0091] The original video data from the host (server operating system interface) is transmitted to the VGA module inside the baseboard management control chip through PCIe. VGA generates the original video data in RGB format (such as Figure 1 As shown, space A in DDR is needed, where space A stores the original video data in RGB format). The data in RGB format is converted into color space according to the matrix conversion formula to obtain video data in YUV format. The matrix conversion formula is:

[0092] Y=(0.257*R)+(0.504*G)+(0.098*B)+16;

[0093] U=0.148*R–0.291*G+0.439*B+128;

[0094] V=0.439*R–0.368*G–0.071*B+128.

[0095] The video data in YUV format has two channels, one is input to the post-stage BLOCK conversion module to generate BLOCK format data, and the other is directly input to the output control module and then written into DDR.

[0096] The BLOCK conversion module implements the BLOCK conversion of YUV data. According to the YUV444 / YUV422 / YUV422 format, the size of the BLOCK is 16*16 or 8*8 or 16*8. The core compression module receives the YUV data in BLOCK format, and then compresses the video to obtain compressed data such as JPEG format, and outputs it to the output control module. The output control module receives the video in compressed format or YUV format, processes it internally, and writes it to DDR in the form of a complete frame (write to Figure 1 The space B in the output control module is the frame data storage space output by the output control module. Meanwhile, the output interface of the output control module is the master device of the advanced eXtensible Interface Master (AXIMaster) bus, which is connected to the DDR bus.

[0097] The EMAC driver (network driver) reads the data to be sent from DDR through the AXI Master interface of EMAC. (The DDR address read at this time Figure 1 Space B in the address space, and then write it to the specified DDR address space (such as TX_DDR). The EMAC driver sends the send descriptor and the receive descriptor to the DDR. The EMAC reads back the descriptor and parses it to obtain the address of the data to be sent (TX_ADDR), the length of the data to be read, and the address for storing the received data (RX_ADDR). The EMAC then obtains the send data from TX_ADDR, reads the data from DDR back to the EMAC for packetization, and then sends it to the remote end. At the same time, it receives the data packet from the remote end and writes it into RX_ADDR.

[0098] Since the DDR of the SoC chip will not always be used for video functions and network functions, that is, a large number of other programs or processes are running on the SoC chip, which are occupying DDR or system bus. At the same time, in the traditional solution, video data (including compressed data and YUV video) will be written into DDR first, and then read out of DDR by the network module, which requires a large amount of system bus and memory bandwidth. Therefore, the video data generated by the local end cannot be written into DDR in time or EMAC cannot read video data from DDR in time, which also causes the video of the local end to be unable to be transmitted to the remote end for display in time, resulting in a large delay in the video display at the remote end, especially for the remote end to see the local video information in time (such as the baseboard management control chip is in management mode, that is, the remote end user modifies the local server system through the network, and the remote end user wants to quickly know the result after the modification), the traditional solution has huge drawbacks, the remote end display delay is large, and the human-computer interaction experience is poor. The video data transmission method provided by the present invention can solve the above technical problems.

[0099] Figure 2 A flowchart of a video data transmission method provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the method includes:

[0100] S11: recording first moment data written into the double data rate synchronous dynamic random access memory with a preset number of consecutive frame data of the current frame and second moment data read from the double data rate synchronous dynamic random access memory within a preset time;

[0101] S12: performing difference processing on each second moment data and each first moment data to obtain corresponding first delayed moment data; and performing average processing on each first delayed moment data to obtain second delayed moment data;

[0102] S13: determining a transmission mode of the frame data to be transmitted currently according to the relationship between the second delay time data and the preset value; and transmitting the frame data to be transmitted currently to the remote end according to the transmission mode;

[0103] Among them, the transmission mode includes a normal mode and a low-latency mode; in the low-latency mode, the local end skips the storage process of the double data rate synchronous dynamic random access memory and directly transmits it to the remote end.

[0104] Specifically, the continuous preset number of frame data in step S11 is a continuous preset number of frames reversed from the current frame, and the preset time is not specifically limited and can be set according to actual conditions. In addition, the corresponding continuous preset number of frames can change the number of frames according to actual conditions, such as the number of times the low-latency mode is entered is less than the preset number, for example, the low-latency mode has not been entered, resulting in the delay process always existing, and the continuous preset data amount frames can be reduced. Or if the calculation time of the data at the first delay moment based on the data is long, the continuous preset data amount frames can also be reduced, which is not limited here.

[0105] The separate recording process in step S11 is based on the complete writing of each frame data of a continuous preset number of frame data and the complete writing of the corresponding first moment data and second moment data. It can be understood that the number of first moment data and second moment data corresponding to each other is a preset number.

[0106] For the recording process of the first moment data and the second moment data, the frame header and frame tail of the frame data can be used as the calculation standard. The frame header and frame tail here need to be added in advance. The frame header is added before the current frame is transmitted and written to the DDR. During transmission, the frame tail is added during the transmission process through the DDR bus to facilitate the moment when the first moment data is recorded and written to the DDR, and the second moment data is recorded when the transmission leaves the local end and the moment when the frame data is read back from the DDR. The second moment data of the frame data is differenced with the first moment data to obtain the first delayed moment data of the frame data. Since the first delayed moment data corresponding to a preset number of consecutive frame data are all processed, the second delayed moment data is obtained by averaging at this time. It should be noted that the second delayed moment data can also be obtained through standard processing.

[0107] The transmission mode of the frame data to be transmitted is determined according to the relationship between the second delay time data and the preset value. It can be understood that the preset value can be a default value or a value dynamically modified by the user through register configuration, which is not limited here. If the second delay time data is greater than the preset value, it means that there may be a delay at present, and it is necessary to enter the delay mode for transmission. The delay mode here skips the write and read operations of DDR and only transmits directly to the remote end through the local end.

[0108] Figure 3 A schematic diagram of a server management control chip video function provided by an embodiment of the present invention, such as Figure 3 As shown, the data is directly transmitted to the network module through the output control module, without having to go through the writing storage and re-reading operation process of the DDR space B. Figure 3 The storage space A in Figure 1 The storage of space A in is the same and will not be repeated here.

[0109] The data transmission corresponding to the normal mode is through Figure 1 That is to say, the corresponding transmission mode is determined based on the relationship between the current corresponding second delay moment data and the preset value. It is not only the normal mode or the low delay mode, but also the switching of the two transmission modes can be determined based on the relationship between the second delay moment data corresponding to different moments and the preset value. This embodiment can realize data transmission by dynamically switching between the two transmission modes, avoiding the delay caused by transmission only in the normal mode.

[0110] In low-latency mode, the display quality of the remote end can be greatly changed. However, because in low-latency mode, the network module is only used for video functions and cannot transmit other data to the remote end, dynamic switching between normal mode and low-latency mode is supported.

[0111] The present invention provides a method for transmitting video data. By configuring a control system at a local end, a preset number of frame data is continuously written into a DDR, and the time taken to completely read back the preset number of frame data from the DDR is recorded, that is, the first moment data corresponding to the writing of the DDR is recorded for each frame data of the preset number, and the second moment data used to completely read back the frame data from the DDR is recorded. The first delay moment data of each frame data can be determined based on the first moment data and the second moment data, and the second delay moment data is obtained by performing mean processing based on the first delay moment data of the preset number of consecutive frames. The corresponding transmission mode is determined according to the relationship between the second delay moment data and the preset value. The corresponding transmission mode is determined according to the relationship between the second delay moment data and the preset value, so that the control system at the local end enters the low-delay mode for transmission, and the frame data currently to be transmitted skips the writing and reading process of the DDR, and is directly transmitted from the local end to the remote end, thereby reducing the transmission delay of the video data and improving the transmission efficiency. At the same time, by switching different transmission modes, the transmission mode of the video data of the SoC chip is diversified and flexible, and the occupation of the system bus and memory bandwidth by the video function is reduced without affecting the normal function of the chip, thereby improving the overall performance of the chip, and improving the display quality and human-computer interaction experience of the remote end.

[0112] In some embodiments, before writing a preset number of consecutive frame data into the double data rate synchronous dynamic random access memory, the frame header attribute information is pre-added to the frame header of each frame data;

[0113] Before a preset number of consecutive frame data are read out from the double data rate synchronous dynamic random access memory, frame tail attribute information is pre-added to the frame tail of each frame data.

[0114] Specifically, the frame header attribute information and the frame tail attribute information in this embodiment are not added to the frame header and frame tail of a preset number of consecutive frame data, but are added based on the frame header and frame tail of the frame data of each frame, so that the frame data of each frame constitutes a complete data, so as to facilitate the recording of time data when writing DDR and reading DDR.

[0115] In some embodiments, both the frame header attribute information and the frame tail attribute information include a frame sequence number, and the frame header attribute information and the frame tail attribute information of the same frame data have the same frame sequence number.

[0116] It is understandable that both the frame header attribute information and the frame tail attribute information include a frame sequence number, that is, the frame number of the corresponding continuous preset number of frame data, so as to facilitate averaging processing in the subsequent statistical process. For statistical convenience, the frame header attribute information and the frame tail attribute information of the same frame data have the same frame sequence number, so as to fully record the moment data of writing and reading a frame of data.

[0117] It should be noted that the frame number setting of the preset data amount in this embodiment can be set according to the bit width setting of the frame sequence number record. For example, the frame number occupies a bit width of 8 bits, and its corresponding frame number bit is 2 to the sixth power, that is, 0-255. In order to record the number of frames of continuous preset data amount, it can be less than or equal to 256 data, so as to facilitate the calculation based on the frame sequence number during statistics. If it exceeds 256 frames, the corresponding statistics may be confused, and statistical errors caused by repeated frame numbers may occur.

[0118] In some embodiments, the bit width data of the frame header attribute information and the frame tail attribute information of each frame data are the same.

[0119] It can be understood that the bit width data corresponding to the frame header attribute information and the frame tail attribute information in this embodiment are the same, so that the subsequent recording time data is accurate. If the bit widths are different, the first moment data written to the DDR and the second moment data read from the DDR in the corresponding data transmission record will have deviations. As a result, because the bit width data corresponding to the frame header and the frame tail are different, there will be deviations in the delayed moment data, which will in turn lead to deviations in the judgment of the transmission mode, affecting the data transmission efficiency.

[0120] For example, at the beginning of a frame, the predetermined indication information and the serial number of the current frame are added, totaling 32 bits, {frame_header+frame_num}, frame_header occupies 24 bits and can be customized by the user, such as 24'h fafafa, frame_num occupies 8 bits, which is the serial number of the current frame, 0-255 loop. At the end of the frame, the predetermined indication information and the serial number of the current frame are added, totaling 32 bits, {frame_end+frame_num}, frame_end occupies 24 bits and can be customized by the user, such as 24'h 5F5F5F, frame_num occupies 8 bits, which is the serial number of the current frame, 0-255 loop.

[0121] The frame header attribute information and frame tail attribute information of each frame data provided in this embodiment have the same bit width data, which ensures the accuracy of the data at the time of data recording at the source and ensures the accuracy of the data transmission mode.

[0122] In some embodiments, the frame header attribute information is pre-added to the frame header of each frame data, including:

[0123] Before the system-level chip outputs the current frame data, the frame header attribute information is filled into the advanced expansion interface bus transmitted between the system-level chip and the double data rate synchronous dynamic random access memory;

[0124] Correspondingly, the frame tail attribute information is pre-added to the frame tail of each frame data, including:

[0125] When the filled current frame data is outputted by the system-level chip, the frame tail attribute information is filled into the advanced expansion interface bus transmitted between the system-level chip and the double data rate synchronous dynamic random access memory so as to be written into the double data rate synchronous dynamic random access memory.

[0126] Specifically, before outputting the first data of the current frame, {frame_header+frame_num} is filled into the AXI Master bus and written into the DDR. After outputting the last data of the current frame, {frame_header+frame_num} is filled into the AXI Master bus and written into the DDR.

[0127] like Figure 3 As shown, the channel detection from the output control module to the delay prediction module Figure 3 The AXI Master interface of the output control module in the MCU starts writing the first moment data T1 of the DDR frame, and at the same time, the channel detection from the network module to the delay prediction module Figure 3The AXI Master interface reads back the second moment data T2 of the frame data completely after reading from DDR. The frame data refers to the same as frame_num. Continuously detect N frames (N can use the default value, such as 5 frames, which can be flexibly configured by the user register), calculate T2-T1 of N frames of video, and get the average value of N frame delay {(T2_0-T1_0)+(T2_1-T1_1)+(T2_2-T1_2)+...(T2_N-1-T1_N-1)} / N. If the average value of N frame delay is greater than the preset value (DELAY_TIME_STANDARD_0, which can use the default value or be configured by the user through registers and dynamically modified), then configure Figure 3 The configuration module in enables the video processing subsystem to enter low-latency mode.

[0128] The recording process based on the frame header and the frame tail provided in this embodiment obtains the statistical process of the first moment data and the second moment data, thereby ensuring the accuracy of data recording at the source and improving the accuracy of the subsequent data transmission mode.

[0129] In some embodiments, determining the transmission mode of the frame data to be transmitted currently according to the relationship between the second delay time data and the preset value includes:

[0130] When the data at the second delay time exceeds the preset value and the current mode is the normal mode, the normal mode is switched to the low-delay mode;

[0131] When the data at the second delay time does not exceed the preset value and the current mode is the normal mode, maintain the normal mode;

[0132] When the data at the second delay moment exceeds the preset value and the current mode is the low delay mode, maintaining the low delay mode;

[0133] When the data at the second delay moment does not exceed the preset value and the current mode is the low-latency mode, the low-latency mode is exited and switched to the normal mode.

[0134] Specifically, the transmission mode includes a normal mode and a low-latency mode. In the corresponding current transmission process, there must be a transmission mode. Therefore, this embodiment is set based on different situations. When the data at the second delay moment exceeds the preset value and the current mode is the normal mode, the normal mode will be switched to the low-latency mode. If the mode is the low-latency mode at this time, the current low-latency mode can be maintained.

[0135] If the data at the second delay time does not exceed the preset value and the current mode is the normal mode, there is no need to switch, and the normal mode can be maintained. If the current mode is the low-latency mode, it means that the data has been improved in the low-latency mode, and there is no need to enter the low-latency mode. You can exit the low-latency mode and switch to the normal mode.

[0136] The flexible switching process between the two transmission modes provided in this embodiment reduces the occupation of the system bus and memory bandwidth by the video function without affecting the normal function of the chip, improves the overall performance of the chip, and improves the display quality and human-computer interaction experience of the remote end.

[0137] In some embodiments, after determining the transmission mode of the frame data to be transmitted according to the relationship between the second delay time data and the preset value, and before transmitting the frame data to be transmitted to the remote end according to the transmission mode, the method further includes:

[0138] Configure a corresponding first register value according to the determined transmission mode;

[0139] The corresponding switching instruction is started according to the corresponding first register value so as to switch to the determined transmission mode to transmit the frame data currently to be transmitted.

[0140] Specifically, the corresponding first register value is configured according to the determined transmission mode, such as Figure 3 The configuration sent by the configuration module shown, the corresponding first register value model_sel, such as: ode_sel=0, indicates that the video processing subsystem enters the normal mode. When mode_sel=1, it indicates that the video processing subsystem enters the low-latency mode, that is, the corresponding switching instruction is turned on according to the corresponding first register value, so as to switch to the determined transmission mode and perform subsequent transmission of the current frame data to be transmitted.

[0141] The present embodiment provides a switching process of different output modes by converting the switching instruction of the hardware flag bit of the first register value, thereby ensuring that data is transmitted in the current correct transmission mode and improving data transmission efficiency.

[0142] In some embodiments, transmitting the frame data currently to be transmitted to the remote end according to the low-latency mode includes:

[0143] Setting the write address information output by the master device of the advanced expansion interface bus transmitted between the system-on-chip and the double data rate synchronous dynamic random access memory as the receiving first-in-first-out address of the driver of the embedded media access controller;

[0144] The frame data to be transmitted currently is transmitted to the remote end according to the burst transmission mode and the receiving first-in-first-out address of the driver of the embedded media access controller.

[0145] Specifically, Figure 3 As shown, the output control module performs functions at night, improves the control logic of the output end, and in the low-latency mode of the video processing system, the write address information output by the master device of the advanced extension interface bus transmitted through the system-level chip and the double data rate synchronous dynamic random access memory is set to the receive first-in-first-out address of the driver of the embedded media access controller, that is, the write address information issued by the AXI Master of the output control module is the FIFO address inside the EMAC, and the burst transmission mode (burst) is also used to send data, that is, the burst transmission mode is used to transmit the data to the FIFO address inside the EMAC corresponding to the remote end.

[0146] The embodiment provides a burst transmission method for transmitting data, thereby improving transmission efficiency and ensuring the integrity of data transmission.

[0147] In some embodiments, the frame data to be transmitted is transmitted to the remote end according to the burst transmission mode and the receiving first-in-first-out address of the driver of the embedded media access controller, including:

[0148] Obtaining the remaining capacity of the first-in-first-out buffer and the output data bit width of the system-level chip corresponding to the first-in-first-out address of the driver of the embedded media access controller;

[0149] Determine the burst length of the frame data to be transmitted currently according to the remaining capacity of the first-in-first-out buffer and the output data bit width;

[0150] The frame data to be transmitted currently is transmitted to the remote end according to the burst length and the receiving first-in-first-out address of the driver of the embedded media access controller.

[0151] Specifically, the remaining capacity of the FIFO corresponding to the FIFO address of the EMAC driver and the output data bit width of the SoC chip are obtained, and the burst length of the frame data to be transmitted is determined based on the two data. In other words, the burst length can be set dynamically to achieve the fastest transmission efficiency. The indication signal avaible_buf of the remaining capacity of the FIFO buffer here is used to dynamically set the burst length according to the value. For example, avaible_buf = 2048 bytes, and the output data bit width of the output end is 128, then the burst length is set to 2048*8 / 128=128.

[0152] The burst length determination process provided in this embodiment improves data transmission efficiency by balancing the remaining capacity of the FIFO and the output data bit width.

[0153] In some embodiments, transmitting the frame data currently to be transmitted to the remote end according to the low-latency mode includes:

[0154] When the first register value is the register value corresponding to the low latency mode, a first interface conversion instruction is triggered; according to the first interface conversion instruction, the master device of the advanced extension interface bus is connected to the slave device of the advanced extension interface bus, and an indication signal of the remaining capacity of the first-in-first-out buffer corresponding to the first-in-first-out address of the driver of the embedded media access controller is received to facilitate direct connection to the remote end.

[0155] Specifically, the first register value in the low-latency mode triggers the first interface conversion instruction, and based on the instruction, the AXI Master of the output control module is connected to the AXI Slave interface of the network module, and the indication signal avaible_buf of the remaining capacity of the first-in-first-out buffer corresponding to the first-in-first-out address of the EMAC driver is received, and connected to the output control module.

[0156] The port connection control and conversion process in the low-latency mode provided in this embodiment, in conjunction with the trigger instruction of the first register value, ensures data transmission in the low-latency mode.

[0157] In some embodiments, after receiving the indication signal of the remaining capacity of the first-in-first-out buffer corresponding to the first-in-first-out address of the driver of the embedded media access controller, the method further includes:

[0158] Receive the frame data to be transmitted from the slave device via the advanced expansion interface bus;

[0159] The frame data to be transmitted is processed according to the Ethernet packet grouping method and sent to the remote end.

[0160] Specifically, the video data is received in the form of an AXI Slave interface, and Ethernet packet processing is performed and sent to the remote end. The Ethernet packet processing process is not limited here and can be the same as or different from the conventional packet processing method.

[0161] The Ethernet packet processing provided in this embodiment performs Ethernet transmission to ensure efficient data transmission.

[0162] In some embodiments, transmitting the frame data currently to be transmitted to the remote end according to the conventional mode includes:

[0163] When the first register value is a register value corresponding to the normal mode, a second interface conversion instruction is triggered; according to the second interface conversion instruction, the master device of the advanced extension interface bus is connected to the system bus of the system-level chip, so as to output the current video data to be transmitted to the double data rate synchronous dynamic random access memory and transmit it to the remote end;

[0164] The slave device of the advanced expansion interface bus is turned off so as not to receive the video data currently to be transmitted.

[0165] Specifically, mode_sel=0, that is, in normal mode, the submodule connects the AXI Master interface of the output control module to the internal system bus of the chip, so that the video data is output to DDR, and closes the AXI Slave port of the network module, so that the network module does not directly obtain data from the output control module.

[0166] The data transmission in the normal mode provided by this embodiment is controlled by the connection conversion of the port, thereby ensuring the flexibility and diversity of the interface conversion and facilitating the data transmission.

[0167] In some embodiments, before transmitting the frame data to be transmitted to the remote end according to the low-latency mode, the method further includes:

[0168] When the first register value is a register value corresponding to the low-latency mode, controlling the embedded media access controller to receive the frame data currently to be transmitted and the corresponding data capacity;

[0169] Obtaining the remaining space capacity of a receiving first-in-first-out buffer in the embedded media access controller;

[0170] Determine the remaining space capacity of the current receiving FIFO buffer according to the relationship between the remaining space capacity of the receiving FIFO buffer in the embedded media access controller and the data capacity of the frame data to be currently transmitted;

[0171] If the remaining space capacity of the current receiving FIFO buffer exceeds the preset range, the frame data to be transmitted is transmitted.

[0172] Specifically, based on the first register value, the relationship between the remaining space capacity of the FIFO buffer of the control EMAC and the data capacity of the frame data to be transmitted currently is used to determine the remaining space capacity of the current receiving FIFO buffer. When the remaining space capacity of the current receiving FIFO buffer exceeds a preset range, the frame data to be transmitted currently is transmitted.

[0173] Figure 4 A logical schematic diagram of a network mode provided by an embodiment of the present invention, such as Figure 4 As shown, an AXI Slave port of the advanced expansion interface bus and a first-in-first-out margin statistics module (FIFO margin statistics module) are added. The FIFO margin statistics module is used to count the remaining space capacity of the receiving first-in-first-out buffer in the embedded media access controller, and transmit the remaining space of the receiving FIFO to the output control module through indication information.

[0174] Figure 5 A flowchart of a network module processing data to be sent provided by an embodiment of the present invention, such as Figure 5 As shown, including:

[0175] S21: The output control module switches the output interface;

[0176] S22: The network module switches the data acquisition interface and changes the acquisition mode;

[0177] S23: The output control module sends data in the form of burst transmission;

[0178] S24: The network module changes the framing process;

[0179] S25: The network module sends data.

[0180] The present embodiment provides a method for triggering the remaining space statistics of the receiving FIFO through the first register value in the low-latency mode, so as to take into account the two processes of receiving by the network module and sending by the output control module, thereby ensuring that data transmission is congested and that there is enough remaining space capacity in the network module for storage.

[0181] In some embodiments, before transmitting the frame data to be transmitted to the remote end according to the conventional mode, the method further includes:

[0182] When the first register value is a register value corresponding to the normal mode, the embedded media access controller is controlled to use the master device and descriptor mechanism of the advanced expansion interface bus to obtain the current frame data to be transmitted from the double data rate synchronous dynamic random access memory for transmission to the remote end.

[0183] Specifically, when mode_sel=0, the network module obtains video data from DDR using the AXI Master interface and descriptor mechanism, which is the function of the network module in the traditional solution. Figure 6 A flowchart of EMAC data transmission in a normal mode provided by an embodiment of the present invention is as follows: Figure 6 As shown, the execution subject is the driver of the embedded media access controller, including:

[0184] S31: reading the data to be sent from the double data rate synchronous dynamic random access memory;

[0185] S32: moving the data to be sent to a designated double data rate synchronous dynamic random access memory address;

[0186] S33: Set the send / receive descriptor and send it to the double data rate synchronous dynamic random access memory;

[0187] S34: read the send descriptor and parse the descriptor;

[0188] S35: Read the data to be sent from the designated address in the double data rate synchronous dynamic random access memory;

[0189] S36: Packaging the read data and sending it to the remote end;

[0190] S37: Receive a data packet, and write the received data into a designated address of the double data rate synchronous dynamic random access memory according to the receive descriptor.

[0191] The data transmission method in the normal mode provided in this embodiment ensures normal reading and transmission of data through a descriptor mechanism.

[0192] In some embodiments, the process of establishing a receive FIFO buffer in an embedded media access controller includes:

[0193] When the first register value is a register value corresponding to the low latency mode, writing the write data of the slave device of the advanced expansion interface bus into the receiving first-in-first-out buffer in the embedded media access controller;

[0194] When the first register value is a register value corresponding to the normal mode, the read return data of the master device of the advanced expansion interface bus is written into the receiving first-in-first-out buffer in the embedded media access controller.

[0195] Specifically, in Figure 6 The receive FIFO control module in the AXI Master writes the read return data into the receive FIFO when mode_sel=0, and writes the write data in the AXI Slave into the receive FIFO when mode_sel=1.

[0196] The internal logic of the network module and the establishment process of the FIFO buffer of the EMAC provided in this embodiment ensure that data can be normally written into the receiving FIFO buffer regardless of the transmission mode.

[0197] In some embodiments, after transmitting the frame data to be transmitted to the remote end according to the transmission mode, the method further includes:

[0198] Receiving request information corresponding to the display status of video data at the remote end;

[0199] Configure the corresponding video function register value according to the request information;

[0200] Correspondingly, the corresponding video function register value is configured according to the request information, including:

[0201] When the video function is normal, configuring the first video function normal register value according to the request information of the video function being normal;

[0202] In case of video function abnormality, the corresponding second video function abnormality register value is configured according to the request information of the video function abnormality.

[0203] Specifically, in normal mode, the software driver of the video function needs to read the video data from the DDR cache, while in low latency mode, the software driver of the video function no longer needs to read the video data from the DDR cache. At the same time, the input end of the output control module and the corresponding control logic are improved ( Figure 3 The compressed video data path in the video processing path, the YUV mode video data path, and the video path output by the output control module in the RGB mode) are configured. Due to the large number of sub-modules in the video processing path and the complex video application scenarios, video function abnormalities may occur occasionally. At this time, the remote end user perceives the video function abnormality (such as video distorted screen, image shift, or even black screen, etc.). Figure 3 In the "Configuration Module", set the register port_sel. When the video function is abnormal, set port_sel=1, and when the video function is normal, set port_sel=0.

[0204] The corresponding video function register value is configured according to the request information corresponding to the display status of the video data at the remote end. When the video function is normal, the first video function normal register value is configured according to the request information of the normal video function; when the video function is abnormal, the corresponding second video function abnormal register value is configured according to the request information of the abnormal video function.

[0205] In some embodiments, after outputting the second video function abnormality register value, the method further includes:

[0206] Obtain the current frame data to be transmitted and the corresponding line and field synchronization signal;

[0207] Calculate the video resolution information corresponding to the frame data to be transmitted according to the line field synchronization signal;

[0208] transmitting the frame data to be transmitted currently to the double data rate synchronous dynamic random access memory according to the video resolution information;

[0209] If frame loss occurs when the frame data to be currently transmitted is transmitted to the double data rate synchronous dynamic random access memory, the frame data to be currently transmitted is discarded according to the video resolution information of the frame data to be currently transmitted.

[0210] Specifically, Figure 3As shown in the figure, when port_sel=1, the output control module selects the channel with DDR as the data input end of the module. The channel transmits the original RGB data output by VGA and the corresponding line field synchronization signal. Therefore, the control logic for calculating the current video resolution information based on the line field synchronization signal is added in the output control module. There is a frame drop control logic in the output control module. The frame drop needs to be performed as a complete frame, so it is necessary to know the data volume of one frame of the current video. When the video function is normal, the resolution information is transmitted through the previous module (color space conversion or core compression module), but the VGA output port is a DVI interface, which does not contain resolution information, so the resolution needs to be calculated inside the output control module.

[0211] The present embodiment provides that by knowing the video resolution information, the data length of the frame data to be transmitted can be determined, and the frame data corresponding to the data length needs to be discarded to ensure that the transmission of other frame data is not affected during the data transmission process, thereby improving the robustness of the video function.

[0212] In some embodiments, after transmitting the frame data to be transmitted to the remote end according to the transmission mode, the method further includes:

[0213] Receive feedback data sent by the remote end;

[0214] The frame data of the next frame is obtained according to the flag information of the feedback data; and the frame data of the next frame is transmitted back to the step of recording the first moment data written into the double data rate synchronous dynamic random access memory with the continuous preset number of frame data of the current frame and the second moment data read from the double data rate synchronous dynamic random access memory within the preset time.

[0215] It can be understood that the present embodiment determines the corresponding transmission mode based on the data of each frame, so as to improve the transmission efficiency of each frame of data. Therefore, after the current frame of data is sent to the remote end, the feedback data sent by the remote end is received, and based on the symbolic information of the feedback data, it is determined that the data of the current frame has been transmitted to the remote end, and the frame data of the next frame can be sent, so it is necessary to return to step S11 to continue the judgment.

[0216] The transmission based on multiple frames of data provided in this embodiment forms a cyclic judgment transmission, thereby ensuring the automation and high efficiency of data transmission.

[0217] The above describes in detail various embodiments corresponding to the video data transmission method. On this basis, the present invention also discloses a video data transmission device corresponding to the above method. Figure 7 FIG. 1 is a structural diagram of a video data transmission device provided by an embodiment of the present invention. Figure 7 As shown, the video data transmission device includes:

[0218] The recording module 11 is used to record the first moment data written into the double data rate synchronous dynamic random access memory with a continuous preset number of frame data of the current frame and the second moment data read from the double data rate synchronous dynamic random access memory within a preset time;

[0219] The processing module 12 is used to perform difference processing on each second moment data and each first moment data to obtain corresponding first delayed moment data; and perform average processing on each first delayed moment data to obtain second delayed moment data;

[0220] The transmission module 13 is used to determine the transmission mode of the frame data to be transmitted according to the relationship between the second delay time data and the preset value; and transmit the frame data to be transmitted to the remote end according to the transmission mode; wherein the transmission mode includes a normal mode and a low-delay mode; in the low-delay mode, the local end skips the storage process of the double data rate synchronous dynamic random access memory and directly transmits it to the remote end.

[0221] Since the embodiments of the device part correspond to the above embodiments, please refer to the description of the embodiments of the method part for the embodiments of the device part, and will not be repeated here.

[0222] For an introduction to a video data transmission method provided by the present invention, please refer to the above method embodiment, which will not be described in detail herein. The method has the same beneficial effects as the above video data transmission method.

[0223] Figure 8 A structural diagram of a video data transmission device provided by an embodiment of the present invention, such as Figure 8 As shown, the device comprises:

[0224] A memory 21, used for storing computer programs;

[0225] The processor 22 is used to implement the steps of the video data transmission method when executing the computer program.

[0226] Among them, the processor 22 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 22 may be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 22 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 22 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0227] The memory 21 may include one or more non-volatile storage media, which may be non-transitory. The memory 21 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 21 is at least used to store the following computer program 211, wherein, after the computer program is loaded and executed by the processor 22, it can implement the relevant steps of the video data transmission method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include but is not limited to data involved in the video data transmission method, etc.

[0228] In some embodiments, the video data transmission device may further include a display screen 23 , an input / output interface 24 , a communication interface 25 , a power supply 26 , and a communication bus 27 .

[0229] Those skilled in the art can understand that Figure 8 The structure shown in the figure does not constitute a limitation on the video data transmission device, and may include more or less components than those shown in the figure.

[0230] The processor 22 implements the video data transmission method provided by any of the above embodiments by calling the instructions stored in the memory 21.

[0231] For an introduction to a video data transmission device provided by the present invention, please refer to the above method embodiment, and the present invention will not be repeated here. It has the same beneficial effects as the above video data transmission method.

[0232] Furthermore, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by the processor 22, the steps of the above-mentioned video data transmission method are implemented.

[0233] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0234] For an introduction to a non-volatile storage medium provided by the present invention, please refer to the above method embodiment, which will not be described in detail herein. The non-volatile storage medium has the same beneficial effects as the above video data transmission method.

[0235] Furthermore, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the video data transmission method when executed by a processor.

[0236] For an introduction to a computer program product provided by the present invention, please refer to the above method embodiment, which will not be described in detail herein. The computer program product has the same beneficial effects as the above video data transmission method.

[0237] The above is a detailed introduction to a video data transmission method, device, equipment, medium and product provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

[0238] It should also be noted that, in this specification, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A method for transmitting video data, characterized in that: include: Recording first moment data written into a double data rate synchronous dynamic random access memory with a preset number of consecutive frame data of a current frame and second moment data read from the double data rate synchronous dynamic random access memory within a preset time; Performing difference processing on each of the second moment data and each of the first moment data to obtain corresponding first delayed moment data; and averaging the first delay time data to obtain the second delay time data; Determine the transmission mode of the frame data to be transmitted according to the relationship between the second delay moment data and the preset value; and transmit the frame data to be transmitted to the remote end according to the transmission mode; wherein the transmission mode includes a normal mode and a low-delay mode; in the low-delay mode, the local end skips the storage process of the double data rate synchronous dynamic random access memory and directly transmits it to the remote end; if the second delay moment data is greater than the preset value, enter the low-delay mode.

2. The video data transmission method according to claim 1, characterized in that: Before writing a preset number of consecutive frame data into the double data rate synchronous dynamic random access memory, adding frame header attribute information to the frame header of each frame data in advance; Before a preset number of consecutive frame data are read out from the double data rate synchronous dynamic random access memory, frame tail attribute information is pre-added to the frame tail of each frame data.

3. The video data transmission method according to claim 2, characterized in that: The frame header attribute information and the frame tail attribute information both include a frame sequence number, and the frame header attribute information and the frame tail attribute information of the same frame data have the same frame sequence number.

4. The video data transmission method according to claim 3, characterized in that: The bit width data of the frame header attribute information and the frame tail attribute information of each frame data are the same.

5. The video data transmission method according to any one of claims 2 to 4, characterized in that: The frame header attribute information is added to the frame header of each frame data in advance, including: Before the system-level chip outputs the current frame data, the frame header attribute information is filled into the advanced expansion interface bus transmitted between the system-level chip and the double data rate synchronous dynamic random access memory; Correspondingly, the frame tail attribute information is pre-added to the frame tail of each frame data, including: When the filled current frame data is outputted by the system-level chip, the frame tail attribute information is filled into the advanced expansion interface bus transmitted between the system-level chip and the double data rate synchronous dynamic random access memory so as to be written into the double data rate synchronous dynamic random access memory.

6. The video data transmission method according to claim 1, characterized in that: Determining a transmission mode of the frame data to be transmitted according to a relationship between the second delay time data and a preset value includes: When the second delay time data exceeds the preset value and the current mode is the normal mode, the normal mode is switched to the low delay mode; When the second delay time data does not exceed the preset value and the current mode is the normal mode, maintaining the normal mode; When the data at the second delay time exceeds the preset value and the current mode is the low delay mode, maintaining the low delay mode; When the data at the second delay moment does not exceed the preset value and the current mode is the low-delay mode, exit the low-delay mode and switch to the normal mode.

7. The video data transmission method according to claim 6, characterized in that: After determining the transmission mode of the frame data to be transmitted according to the relationship between the second delay time data and the preset value, and before transmitting the frame data to be transmitted to the remote end according to the transmission mode, the method further includes: Configure a corresponding first register value according to the determined transmission mode; The corresponding switching instruction is started according to the corresponding first register value so as to switch to the determined transmission mode to transmit the frame data currently to be transmitted.

8. The video data transmission method according to claim 7, characterized in that: Transmitting the frame data to be transmitted to the remote end according to the low-delay mode includes: Setting the write address information output by the master device of the advanced expansion interface bus transmitted between the system-level chip and the double data rate synchronous dynamic random access memory as the receiving first-in-first-out address of the driver of the embedded media access controller; The frame data to be transmitted currently is transmitted to the remote end according to the burst transmission mode and the receiving first-in-first-out address of the driver of the embedded media access controller.

9. The video data transmission method according to claim 8, characterized in that: The method transmits the frame data to be transmitted to the remote end according to the burst transmission mode and the receiving first-in-first-out address of the driver of the embedded media access controller, including: Acquire the remaining capacity of the first-in-first-out buffer corresponding to the first-in-first-out address of the driver of the embedded media access controller and the output data bit width of the system-level chip; Determining the burst length of the frame data to be transmitted currently according to the remaining capacity of the FIFO buffer and the output data bit width; The frame data to be transmitted currently is transmitted to the remote end according to the burst length and a receiving first-in-first-out address of a driver of the embedded media access controller.

10. The video data transmission method according to claim 7, characterized in that: Transmitting the frame data to be transmitted to the remote end according to the low-delay mode includes: When the first register value is the register value corresponding to the low latency mode, a first interface conversion instruction is triggered; according to the first interface conversion instruction, a master device of the advanced extension interface bus is connected to a slave device of the advanced extension interface bus, and an indication signal of the remaining capacity of the first-in-first-out buffer corresponding to the first-in-first-out address of the driver of the embedded media access controller is received to facilitate direct connection to the remote end.

11. The video data transmission method according to claim 10, characterized in that: After receiving the indication signal of the remaining capacity of the first-in-first-out buffer corresponding to the first-in-first-out address of the driver of the embedded media access controller, the method further includes: Receiving the frame data to be transmitted currently through the slave device of the advanced expansion interface bus; The frame data to be transmitted is processed according to an Ethernet packetization method to send the data to the remote end.

12. The video data transmission method according to claim 7, characterized in that: Transmitting the frame data currently to be transmitted to the remote end according to the conventional mode includes: When the first register value is a register value corresponding to the normal mode, a second interface conversion instruction is triggered; according to the second interface conversion instruction, a master device of the advanced extension interface bus is connected to a system bus of the system-level chip, so as to output the video data currently to be transmitted to the double data rate synchronous dynamic random access memory, and transmit it to the remote end; The slave device of the advanced expansion interface bus is turned off so as not to receive the video data currently to be transmitted.

13. The video data transmission method according to claim 10, characterized in that: Before transmitting the frame data to be transmitted to the remote end according to the low-delay mode, the method further includes: When the first register value is a register value corresponding to the low-latency mode, controlling the embedded media access controller to receive the frame data currently to be transmitted and the corresponding data capacity; Obtaining the remaining space capacity of a receiving first-in-first-out buffer in the embedded media access controller; Determine the remaining space capacity of the current receiving FIFO buffer according to the relationship between the remaining space capacity of the receiving FIFO buffer in the embedded media access controller and the data capacity of the frame data to be currently transmitted; If the remaining space capacity of the current receiving FIFO buffer exceeds a preset range, the frame data to be transmitted is transmitted.

14. The video data transmission method according to claim 12, characterized in that: Before transmitting the frame data to be transmitted to the remote end according to the conventional mode, the method further includes: When the first register value is a register value corresponding to the normal mode, the embedded media access controller is controlled to use the master device and descriptor mechanism of the advanced expansion interface bus to obtain the current frame data to be transmitted from the double data rate synchronous dynamic random access memory for transmission to the remote end.

15. The video data transmission method according to claim 13, characterized in that: The process of establishing the receiving first-in-first-out buffer in the embedded media access controller includes: When the first register value is a register value corresponding to the low latency mode, writing the write data of the slave device of the advanced expansion interface bus into the receiving first-in-first-out buffer in the embedded media access controller; When the first register value is a register value corresponding to the normal mode, the read return data of the master device of the advanced expansion interface bus is written into the receiving first-in-first-out buffer in the embedded media access controller.

16. The video data transmission method according to claim 1, characterized in that: After transmitting the frame data to be transmitted to the remote end according to the transmission mode, the method further includes: Receiving request information corresponding to the display status of the video data at the remote end; Configure the corresponding video function register value according to the request information; Correspondingly, configuring the corresponding video function register value according to the request information includes: When the video function is normal, configuring the first video function normal register value according to the request information of the video function being normal; In case of video function abnormality, the corresponding second video function abnormality register value is configured according to the request information of the video function abnormality.

17. The video data transmission method according to claim 16, characterized in that: After outputting the second video function abnormality register value, the method further includes: Acquire the frame data to be transmitted and the corresponding line and field synchronization signal; Calculating the video resolution information corresponding to the frame data to be transmitted currently according to the line field synchronization signal; transmitting the frame data to be transmitted currently to the double data rate synchronous dynamic random access memory according to the video resolution information; If frame loss occurs when the frame data currently to be transmitted is transmitted to the double data rate synchronous dynamic random access memory, the frame data currently to be transmitted is discarded according to the video resolution information of the frame data currently to be transmitted.

18. The video data transmission method according to claim 1, characterized in that: After transmitting the frame data to be transmitted to the remote end according to the transmission mode, the method further includes: receiving feedback data sent by the remote end; The frame data of the next frame is obtained according to the flag information of the feedback data; and the frame data of the next frame is returned to the step of recording the first moment data written into the double data rate synchronous dynamic random access memory with a continuous preset number of frame data of the current frame within a preset time, and the second moment data read from the double data rate synchronous dynamic random access memory to transmit the frame data of the next frame.

19. A video data transmission device, characterized in that: include: A recording module, used to record first moment data written into a double data rate synchronous dynamic random access memory with a preset number of consecutive frame data of a current frame and second moment data read from the double data rate synchronous dynamic random access memory within a preset time; A processing module, used for performing difference processing on each of the second moment data and each of the first moment data to obtain corresponding first delayed moment data; and averaging the first delay time data to obtain the second delay time data; A transmission module, used for determining a transmission mode of the frame data to be transmitted currently according to the relationship between the second delay moment data and a preset value; and transmitting the frame data to be transmitted currently to a remote end according to the transmission mode; wherein the transmission mode includes a normal mode and a low-delay mode; in the low-delay mode, the local end skips the storage process of the double data rate synchronous dynamic random access memory and directly transmits it to the remote end; if the second delay moment data is greater than the preset value, the low-delay mode is entered.

20. A video data transmission device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the video data transmission method according to any one of claims 1 to 18 when executing the computer program.

21. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video data transmission method according to any one of claims 1 to 18 are implemented.

22. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the video data transmission method according to any one of claims 1 to 18 are implemented.

Images