Video data transmission method and device, equipment and medium

Abstract

The invention discloses a video data transmission method, device, equipment and medium, which are applied to the technical field of video transmission, and the method comprises the steps: detecting a write video memory address of a video graphic array module for a currently transmitted video frame, and caching color image data corresponding to the write video memory address in a data caching unit; reading first pixel data corresponding to the write video memory address from an image frame cache based on the write video memory address, and comparing the first pixel data with second pixel data corresponding to the write video memory address in the data cache unit; if not, updating the bit value of the bit corresponding to the first pixel data in the bitmap; determining a pixel updating proportion of the target frame based on the bitmap; and under the condition that the pixel updating proportion is smaller than a preset proportion threshold value, reading target pixel data corresponding to the updating bit of the bitmap from a video memory, and updating the target pixel data to the image frame cache. Therefore, the video transmission efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of video transmission technology, and in particular to a video data transmission method, device, equipment and medium. Background Art

[0002] The transmission process of the video data stream of the traditional management controller is as follows: first, the management controller receives data to the VGA control module, and the VGA control module writes the video data to the DDR. One way for the compression module to obtain the RGB video source data is to read the video data from the DDR. The compressed data is written to the DDR, and then the compressed data needs to be read from the DDR to transmit the video data to the remote through the network for remote display. During the video transmission process, there are many read and write operations on the DDR, which affects the read and write performance, thereby affecting the efficiency of video data transmission.

[0003] It can be seen that how to improve the efficiency of video data transmission is a problem that technical personnel in this field need to solve. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a video data transmission method, device, equipment and medium, which can improve the efficiency of video transmission. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a video data transmission method, comprising:

[0006] For the currently transmitted video frame, detecting the write memory address of the video graphics array module, and caching the color image data corresponding to the write memory address in the data cache unit;

[0007] Based on the write display memory address, first pixel data corresponding to the write display memory address is read from the image frame buffer, and compared with second pixel data corresponding to the write display memory address in the data cache unit, wherein the image frame buffer caches color image data of a target frame before the currently transmitted video frame arrives, and the target frame is a previous frame of the currently transmitted video frame;

[0008] If the first pixel data is inconsistent with the second pixel data, updating the bit value of the bit corresponding to the first pixel data in the bitmap, wherein each bit in the bitmap corresponds to one first pixel data;

[0009] Determining a pixel update ratio of the target frame based on the bitmap;

[0010] When the pixel update ratio is less than a preset ratio threshold, the target pixel data corresponding to the update bit of the bitmap is read from the video memory, and the target pixel data is updated to the image frame buffer.

[0011] Optionally, also include:

[0012] Reading color image data from the image frame buffer;

[0013] Convert the read color image data into a target transmission format;

[0014] Using a data block module to block the target transmission format data to obtain block data;

[0015] Detecting a video frame header and a video frame tail from the block data output by the data block module, and caching the block data output by the data block module into a block data cache;

[0016] When a video frame header is detected, block data is read from the block data cache and transmitted to the first compression core for data compression processing. When the video frame tail corresponding to the video frame header is detected, the step of reading block data from the block data cache and transmitting it to the first compression core for data compression processing is stopped. When the next video frame header corresponding to the video frame header is detected, block data is read from the block data cache and transmitted to the second compression core for data compression processing. When the video frame tail corresponding to the next video frame header is detected, the step of reading block data from the block data cache and transmitting it to the second compression core for data compression processing is stopped, so as to realize alternating transmission of video frames to the first compression core and the second compression core for data compression processing.

[0017] Optionally, also include:

[0018] storing the compressed data output by the first compression core into a first compressed data cache;

[0019] The compressed data output by the second compression core is stored in a second compressed data cache.

[0020] Optionally, also include:

[0021] Storing the compressed data in the first compressed data cache and the second compressed data cache into the video memory;

[0022] The compressed data is read from the video memory by using an Ethernet card data link layer controller, and sent to a remote end through a network.

[0023] Optionally, storing the compressed data in the first compressed data cache and the second compressed data cache into the video memory includes:

[0024] The compressed data in the first compressed data cache and the second compressed data cache are stored in a target compression area preset in the video memory for storing compressed data.

[0025] Optionally, after converting the format of the read color image data to obtain target transmission format data, the method further includes:

[0026] storing the target transmission format data in a target format area preset in the video memory for storing the target transmission format data, so that the data segmentation module reads the target transmission format data from the target format area and segments the target transmission format data into blocks;

[0027] Or, directly transmit the target transmission format data to the data segmentation module to trigger the step of segmenting the target transmission format data by using the data segmentation module.

[0028] Optionally, also include:

[0029] When the pixel update ratio is greater than or equal to the preset ratio threshold, the color image pixel data of the entire frame is read from the video memory, and the color pixel data of the entire frame is updated to the image frame buffer.

[0030] In a second aspect, the present application provides a video data transmission device, comprising:

[0031] An address detection module, used for detecting a write memory address of a video graphics array module for a currently transmitted video frame;

[0032] A data cache module, used for caching the color image data corresponding to the write display memory address in the data cache unit;

[0033] a pixel comparison module, configured to read first pixel data corresponding to the write display memory address from an image frame buffer based on the write display memory address, and compare the first pixel data with the second pixel data corresponding to the write display memory address in the data cache unit, wherein the image frame buffer caches color image data of a target frame before the arrival of a currently transmitted video frame, and the target frame is a previous frame of the currently transmitted video frame;

[0034] A bitmap updating module, configured to update a bit value of a bit corresponding to the first pixel data in a bitmap if the pixel comparison module determines that the first pixel data is inconsistent with the second pixel data, wherein each bit in the bitmap corresponds to one first pixel data;

[0035] A ratio determination module, used for determining a pixel update ratio of the target frame based on the bitmap;

[0036] The pixel update module is used to read the target pixel data corresponding to the update bit of the bitmap from the video memory when the pixel update ratio is less than a preset ratio threshold, and update the target pixel data to the image frame buffer.

[0037] In a third aspect, the present application provides an electronic device, including:

[0038] Memory for storing computer programs;

[0039] The processor is used to execute the computer program to implement the steps of the aforementioned video data transmission method.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the video data transmission method are implemented.

[0041] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned disclosed video data transmission method.

[0042] It can be seen from the above scheme that the present application provides a video data transmission method, including: for the currently transmitted video frame, detecting the write memory address of the video graphics array module, and caching the color image data corresponding to the write memory address in the data cache unit; reading the first pixel data corresponding to the write memory address from the image frame cache based on the write memory address, and comparing it with the second pixel data corresponding to the write memory address in the data cache unit, before the arrival of the currently transmitted video frame, the image frame cache caches the color image data of the target frame, and the target frame is the previous frame of the currently transmitted video frame; if the first pixel data is inconsistent with the second pixel data, updating the bit value of the bit corresponding to the first pixel data in the bitmap, wherein each bit in the bitmap corresponds to one first pixel data; determining the pixel update ratio of the target frame based on the bitmap; when the pixel update ratio is less than a preset ratio threshold, reading the target pixel data corresponding to the update bit of the bitmap from the video memory, and updating the target pixel data to the image frame cache.

[0043] It can be seen that the present application detects the write memory address of the video graphics array module, and caches the color image data corresponding to the write memory address in the data cache unit, compares the pixel data in the image frame cache and the data cache unit according to the write memory address, and updates the bitmap to mark the changed pixel data. Each bit in the bitmap corresponds to a pixel data. When the pixel update ratio of the target frame in the image frame cache is less than the preset ratio threshold, the target pixel data corresponding to the update bit of the bitmap is read from the video memory, and the target pixel data is updated to the image frame cache. In this way, there is no need to read the entire frame of the image from the video memory, which saves the read bandwidth of the video memory, reduces the impact on the video memory performance, and thus improves the efficiency of video data transmission.

[0044] Correspondingly, a video data transmission device, equipment and medium provided by the present application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of video data transmission for a management controller;

[0047] Figure 2 A flow chart of a video data transmission method provided by an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of a system architecture using a video transmission method provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of a bitmap control module provided in an embodiment of the present application;

[0050] Figure 5 A bitmap update flow chart provided for an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of a compression control module provided in an embodiment of the present application;

[0052] Figure 7 A compression control module processing flow chart provided in an embodiment of the present application;

[0053] Figure 8 A schematic diagram of a cache control module provided in an embodiment of the present application;

[0054] Fig. 9 A schematic diagram of the structure of a video data transmission device provided by an embodiment of the present invention;

[0055] Fig.10 A structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0058] First, the terms involved in this application are explained:

[0059] JPEG: Joint Photographic Experts Group, a compression standard for continuous-tone still images.

[0060] VGA: Video Graphics Array, video graphics array, the most widely used interface type on graphics cards.

[0061] RGB: red green blue represents the colors of the three channels of red, green and blue. The RGB color mode is a color standard in the industry. RGB image data is color image data.

[0062] YUV: It is a color encoding method. "Y" represents brightness (Luminance or Luma), which is the grayscale value, and "U" and "V" represent chrominance (Chrominance or Chroma).

[0063] DDR: Double Data Rate, double data rate synchronous dynamic random access memory, is a type of memory.

[0064] EMAC: The chip of the data link layer in the Ethernet card is generally referred to as the MAC (Media Access Controller) controller, and the chip of the physical layer is referred to as PHY.

[0065] The present invention relates to a management controller, such as a baseboard management controller, for updating RGB data of a video stream and processing JPEG IP compression, aiming to improve the processing rate of the entire process and improve the quality of the compressed resolution that can be supported. Figure 1 As shown, Figure 1The following is a schematic diagram of video data transmission of a management controller. The transmission of video data stream in a traditional baseboard management controller is as follows: First, the baseboard management controller receives data to the VGA control module through the PCIE interface. The VGA control module writes the video data to the DDR. The JPEG compression module obtains RGB video source data in two ways: one is to obtain it directly from the VGA module, and the other is to read the video data from the DDR. After RGB2YUV converts the acquired RGB data into YUV data, the YUV2BLK module provides the YUV data to the JPEG IP (i.e., the JPEG compression core) in the form of BLOCK, and the JPEG IP writes the compressed data to the DDR. The eMAC module reads the compressed data from the DDR and transmits the video data to the remote through the network for remote display. During the above-mentioned video streaming process, as the quality and frame rate of the video display become higher and higher, the JPEG IP used by the current baseboard management controller can no longer meet higher resolutions and frame rates under the current operating frequency and process. Therefore, in order to improve the processing rate on the video data stream, the present invention proposes a method for updating image data by maintaining a bitmap, and increases the JPEG IP to use a ping-pong operation to increase the compression throughput, thereby meeting the frame rate compression display frame rate within one second, thereby reducing the display delay of the remote control.

[0066] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0067] Next, a video data transmission method provided by an embodiment of the present invention is introduced in detail. Figure 2 A flow chart of a video data transmission method provided in an embodiment of the present invention, the video data transmission method comprising:

[0068] Step S11: for the currently transmitted video frame, detecting the write memory address of the video graphics array module, and caching the color image data corresponding to the write memory address in the data cache unit.

[0069] In the embodiment of the present application, the video graphics array module, i.e., the VGA module, can detect the write video memory address of the video graphics array module after detecting the VSYNC (Vertical Synchronization) signal. The VSYNC signal is a key synchronization signal in the video display system, which marks the beginning of a new frame of image. The video graphics array module will write the video data into the video memory. The embodiment of the present application can detect the write data bus, detect the corresponding write memory address, and cache the color image data (i.e., RGB image data) corresponding to the write video memory address in the data cache unit. In the embodiment of the present application, DDR can be used as the video memory.

[0070] Step S12: based on the write display memory address, first pixel data corresponding to the write display memory address is read from the image frame cache, and compared with second pixel data corresponding to the write display memory address in the data cache unit, the image frame cache has color image data of a target frame before the currently transmitted video frame arrives, and the target frame is a previous frame of the currently transmitted video frame.

[0071] The image frame buffer is used to cache a frame of image data. When initialized, that is, when the image frame buffer is empty, data can be taken from the data buffer unit and cached in the image frame buffer in pixel order, so that the first frame of video data is stored in the image frame buffer, and a frame of color image data is also written in pixel order in the video memory. For pixel data that has changed in subsequent frame data, the pixel data at the corresponding position in the video memory is updated. That is, the pixel data at the corresponding position can be overwritten.

[0072] Therefore, based on the write video memory address, the pixel data at the corresponding position in the image frame buffer and the data buffer unit can be determined, that is, the pixel data at the same position in the frame.

[0073] Step S13: if the first pixel data is inconsistent with the second pixel data, updating the bit value of the bit corresponding to the first pixel data in the bitmap, wherein each bit in the bitmap corresponds to one first pixel data.

[0074] It can be understood that if the pixel data is inconsistent, the pixel data at the corresponding position has changed. The embodiment of the present application can use a bitmap to record the change of pixel data, for example, if it has changed, it is 1, otherwise it is 0.

[0075] Step S14: determining a pixel update ratio of the target frame based on the bitmap.

[0076] Since each video frame has a corresponding identification signal, such as a VSYNC signal, the embodiment of the present application can determine the pixel data of a frame of data comparison, so as to determine the pixel update ratio of the target frame corresponding to the currently transmitted video frame. For example, the number of 1s can be counted to determine the ratio to the number of all bits to obtain the pixel update ratio.

[0077] Step S15: when the pixel update ratio is less than a preset ratio threshold, the target pixel data corresponding to the update bit of the bitmap is read from the video memory, and the target pixel data is updated to the image frame buffer.

[0078] The embodiment of the present application can combine the performance of the video memory and experience to determine a preset ratio threshold. When the pixel update ratio is less than the preset ratio threshold, the target pixel data corresponding to the updated bit of the bitmap is read from the video memory instead of the image data of the entire frame, thereby reducing the amount of data read from the video memory.

[0079] Furthermore, when the pixel update ratio is greater than or equal to the preset ratio threshold, the color pixel data of the entire frame is read from the video memory, and the color pixel data of the entire frame is updated to the image frame cache.

[0080] In the embodiment of the present application, one pixel data occupies 4 bytes, and the position of the target pixel data corresponding to the update bit in the video memory can be determined based on the base address of the video memory, specifically the base address corresponding to the area storing the color image data, and the position of the update bit in the image, and the target pixel data is updated to the image frame buffer. It can be understood that the bitmap is marked according to the position of the pixel, 1 bit corresponds to one pixel, and one pixel occupies 4 bytes. The storage space increases one address by one byte. Each bit corresponds to 4 bytes, and each byte increases one address. Based on the bit with a value of 1 in the bitmap, the address of the data change can be calculated, and the changed pixel value can be found in the video memory by adding the base address of the DDR (i.e., the video memory).

[0081] In the implementation of this application, it is possible to reduce the amount of data read from DDR and reduce the write bandwidth of DDR in BUFFER mode, that is, when data needs to be read from DDR. In non-BUFER mode, data can be updated from the data cache to the image frame cache, and the video frame is the image frame. There is only pixel data in the RGB channel data, and the BUFFER mode has not only pixel data but also fast cursor data. Therefore, when using the fast cursor function, it is necessary to use the data originally written to the DDR video memory. Therefore, the BITMAP UPDATE module records the address of the VGA written to DDR when recording, thereby ensuring that the write bandwidth of DDR is reduced in BUFFER mode.

[0082] Furthermore, the embodiment of the present application can read color image data from the image frame buffer; perform format conversion on the read color image data to obtain target transmission format data; use a data blocking module to block the target transmission format data to obtain block data; detect a video frame header and a video frame tail from the block data output by the data blocking module, and cache the block data output by the data blocking module into a block data cache; when a video frame header is detected, the block data is read from the block data cache and transmitted to the first compression core for data compression processing; when the video frame tail corresponding to the video frame header is detected, the step of reading the block data from the block data cache and transmitting it to the first compression core for data compression processing is stopped; when the next video frame header corresponding to the video frame header is detected, the block data is read from the block data cache and transmitted to the second compression core for data compression processing; when the video frame tail corresponding to the next video frame header is detected, the step of reading the block data from the block data cache and transmitting it to the second compression core for data compression processing is stopped, so as to realize the step of alternately transmitting the video frames to the first compression core and the second compression core for data compression processing.

[0083] Among them, the format conversion module can be used to read the color image data from the image frame buffer; the read color image data is format converted to obtain the target transmission format data. The format conversion module, namely the RGB2YUV module, is used to convert the acquired RGB data into YUV data. When the format conversion module reads the color image data from the image frame buffer, it is necessary to control the reading speed, and it is necessary to ensure that the reading speed is slower than the writing speed. If the reading speed exceeds the writing speed, new data will be read, so it is necessary to limit the address when reading, and it cannot exceed the writing address. The data block module, namely the YUV2BLK module, outputs the YUV data in BLOCK mode.

[0084] In an optional embodiment, after the read color image data is format-converted to obtain the target transmission format data, the method further includes: storing the target transmission format data in a target format area preset in the video memory for storing the target transmission format data, so that the data segmentation module reads the target transmission format data from the target format area and segments the target transmission format data; or directly transmitting the target transmission format data to the data segmentation module to trigger the step of segmenting the target transmission format data using the data segmentation module. How to perform can be selected depending on the cache size of the data segmentation module.

[0085] This embodiment can utilize a compression control module to detect a video frame header and a video frame tail from the block data output by the data blocking module, and cache the block data output by the data blocking module into a block data cache; when a video frame header is detected, the block data is read from the block data cache and transmitted to the first compression core for data compression processing; when a video frame tail corresponding to the video frame header is detected, the step of reading the block data from the block data cache and transmitting it to the first compression core for data compression processing is stopped; when a next video frame header corresponding to the video frame header is detected, the block data is read from the block data cache and transmitted to the second compression core for data compression processing; when a video frame tail corresponding to the next video frame header is detected, the step of reading the block data from the block data cache and transmitting it to the second compression core for data compression processing is stopped, so as to realize alternating transmission of video frames to the first compression core and the second compression core for data compression processing.

[0086] The compression control module may include a frame detection module for detecting the header and the tail of a video frame from the block data output by the data segmentation module, a block data cache and a selection control module, and the selection control module is used to transfer the video frames alternately to the first compression core and the second compression core for data compression processing according to the detection result of the frame detection module. The embodiment of the present application can detect the header and tail of the YUV data converted into BLOCK data, so as to determine to separate the image data frame by frame, so as to control the data flow to enter the two compressed IPs (compression cores). The compression core can be a JPEG IP compression core.

[0087] Furthermore, the embodiment of the present application may also store the compressed data output by the first compression core into a first compressed data cache; and store the compressed data output by the second compression core into a second compressed data cache.

[0088] Furthermore, the compressed data in the first compressed data cache and the second compressed data cache may be stored in the video memory; the compressed data may be read from the video memory using an Ethernet card data link layer controller and sent to a remote end through a network.

[0089] In an optional implementation, storing the compressed data in the first compressed data cache and the second compressed data cache into the video memory includes: storing the compressed data in the first compressed data cache and the second compressed data cache into a target compression area preset in the video memory for storing compressed data. The target compression area includes a plurality of partitions, for example, the compressed data can be stored in the plurality of partitions in sequence, such as storing the first frame in the first partition, the second frame in the second partition, the third frame in the fourth partition, and the fifth frame in the first partition.

[0090] It can be seen that the embodiment of the present application detects the write memory address of the video graphics array module, and caches the color image data corresponding to the write memory address in the data cache unit, compares the pixel data in the image frame cache and the data cache unit according to the write memory address, and updates the bitmap to mark the changed pixel data. Each bit in the bitmap corresponds to a pixel data. When the pixel update ratio of the target frame in the image frame cache is less than the preset ratio threshold, the target pixel data corresponding to the update bit of the bitmap is read from the video memory, and the target pixel data is updated to the image frame cache. In this way, there is no need to read the entire frame of the image from the video memory, which saves the read bandwidth of the video memory, reduces the impact on the video memory performance, and thus improves the efficiency of video data transmission.

[0091] Furthermore, the compression core is added to adopt ping-pong operation to increase the compression throughput, so as to meet the frame rate compression display frame rate within one second, thereby reducing the display delay of the remote control.

[0092] For further information, see Figure 3 As shown, Figure 3A schematic diagram of a system architecture using a video transmission method disclosed in an embodiment of the present application. A bitmap update module is added, a JPEG IP control module is added, and one JPEG IP is added. The present invention adds a PIX_MAP module (bitmap control module), a CMP_CTRL module (compression control module), a CMP_IP1 module (second compression core) and a JPEG_2_DDR module (cache control module). Among them, the PIX_MAP module mainly adds a pixel bitmap maintenance process for changes in RGB_DATA data (color image data) in the process, and a cache area for a frame of image and other functions; the CMP_CTRL module is mainly responsible for the distribution and ping-pong operation of the upper module YUV2BLK data to the CMP_IP0 / 1 module (i.e., the first compression core and the second compression core); the JPEG_2_DDR module mainly modifies and adds data caching of the two modules CMP_IP0 / 1 and controls writing to DDR. That is, the management controller may include a video graphics array module and a compression processing module (i.e., a JPEG module) connected to the video graphics array module, an EMCA module, and a DDR CONTROLLER (DDR controller, used to control DDR reading and writing). The compression processing module includes a PIX_MAP module (bitmap control module) connected to the video graphics array module, an RGB2YUV module connected to the output of the PIX_MAP module, a YUV2BLK module connected to the output of the RGB2YUV module, a CMP_CTRL module connected to the output of the YUV2BLK module, a CMP_IP0 and CMP_IP 1 module, and a JPEG_2_DDR module. In the embodiment of the present application, the management controller can be a BMC (i.e., Baseboard Management Controller), an Integrated Light-Out (iLO) system, an Integrated Dell Remote Access Control (IDRAC), or one of the processing chips. Figure 3 Taking the baseboard management controller as an example, it may be other management controllers.

[0093] See also Figure 4 As shown, the design structure of the PIX_MAP module of the present invention is as follows Figure 4 As shown, Figure 4This is a schematic diagram of the structure of a bitmap control module disclosed in an embodiment of the present application. The module uses three cache units, DATA_FIFO (i.e., data cache unit) is used to cache the data on the data bus written by the VGA module (i.e., video graphics array module), FRAME_RAM (i.e., image frame cache) is used to cache a frame of RGB image data, and the BITMAP_UPDATE module (i.e., bitmap update module) is responsible for confirming the write address of the VGA module, comparing the VGA write data with the pixel data of the corresponding address previously cached, so as to confirm which address has changed the pixel, and update the bit indicating the pixel change of the corresponding address. After the frame data is used up, the pixel bitmap is cleared to zero. After use, the YUV data is converted. That is, the bitmap update module in the bitmap control module is used to detect the write memory address of the video graphics array module for the currently transmitted video frame, and cache the color image data corresponding to the write memory address in the data cache unit; read the first pixel data corresponding to the write memory address from the image frame cache based on the write memory address, and compare it with the second pixel data corresponding to the write memory address in the data cache unit, before the arrival of the currently transmitted video frame, the image frame cache caches the color image data of the target frame, and the target frame is the previous frame of the currently transmitted video frame; if the first pixel data is inconsistent with the second pixel data, update the bit value of the bit corresponding to the first pixel data in the bitmap, wherein each bit in the bitmap corresponds to one first pixel data; determine the pixel update ratio of the target frame based on the bitmap; when the pixel update ratio is less than a preset ratio threshold, read the target pixel data corresponding to the update bit of the bitmap from the video memory, and update the target pixel data to the image frame cache.

[0094] The detailed design and process of the PIX_MAP module are as follows: This module mainly controls the module in the BITMAP_UPDATE module, decodes according to the address written to the DDR by the VGA module, compares the pixel at the corresponding address of FRAME_RAM with the data in the DATA_FIFO cache, and if the data is consistent, the bitmap update is abandoned. If the data is inconsistent, it is considered that there is data update, and the corresponding pixel data in the bitmap and FRAME_RAM are updated. During initialization, the first frame of image data is cached in FRAME_RAM. The subsequent image cache updates the pixel data of FRAME_RAM according to the address and data written to the DDR video memory by VGA, and at the same time, the pixel change mapping table in PIX_BITMAP_RAM (bitmap cache) is updated, that is, the bitmap. This table is mainly used to update the changed data read from the video memory in compatible BUFER mode, and there is no need to read out all the pixel data in the DDR video memory space, thereby saving the read bandwidth of DDR. The bit width of DATA_FIFO and FRAME_RAM is 32bit, which is consistent with the data width of RGB8888 pixels. One bit in PIX_BITMAP_RAM represents the change of one pixel value. The process of updating the bitmap in this module is as follows Figure 5 As shown, Figure 5 A bitmap update flow chart disclosed in the embodiment of the present application. The BITMAP_UPDATE module detects a reset signal and initializes PIX_BITMAP_RAM to 0; the BITMAP_UPDATE module detects the VSYNC signal and then detects the arrival of the data valid signal disp_en and then detects the change in the VGA write memory address channel; the BITMAP_UPDATE module detects the VGA write memory address, caches pixel data in DATA_FIFO according to the address valid signal, reads the data in FRAME_RAM according to the address and performs an XOR operation with the data in DATA_FIFO, if the result is 0, it is considered that the bitmap RAM is the same and does not need to be updated, if the result is 1, it is considered that the bitmap RAM is different and needs to be updated; BITMAP_UPDATE updates the bit in PIX_BITMAP_RAM according to the address; when BITMAP_UPDATE receives the completion of the back-end module reading FRAME_RAM, the BITMAP_UPDATE module clears all the bitmaps in PIX_BITMAP_RAM to 0. Wait for the VSYNC signal of the next frame.

[0095] When the BITMAP_UPDATE module switches to BUFER mode, it can use the bitmap to synchronize the pixel data in the DDR video memory that has not been synchronized to the FRAME_RAM. The synchronization strategy is to synchronize the data according to the set update threshold. For example, if the threshold is set to 30%, all the image data in the video memory space will be read out and updated when the update data is greater than 30%. If it is less than 30%, you can choose to read the image data from the DDR video memory space and synchronize it to the FRAME_RAM. When the back-end module (i.e., RGB2YUV module) reads data from FRAME_RAM, it is necessary to control the reading speed and ensure that the reading speed is slower than the writing speed. If the reading speed exceeds the writing speed, new data will be read. Therefore, it is necessary to limit the address when reading and it cannot exceed the writing address.

[0096] For further information, see Figure 6 As shown, the embodiment of the present application discloses a design structure of a CMP_CTRL module. Figure 6 A schematic diagram of the structure of a compression control module provided for an embodiment of the present application. The CMP_CTRL module includes: a FRAME_DEC module (i.e., a frame detection module), a DATA_PIPE module (i.e., a block data cache), and a MUX_CTRL module (i.e., a selection control module). The CMP_CTRL module is mainly used to detect the frame header and frame tail of the YUV data converted into BLOCK data, so as to determine the separation of the image data frame by frame, thereby controlling the data flow to enter the two compressed IPs. The function of FRAME_DEC is to detect the frame header and frame tail of the image data output in the YUV2BLK module, the DATA_PIPE module is to cache and queue the data stream of YUV2BLK, and the MUX_CTRL module is to determine whether the data output by DATA_PIPE is a frame of data based on the frame header and frame tail, thereby giving the current frame to the compression IP of CMP_IP0 (the first compression core), and giving the next frame of data to the compression IP of CMP_IP1 (the second compression core). See Figure 7 As shown, Figure 7 A processing flow chart of a compression control module provided in an embodiment of the present application. FRAME_DEC detects the frame header of the image data output in the YUV2BLK module, FRAME_DEC sends a data valid signal to MUX_CTRL, MUX_CTRL receives the data valid signal and starts to transmit the data of DATA_PIPE to CMP_IPx, FRAME_DEC detects the frame tail of the image data output in the YUV2BLK module, FRAME_DEC sends a data invalid signal to MUX_CTRL, and MUX_CTRL receives the invalid data signal and stops data transmission.

[0097] CMP_IPx is a JPEG compression IP, a standard image compression module, and a standard unit module that compresses YUV data in 8x8 data blocks. Its compression speed cannot meet the high-resolution image mode of 1920x1080 or 1920x1200, so two CMP_IPs are used here. At a resolution of 1920x1080 or 1920x1200, there is a situation where one frame of data is not compressed but the next frame of data comes. Therefore, when CMP_IP0 compresses the end of a frame of data, CMP_IP1 is about to start compressing the beginning of the next frame of image data.

[0098] The function of the JPEG_2_DDR module (cache control module) is to store the data compressed by CMP_IP0 and CMP_IP1 in the DDR cache space, so that ARM can use eMAC to send it to the remote end through DMA. In the JPEG_2_DDR module, CMP_IP0 and CMP_IP1 need to be cached separately, and then the data in the cache is stored in DDR. Figure 8 As shown, the embodiment of the present application discloses the design structure of the JPEG_2_DDR module. Figure 8 A schematic diagram of a cache control module provided in an embodiment of the present application. The module is mainly used to open two IPx_FIFO caches, IPx_FIFO and IPx_FIFO are respectively the first compressed data cache and the second compressed data cache, and the function of the CMP_DATA_WR module (compressed cache judgment module) in the JPEG_2_DDR module determines whether there is data in the two cache FIFOs. If there is data in the FIFO, the data in the cache is stored in the CMP_DATA (i.e., compressed data) area in the DDR space. According to the start and end of compression in CMP_IPx, the compressed data is stored in the partitions divided by CMP_DATA respectively.

[0099] The video data transmission control process may include that the video information of the host HOST is transmitted to the compression processing module through the VGA module (in this embodiment, the compression processing module includes adding a PIX_MAP module, a CMP_CTRL module, a CMP_IP1 module and a JPEG_2_DDR module), and the transmitted video format is RGB format; if the host is connected to the VGA display, the VGA module will store the video data transmitted by the host through PCIe in the DDR, and then the VGA module reads it from the DDR and sends it to the RGB display channel; if it is a remote display, there are two ways to obtain the RGB data, one is to directly obtain the RGB data through the RGB display channel, and the other is that the JPEG module (i.e., the compression processing module) reads the RGB data from the DDR. Because the video compression requires the input YUV format, the JPEG module is processed by the PIX_MAP module to perform RGB to YUV format conversion; after conversion to the YUV format, according to the compression requirements of the JPEG format, it is converted into BLOCK in the order of BLOCK macroblock data input, so it is necessary to first store the YUV data in the DDR, and then read the DDR according to the order of BLOCK. After the macroblock data in BLOCK units is compressed, the compressed video data is obtained and then written into DDR. The network driver initiates a command to read the compressed video data, reads DDR, and performs network packaging. EMAC converts the network packet containing the video data into Ethernet frame format and transmits the video data to the remote end through the network.

[0100] See also Fig. 9 As shown, Fig. 9 A schematic diagram of the structure of a video data transmission device provided by an embodiment of the present invention. The video data transmission device includes:

[0101] An address detection module 11 is used to detect the write memory address of the video graphics array module for the currently transmitted video frame;

[0102] A data cache module 12, used for caching the color image data corresponding to the write display memory address in a data cache unit;

[0103] A pixel comparison module 13 is used to read the first pixel data corresponding to the write display memory address from the image frame buffer based on the write display memory address, and compare it with the second pixel data corresponding to the write display memory address in the data cache unit, wherein the image frame buffer caches the color image data of the target frame before the arrival of the currently transmitted video frame, and the target frame is the previous frame of the currently transmitted video frame;

[0104] A bitmap updating module 14, configured to update the bit value of a bit corresponding to the first pixel data in a bitmap if the pixel comparison module determines that the first pixel data is inconsistent with the second pixel data, wherein each bit in the bitmap corresponds to one first pixel data;

[0105] A ratio determination module 15, configured to determine a pixel update ratio of the target frame based on the bitmap;

[0106] The pixel update module 16 is used to read the target pixel data corresponding to the update bit of the bitmap from the video memory when the pixel update ratio is less than a preset ratio threshold, and update the target pixel data to the image frame buffer.

[0107] The device may also include:

[0108] A format conversion module is used to read color image data from the image frame buffer; and perform format conversion on the read color image data to obtain target transmission format data;

[0109] A data block module, used for obtaining block data by dividing the target transmission format data into blocks;

[0110] A compression control module is used to detect a video frame header and a video frame tail from the block data output by the data blocking module, and cache the block data output by the data blocking module into a block data cache; when a video frame header is detected, the block data is read from the block data cache and transmitted to a first compression core for data compression processing; when a video frame tail corresponding to the video frame header is detected, the step of reading the block data from the block data cache and transmitting it to the first compression core for data compression processing is stopped; when a next video frame header corresponding to the video frame header is detected, the block data is read from the block data cache and transmitted to a second compression core for data compression processing; when a video frame tail corresponding to the next video frame header is detected, the step of reading the block data from the block data cache and transmitting it to the second compression core for data compression processing is stopped, so as to realize the alternate transmission of video frames to the first compression core and the second compression core for data compression processing.

[0111] A cache control module, configured to store the compressed data output by the first compression core into a first compressed data cache; store the compressed data output by the second compression core into a second compressed data cache. Store the compressed data in the first compressed data cache and the second compressed data cache into the video memory;

[0112] The Ethernet card data link layer controller is used to read the compressed data from the video memory and send it to the remote end through the network.

[0113] The cache control module is specifically used for:

[0114] The compressed data in the first compressed data cache and the second compressed data cache are stored in a target compression area preset in the video memory for storing compressed data.

[0115] The format conversion module is further used to store the target transmission format data in a target format area preset in the video memory for storing the target transmission format data, so that the data blocking module reads the target transmission format data from the target format area and blocks the target transmission format data; or directly transmits the target transmission format data to the data blocking module to trigger the step of using the data blocking module to block the target transmission format data.

[0116] Furthermore, the pixel update module 16 is also used to: when the pixel update ratio is greater than or equal to the preset ratio threshold, read the color pixel data of the entire frame from the video memory, and update the color pixel data of the entire frame to the image frame cache.

[0117] It can be seen that the embodiment of the present application detects the write memory address of the video graphics array module, and caches the color image data corresponding to the write memory address in the data cache unit, compares the pixel data in the image frame cache and the data cache unit according to the write memory address, and updates the bitmap to mark the changed pixel data. Each bit in the bitmap corresponds to a pixel data. When the pixel update ratio of the target frame in the image frame cache is less than the preset ratio threshold, the target pixel data corresponding to the update bit of the bitmap is read from the video memory, and the target pixel data is updated to the image frame cache. In this way, there is no need to read the entire frame of the image from the video memory, which saves the read bandwidth of the video memory, reduces the impact on the video memory performance, and thus improves the efficiency of video data transmission.

[0118] Furthermore, the compression core is added to adopt ping-pong operation to increase the compression throughput, so as to meet the frame rate compression display frame rate within one second, thereby reducing the display delay of the remote control.

[0119] Fig. 9 The description of the features in the corresponding embodiments can be found in Figure 2 The relevant descriptions of the corresponding embodiments will not be repeated here one by one.

[0120] Fig.10 A structural diagram of an electronic device provided by an embodiment of the present invention, such as Fig.10 As shown, the electronic device includes: a memory 20 for storing a computer program;

[0121] The processor 21 is used to implement the steps of the video data transmission method in the above embodiment when executing a computer program.

[0122] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0123] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the video data transmission method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include but is not limited to video data, etc.

[0124] In some embodiments, the electronic device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power source 25 , and a communication bus 26 .

[0125] Those skilled in the art will understand that Fig.10 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure.

[0126] It is understandable that if the video data transmission method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the current technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk and other media that can store program codes.

[0127] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned video data transmission method are implemented.

[0128] A computer program product provided in an embodiment of the present application is introduced below. The computer program product described below can be cross-referenced with other embodiments described in this document.

[0129] A computer program product comprises a computer program / instruction, which implements the steps of the above-disclosed video data transmission method when executed by a processor.

[0130] The above is a detailed introduction to a video data transmission method, device, equipment and medium provided by an embodiment of the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0131] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0132] The above is a detailed description of a video data transmission method, device, equipment and medium provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A video data transmission method, characterized in that: include: For the currently transmitted video frame, detecting the write memory address of the video graphics array module, and caching the color image data corresponding to the write memory address in the data cache unit; Based on the write display memory address, first pixel data corresponding to the write display memory address is read from the image frame buffer, and compared with second pixel data corresponding to the write display memory address in the data cache unit, wherein the image frame buffer caches color image data of a target frame before the currently transmitted video frame arrives, and the target frame is a previous frame of the currently transmitted video frame; If the first pixel data is inconsistent with the second pixel data, updating the bit value of the bit corresponding to the first pixel data in the bitmap, wherein each bit in the bitmap corresponds to one first pixel data; Determining a pixel update ratio of the target frame based on the bitmap; When the pixel update ratio is less than a preset ratio threshold, the target pixel data corresponding to the update bit of the bitmap is read from the video memory, and the target pixel data is updated to the image frame buffer.

2. The data transmission method according to claim 1, characterized in that: Also includes: Reading color image data from the image frame buffer; Convert the read color image data into a target transmission format; Using a data block module to block the target transmission format data to obtain block data; Detecting a video frame header and a video frame tail from the block data output by the data block module, and caching the block data output by the data block module into a block data cache; When a video frame header is detected, block data is read from the block data cache and transmitted to the first compression core for data compression processing. When the video frame tail corresponding to the video frame header is detected, the step of reading block data from the block data cache and transmitting it to the first compression core for data compression processing is stopped. When the next video frame header corresponding to the video frame header is detected, block data is read from the block data cache and transmitted to the second compression core for data compression processing. When the video frame tail corresponding to the next video frame header is detected, the step of reading block data from the block data cache and transmitting it to the second compression core for data compression processing is stopped, so as to realize alternating transmission of video frames to the first compression core and the second compression core for data compression processing.

3. The data transmission method according to claim 2, characterized in that: Also includes: storing the compressed data output by the first compression core into a first compressed data cache; The compressed data output by the second compression core is stored in a second compressed data cache.

4. The data transmission method according to claim 3, characterized in that: Also includes: Storing the compressed data in the first compressed data cache and the second compressed data cache into the video memory; The compressed data is read from the video memory by using an Ethernet card data link layer controller, and sent to a remote end through a network.

5. The data transmission method according to claim 4, characterized in that: Storing the compressed data in the first compressed data cache and the second compressed data cache into the video memory includes: The compressed data in the first compressed data cache and the second compressed data cache are stored in a target compression area preset in the video memory for storing compressed data.

6. The data transmission method according to claim 2, characterized in that: After converting the format of the read color image data to obtain the target transmission format data, the method further includes: storing the target transmission format data in a target format area preset in the video memory for storing the target transmission format data, so that the data segmentation module reads the target transmission format data from the target format area and segments the target transmission format data into blocks; Or, directly transmit the target transmission format data to the data segmentation module to trigger the step of segmenting the target transmission format data by using the data segmentation module.

7. The data transmission method according to any one of claims 1 to 6, characterized in that: Also includes: When the pixel update ratio is greater than or equal to the preset ratio threshold, the color image pixel data of the entire frame is read from the video memory, and the color pixel data of the entire frame is updated to the image frame buffer.

8. A video data transmission device, characterized in that: include: An address detection module, used for detecting a write memory address of a video graphics array module for a currently transmitted video frame; A data cache module, used for caching the color image data corresponding to the write display memory address in the data cache unit; a pixel comparison module, configured to read first pixel data corresponding to the write display memory address from an image frame buffer based on the write display memory address, and compare the first pixel data with the second pixel data corresponding to the write display memory address in the data cache unit, wherein the image frame buffer caches color image data of a target frame before the arrival of a currently transmitted video frame, and the target frame is a previous frame of the currently transmitted video frame; A bitmap updating module, configured to update a bit value of a bit corresponding to the first pixel data in a bitmap if the pixel comparison module determines that the first pixel data is inconsistent with the second pixel data, wherein each bit in the bitmap corresponds to one first pixel data; A ratio determination module, used for determining a pixel update ratio of the target frame based on the bitmap; The pixel update module is used to read the target pixel data corresponding to the update bit of the bitmap from the video memory when the pixel update ratio is less than a preset ratio threshold, and update the target pixel data to the image frame buffer.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the video data transmission method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video data transmission method according to any one of claims 1 to 7 are implemented.

Images