Video frame cache processing method and system

Abstract

The invention discloses a cache processing method and system for video frames, and the method carries out the following steps when reading video data: obtaining a write cache pointer, setting a read cache pointer according to the write cache pointer, and enabling the set read cache pointer to be smaller than the write cache pointer; when a read starting instruction is received, judging whether the amount of data stored in the read buffer is smaller than a first preset value or not; when the amount of data stored in the read buffer is smaller than the first preset value, a read base address is determined according to the read cache pointer, the read base address is sent to the storage interface, and video data sent by the DDR through the storage interface is received until the amount of data stored in the read buffer is not smaller than the first preset value; and when the value is not less than the first preset value, outputting the video data in the read buffer. According to the technical scheme, the image can be prevented from being torn.

Description

Technical Field

[0001] The present invention relates to the field of video processing, and in particular to a video frame cache processing method and system. Background Art

[0002] In the field of video processing, video frame buffering is an important component. Video data is usually transmitted in the form of frames. In order to ensure real-time processing and playback, video frame data must be effectively stored and managed. The traditional video buffering method uses a ping-pong operation to read the frame buffer area in turn. However, when the difference between the read and write frequencies is large (the read data frequency is usually greater than the write data frequency), the read and write base address pointers will overlap in the same frame buffer area, resulting in image tearing, reducing the stability of video stream transmission and system reliability. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a video frame cache processing method and system in view of the defects of low stability of video stream transmission and low system reliability in the prior art.

[0004] The technical solution adopted by the present invention to solve the technical problem is: construct a video frame cache processing method, which is applied to a video processor, and the video processor is connected to the DDR through a storage interface, and when reading video data, the following steps are performed:

[0005] Obtaining a write cache pointer, and setting a read cache pointer according to the write cache pointer, wherein the set read cache pointer is smaller than the write cache pointer;

[0006] Upon receiving a read enable instruction, determining whether the amount of data stored in the read buffer is less than a first preset value;

[0007] When the amount of data stored in the read buffer is less than the first preset value, determining a read base address according to the read cache pointer, sending the read base address to the storage interface, and receiving video data sent by the DDR through the storage interface until the amount of data stored in the read buffer is not less than the first preset value;

[0008] When it is not less than the first preset value, the video data in the read buffer is output.

[0009] Preferably, determining the read base address according to the read cache pointer includes:

[0010] The product of the read cache pointer and the offset address is used as the read base address, wherein the offset address is the product of the line resolution and the field resolution of the cached video.

[0011] Preferably, setting the read cache pointer according to the write cache pointer includes:

[0012] The value obtained by subtracting 1 from the write cache pointer is used as the read cache pointer.

[0013] Preferably, it also includes:

[0014] When a read enable disable signal is received from the outside, the read buffer is disabled.

[0015] Preferably, when writing video data, the following steps are performed:

[0016] Receiving video data from the outside and splicing the received video data;

[0017] Storing the spliced ​​video data into a write buffer, and determining whether the amount of data stored in the write buffer reaches a second preset value, the second preset value being less than the first preset value;

[0018] When a second preset value is reached, determining a write base address according to the write cache pointer, and sending the write base address to the storage interface, wherein the write cache pointer is incremented by 1 when a frame of video data is received, and is set to 1 when a reset signal is received or when it points to a maximum frame cache number;

[0019] The video data stored in the write buffer is transmitted to the DDR through the storage interface.

[0020] Preferably, the first preset value is three times the second preset value.

[0021] Preferably, determining the write base address according to the write cache pointer includes:

[0022] When a frame of video data is received, the current write base address is added to the offset address to update the current write base address, wherein the offset address is the product of the row resolution and field resolution of the cached video, and the write base address is initialized to 0 when a reset signal is received or when it points to the maximum frame cache number.

[0023] Preferably, when writing the video data, the following steps are also performed:

[0024] When a field synchronization signal is received from the outside, the rising edge of the field synchronization signal is captured and pulse stretched to obtain a clear signal;

[0025] The video data in the write buffer is cleared according to the clear signal.

[0026] Preferably, it also includes:

[0027] The parameter bit width of the maximum number of frame buffers is received from the outside, and the maximum number of frame buffers is adjusted according to the parameter bit width.

[0028] The present invention also constructs a video frame cache processing system, including a video processor and a DDR connected to the video processor through a storage interface. The video processor implements the above-mentioned video frame cache processing method when executing a computer program.

[0029] Through the technical solution of the present invention, when reading video data, since the read cache pointer is smaller than the write cache pointer, only the video data of the previous frame (not the currently written frame) cached in the DDR can be read. In this way, even in the case of a large difference in the read and write frequencies, the video data of the same address in the DDR can be avoided from being read and written, thereby avoiding the occurrence of image tearing. Moreover, before the video data in the read buffer is output, it is also determined whether the amount of data stored in the read buffer is less than a first preset value. When it is less than the first preset value, the corresponding video data in the DDR needs to be read into the read buffer first, so that the amount of data stored in the read buffer is not less than the first preset value. Then, the read buffer starts the output of video data to the outside, which can prevent the read buffer from being empty due to too fast a reading rate, and further prevent image tearing. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0031] Figure 1 It is a flow chart of Embodiment 1 of the method for caching video frames of the present invention;

[0032] Figure 2 It is a schematic diagram of a read cache pointer and a write cache pointer when writing and reading video data in the present invention;

[0033] Figure 3 It is a flow chart of Embodiment 2 of the method for caching video frames of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] First of all, it is explained that VDMA (Video Direct Memory Access) is used to convert a data stream in AXI Stream format into a Memory Map format or to convert a Memory Map format into an AXI Stream format data stream, that is, the VDMA IP is intended to provide a video read / write transmission function from an AXI4 interface protocol to an AXI4-Stream interface protocol, and vice versa, thereby realizing a large amount of data movement between a system memory (DDR) and an AXI4-Stream-based video processing IP. The present invention provides a video processor with a novel VDMA structure, and the novel VDMA structure can be named LVDMA (Lightweight Video Direct Memory Access).

[0036] Figure 1 This is a flow chart of the first embodiment of the method for caching video frames of the present invention. First, it is explained that the method for caching video frames is applied to a video processor, and the video processor can be a VDMA IP implemented by Xilinx FPGA. The video processor is connected to DDR (external storage unit) through a storage interface, and the storage interface can be MIG (Memory Interface Generator) IP, which is used to realize the interaction between FPGA and DDR, and the FPGA uses AXI4 FULL bus to interact with MIG IP.

[0037] In this embodiment, when reading the video data, the following steps are performed:

[0038] Step S11, obtaining a write cache pointer, and setting a read cache pointer according to the write cache pointer, wherein the set read cache pointer is smaller than the write cache pointer;

[0039] Step S12, upon receiving the read-on instruction, determining whether the amount of data stored in the read buffer is less than a first preset value, if so, executing step S13; if not, executing step S14;

[0040] Step S13, determining a read base address according to the read cache pointer, sending the read base address to the storage interface, and receiving video data sent by the DDR through the storage interface until the amount of data stored in the read buffer is not less than a first preset value;

[0041] Step S14: output the video data in the read buffer.

[0042] Regarding this embodiment, the following points need to be explained:

[0043] 1. Design data reading module, data writing module and address control module (MemoryAddr Control) in the video processor.

[0044] 2. The data reading module is designed based on the read buffer (TX FIFO), and a control signal is added to the top layer of the data reading module to start the burst transmission from the storage interface (MIG IP) to the TX FIFO. For example, the control signal is introduced by the data request module downstream of the VDMA, which is usually the VGA IP module of Xilinx. When the control signal is set (the read start instruction is received), it is determined whether the amount of data in the TX FIFO is less than the first preset value. If so, the AXI read address valid signal is pulled high, and the AXI read address ready signal is waited. When the AXI read address ready signal is received, the AXI read base address is written to the MIG IP, and then the AXI read ready signal is waited for to be pulled high, and the AXI transmission can be started, that is, data is written from the MIG IP to the TX FIFO, and the transmission is stopped after the data of the first preset value is filled, and then the downstream data request module is waited for to pull the data read enable signal high before outputting, thereby preventing the TX FIFO from underflowing and causing the video stream to be interrupted. When the amount of data in the TX FIFO is not less than the first preset value, a start signal is sent to the downstream data request module (such as VGA IP). The VGA IP starts working and pulls up the enable signal according to the video protocol to request data from the data reading module. At this time, the data reading module can output the video data in the TX FIFO to the VGA IP.

[0045] 3. The address control module controls the data reading module to only read the data content of the previous one or more frames written by the data writing module.

[0046] 4. When the amount of data stored in the read buffer is less than the first preset value, it is necessary to start the read burst transmission, and after reading a frame of video data, assign the read base address to the AXI read address, otherwise the AXI read address starts from 0 and the number of read burst bytes is accumulated after each burst transmission.

[0047] Through the technical solution of this embodiment, when reading video data, since the read cache pointer is smaller than the write cache pointer, only the video data of the previous frame (not the currently written frame) cached in the DDR can be read. In this way, even in the case of a large difference in read and write frequencies, it is possible to avoid reading and writing video data at the same address in the DDR, thereby avoiding the occurrence of image tearing. Moreover, before outputting the video data in the read buffer, it is also determined whether the amount of data stored in the read buffer is less than a first preset value. If it is less than the first preset value, the corresponding video data in the DDR needs to be read into the read buffer first, so that the amount of data stored in the read buffer is not less than the first preset value. Then, the read buffer starts outputting video data to the outside, thereby preventing the read buffer from being empty due to too fast a reading rate, and further preventing image tearing.

[0048] Further, in an optional embodiment, step S13 determines the read base address in the following manner: the product of the read cache pointer and the offset address is used as the read base address, wherein the offset address is the product of the row resolution and the field resolution of the cached video.

[0049] Further, in an optional embodiment, setting the read cache pointer according to the write cache pointer in step S11 includes: using the value of the write cache pointer minus 1 as the read cache pointer. In this embodiment, since the read cache pointer is 1 less than the write cache pointer, when reading video data, only the video data written in the previous frame that has been cached in the DDR can be read, thereby avoiding reading and writing video data at the same address in the DDR.

[0050] In a simulation experiment, the data reading frequency of the data reading module is 200Mhz, the data writing frequency of the data writing module is 100Mhz, and the buffer area (in DDR) contains three frames of video data. Figure 2 As shown, when the data writing module writes data in the second frame buffer area, the data reading module can only read data in the first frame buffer area. Moreover, since the frequency of reading data is twice the frequency of writing data, it means that after the data in the first frame buffer area is read, only half of the data is written into the second frame buffer area. After adopting the technical solution of this embodiment, the data reading module will not immediately jump into the second frame buffer area to read data, and its read address pointer will point to the first address of the first frame buffer area, re-read the first frame data, and wait for the data writing module to complete the writing of the second frame buffer area data before entering the second frame buffer area to read data. Therefore, it can be ensured that the data reading module and the data writing module will not read and write data stored at the same address at the same time, which solves the problem of image tearing that is prone to occur in ping-pong operation and improves the stability of video stream transmission.

[0051] Further, in an optional embodiment, the video frame cache processing method of the present invention further includes: when receiving a read enable shutdown signal from the outside, closing the read buffer. In this embodiment, a TX FIFO enable signal can be designed in the data reading module to allow the TX FIFO to be enabled or disabled. Specifically, the TX FIFO enable signal is led out at the top layer, and the user controls the TX FIFO to be turned on or off. Specifically, when the TX FIFO enable signal is set to 1, the TX FIFO is turned on, so that a FIFO is generated in the data reading process; when the TX FIFO enable signal is set to 0, the TX FIFO is turned off, so that a FIFO is not generated in the data reading process, thereby achieving the purpose of saving hardware resources.

[0052] Figure 3 1 is a flow chart of Embodiment 2 of the method for caching video frames of the present invention. In this embodiment, when writing video data, the following steps are performed:

[0053] Step S21, receiving video data from the outside and splicing the received video data;

[0054] In this step, the video acquisition system is a single-input multiple-output system, so when receiving external video data, it is necessary to first splice the input video data into parallel data of the AXI bus data width (for example, 64-bit width).

[0055] Step S22, storing the spliced ​​video data into a write buffer, and determining whether the amount of data stored in the write buffer reaches a second preset value, and the second preset value is less than the first preset value. If so, executing step S23;

[0056] In this step, the second preset value is, for example, a whole line of video data. When a whole line of video data is written into the RX FIFO, burst transmission is enabled to transmit the data to the storage interface via the AXI4 FULL protocol. In addition, the depth of the RX FIFO can be set at the top level to accommodate the cache requirements of video streams of different resolutions, and the highest resolution cache calculation formula that the FPGA of the LVDMA structure of the present invention can support is as follows:

[0057] resolution hs ≤depth fifo *(64÷width pixel )

[0058] Among them, resolution hs Indicates horizontal pixels, depth fifo Indicates the maximum depth of FIFO settings, width pixelIndicates the bit width of a single pixel value. For example, when the maximum depth of the RX FIFO is set to 1024 and the video stream is in 8-bit RAW video format, LVDMA can support a video buffer with a horizontal resolution of 1024*8 or 8K.

[0059] Step S23, determining a write base address according to the write cache pointer, and sending the write base address to the storage interface, wherein the write cache pointer is incremented by 1 when a frame of video data is received, and is set to 1 when a reset signal is received or when it points to the maximum frame cache number;

[0060] In this step, regarding the write cache pointer, it should be noted that the write cache pointer accumulates by 1 when receiving a frame of video data (for example, the falling edge of the field synchronization signal arrives) until it points to the maximum frame cache number, and is set to 1 when it points to the maximum frame cache number or receives a reset signal.

[0061] Step S24: transmitting the video data stored in the write buffer to the DDR through the storage interface.

[0062] Regarding this embodiment, it should be noted that the data writing module is designed based on the write buffer (RX FIFO). When the number of readable data in the RXFIFO reaches the second preset value (burst length), the burst start signal is pulled high, the AXI write address valid signal is pulled high, and the AXI write address increases with the burst length. When the AXI write valid signal is pulled high, the burst transmission is turned on, that is, data is written from the RX FIFO to the MIG IP.

[0063] Further, in an optional embodiment, the first preset value is three times the second preset value. For example, the second preset value is the data volume of one line of video data, and the first preset value is the data volume of three lines of video data.

[0064] Further, in an optional embodiment, step S23 determines the write base address according to the write cache pointer, including:

[0065] When a frame of video data is received, the current write base address is added to the offset address to update the current write base address, wherein the offset address is the product of the row resolution and field resolution of the cached video, and the write base address is initialized to 0 when a reset signal is received or when it points to the maximum frame cache number.

[0066] In this embodiment, after receiving a frame of video data (for example, the falling edge of the field synchronization signal arrives), and the write buffer pointer points to the maximum frame buffer area, the write base address is assigned 0. After receiving a frame of video data (for example, the falling edge of the field synchronization signal arrives), but the write buffer pointer does not point to the maximum frame buffer area, the write base address is accumulated with the offset address. The write base address is assigned to the AXI write bus address after the falling edge of the field synchronization signal arrives, and the AXI write bus address is accumulated with the number of write burst bytes after the AXI transmission is started.

[0067] Furthermore, in an optional embodiment, when writing the video data, the following steps are also performed:

[0068] When a field synchronization signal is received from the outside, the rising edge of the field synchronization signal is captured and pulse stretched to obtain a clear signal;

[0069] The video data in the write buffer is cleared according to the clear signal.

[0070] In this embodiment, since the arrival of the rising edge of the field synchronization signal represents the input of a new frame of video data, when the rising edge of the field synchronization signal is captured, the video data in the RX FIFO can be cleared to prevent the RX FIFO from writing the previous frame of video data into the DDR, thereby avoiding cache errors. Moreover, since the effective clearing of the RX FIFO needs to be maintained for a long time, this is determined by the nature of a specific video processor. For example, the Xilinx FPGA needs to maintain 8 cycles of the slow clock, otherwise the RX FIFO cannot be read and written normally. Based on this, the data writing module stretches the rising edge pulse of the field synchronization signal slowly and then uses it as the clearing signal of the RX FIFO, so that the RX FIFO can effectively clear the video data of the previous frame when a new frame of video data is input. In this way, on the one hand, it can prevent the RX FIFO from being out of control due to insufficient clearing time; on the other hand, it can solve the problem of video stream interruption caused by the image sensor. The specific reason is: if the image sensor has a frame break problem due to physical electromagnetic interference and other reasons, that is, the line field synchronization signal fails, the pixel data fails, and it returns to normal after a period of time. If the RX FIFO is not cleared with the rising edge of the field synchronization signal, when the image sensor returns to normal, the data cached in the RX FIFO is the video data stored before the failure, which will cause the problem of misalignment of the captured image.

[0071] Furthermore, in an optional embodiment, the video frame cache processing method of the present invention further includes:

[0072] A parameter bit width of a maximum number of frame buffers is received from outside, and the maximum number of frame buffers is adjusted according to the parameter bit width.

[0073] In this embodiment, the maximum frame buffer number can be adjusted by controlling the parameter bit width of the maximum frame buffer number. For example, when the parameter bit width of the maximum frame buffer number is set to 10, the maximum frame buffer number is 1024 frames, that is, 1024 frame buffer areas can be opened.

[0074] The present invention also constructs a video frame cache processing system, which includes a video processor and a DDR connected to the video processor through a storage interface. The video processor implements the above-mentioned video frame cache processing method when executing a computer program.

[0075] Finally, under the same configuration, the number of resources occupied by the video processor of the present invention (LVDMA structure) and the video processor of the prior art (VDMA structure) are compared. As shown in Table 1, in the video processor of the LVDMA structure of the present invention, the resource usage of the lookup table (LUTs) and the trigger (FFS) is reduced by more than 80%, and the resource usage of the dedicated RAM (BRAM) increases with the increase of Line depth, because the FIFO inside the structure needs to increase the depth to cache more data. In addition, the resource usage of the video processor of the present invention does not increase with the increase of the number of frame buffers. Therefore, the video processor of the LVDMA structure of the present invention not only has high reliability and a larger number of frame buffers, but also has a lower resource utilization rate.

[0076]

[0077] Table 1

[0078] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of the claims of the present invention.

Claims

1. A video frame cache processing method, applied to a video processor, wherein the video processor is connected to a DDR through a storage interface, characterized in that: When reading video data, perform the following steps: Obtaining a write cache pointer, and setting a read cache pointer according to the write cache pointer, wherein the set read cache pointer is smaller than the write cache pointer; Upon receiving a read enable instruction, determining whether the amount of data stored in the read buffer is less than a first preset value; When the amount of data stored in the read buffer is less than the first preset value, determining a read base address according to the read cache pointer, sending the read base address to the storage interface, and receiving video data sent by the DDR through the storage interface until the amount of data stored in the read buffer is not less than the first preset value; When it is not less than the first preset value, the video data in the read buffer is output.

2. The video frame cache processing method according to claim 1, characterized in that: The determining the read base address according to the read cache pointer comprises: The product of the read cache pointer and the offset address is used as the read base address, wherein the offset address is the product of the line resolution and the field resolution of the cached video.

3. The video frame cache processing method according to claim 1, characterized in that: The step of setting the read cache pointer according to the write cache pointer includes: The value obtained by subtracting 1 from the write cache pointer is used as the read cache pointer.

4. The video frame cache processing method according to claim 1, characterized in that: Also includes: When a read enable disable signal is received from the outside, the read buffer is disabled.

5. The video frame cache processing method according to any one of claims 1 to 4, characterized in that: When writing video data, the following steps are performed: Receiving video data from the outside and splicing the received video data; Storing the spliced ​​video data into a write buffer, and determining whether the amount of data stored in the write buffer reaches a second preset value, the second preset value being less than the first preset value; When a second preset value is reached, determining a write base address according to the write cache pointer, and sending the write base address to the storage interface, wherein the write cache pointer is incremented by 1 when a frame of video data is received, and is set to 1 when a reset signal is received or when it points to a maximum frame cache number; The video data stored in the write buffer is transmitted to the DDR through the storage interface.

6. The video frame cache processing method according to claim 5, characterized in that: The first preset value is three times the second preset value.

7. The video frame cache processing method according to claim 5, characterized in that: Determining the write base address according to the write cache pointer includes: When a frame of video data is received, the current write base address is added to the offset address to update the current write base address, wherein the offset address is the product of the row resolution and field resolution of the cached video, and the write base address is initialized to 0 when a reset signal is received or when it points to the maximum frame cache number.

8. The video frame cache processing method according to claim 5, characterized in that: When writing video data, the following steps are also performed: When a field synchronization signal is received from the outside, the rising edge of the field synchronization signal is captured and pulse stretched to obtain a clear signal; The video data in the write buffer is cleared according to the clear signal.

9. The video frame cache processing method according to claim 5, characterized in that: Also includes: The parameter bit width of the maximum number of frame buffers is received from the outside, and the maximum number of frame buffers is adjusted according to the parameter bit width.

10. A video frame cache processing system, comprising a video processor and a DDR connected to the video processor via a storage interface, characterized in that: The video processor implements the video frame cache processing method described in any one of claims 1-9 when executing the computer program.

Images