High-speed camera data adaptive two-stage caching system and method and storage medium

Abstract

The invention discloses a high-speed camera data self-adaptive two-stage caching system and method and a storage medium, and relates to the technical field of machine vision high-speed imaging and high-speed data storage, a preprocessing unit is used for performing deserialization and parallel connection, ROI cutting and gain adjustment on collected item number data, and writing incremental FrameID and cyclic redundancy check codes into each frame; the DDR annular cache unit comprises a monitoring module, a decision module and an execution module; the decision module is used for calculating an instantaneous cache demand Cnew and a dynamic cache occupancy U; the execution module expands and shrinks an annular buffer space in a single shot through a DDR4 controller, and outputs a Hi / Lo mark in real time through an occupancy rate comparator to drive an NVMe queue manager. According to the invention, on the premise of zero extra burden in a normal working condition, millisecond-level automatic capacity expansion and high-speed flood discharge can be realized in a sudden working condition, and real-time frame-level integrity verification can be completed on a local FPGA (Field Programmable Gate Array) side. The patent is subsidized by national key research and development plans, and the project number is 2023YFF0719700.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision high-speed imaging and high-speed data storage, and in particular to a high-speed camera data adaptive two-level cache system, method and storage medium. BACKGROUND

[0002] In industrial detection, scientific research experiments, and transient imaging tasks of national defense and aerospace, high-speed cameras are usually equipped with external interfaces with sufficient bandwidth, such as CameraLink, CoaXPress-12, or 10GigE. These interfaces can easily transmit the data output by the CMOS sensor when shooting stably. However, when the system enters the "short exposure plus high gain burst shooting" mode, for example, to capture sub-millisecond phenomena such as explosions, impacts, or laser ablation, the frame rate of the camera will be instantly increased by five to twenty times, resulting in a transient bandwidth inversion. The peak data throughput generated at this time often reaches two to ten times the regular output, immediately exceeding the processing capacity of the interface and the backend host. Relying only on on-chip DDR4 cache in FPGA will be full in a very short time due to limited capacity; directly writing to NVMe SSD also has difficulty in quickly digesting the pulse data stream, which is prone to cause PCIe queue congestion. As a result, frame loss during overspeed is inevitable, key experimental samples are missing, and even national defense measurement data is distorted.

[0003] Existing solutions cannot meet the stringent requirements of high-speed camera burst peak scenarios. Most devices still use static cache strategies with fixed bit depth and fixed DMA block size, and the system locks the data packet size at the design stage, without considering the transient peak caused by dynamic changes in frame rate or window. In order to make up for the lack of capacity, some solutions attempt to dynamically allocate buffers at the software level through the operating system, but the additional overhead of interrupt triggering, context switching, and driver layer copying causes the response time to expand from nanoseconds or microseconds to milliseconds, making it impossible to catch up with the burst data flood. Commercially available FPGA plus SSD data recorders, although claiming high write bandwidth, mostly use constant rate write mode and cannot adjust the queue depth and parallelism in real time according to the upstream traffic, and lack a frame-by-frame integrity detection mechanism. Once write congestion or flushing delay occurs, the system can only exchange continuous operation for the "hidden" frame loss. Traditional frame checking is usually performed at the host software level, and by the time the error is discovered, the key image has already been lost, missing the most valuable real-time alarm window. Therefore, the industry urgently needs a two-level cache and storage architecture that is all hardware. SUMMARY

[0004] Based on the technical problems existing in the background art, the application provides a high-speed camera data adaptive two-stage cache system, method and storage medium, which can automatically expand in milliseconds under a sudden working condition, release flood at high speed and complete real-time frame-level integrity verification on the local FPGA side under the premise of no additional burden under normal working conditions.

[0005] The application provides a high-speed camera data adaptive two-stage cache system, which comprises the following components.

[0006] A preprocessing unit is configured to deserializes, ROI crop and gain adjustment on the collected item data, and write an incremental FrameID and a cyclic redundancy check code into each frame.

[0007] A DDR ring cache unit comprises a monitoring module, a decision module and an execution module.

[0008] The monitoring module is configured to count the camera input bandwidth R in and the fixed output bandwidth R out together to send to a bandwidth prediction unit, and calculate the peak duration T burst using an exponential sliding maximum value algorithm.

[0009] The decision module is configured to calculate the instantaneous cache requirement C need and the dynamic cache occupancy U, then refresh the threshold value FSM and update the dynamic allocation total page number N buf and the high and low threshold values U H , U L .

[0010] The execution module is configured to expand or shrink the ring buffer space within a single shot through a DDR4 controller, and drive an NVMe queue manager through a real-time output of Hi / Lo flags by an occupancy rate comparator, and when U>U H , enable the Submission Queue to write at full speed, and when U<U L , automatically shrink to a low-speed write disk.

[0011] Preferably, the calculation formula of the instantaneous cache requirement C need is as follows.

[0012] C need =R cache T burst K safe

[0013] R in (t)=f frame (t)S frame

[0014] R cache (t)=max(0,R in (t)-R out)

[0015] wherein f frame (t) is real-time frame rate; S frame is single-frame data volume; R cache (t) is required buffer net rate; K safe is safety factor.

[0016] Preferably, the calculation formula of dynamic buffer occupancy U is as follows:

[0017]

[0018] wherein S buf is single-page PageBuffer cache size; WP is write pointer; and RP is read pointer.

[0019] Preferably, the address space of the DDR ring cache unit is divided into equal-length PageBuffers and sequentially mapped to corresponding Banks; the write pointer WP is incremented in a ring manner, and a Full flag is set at the beginning of a page and jumps to the next free page when the page is full; the read pointer RP is cleared of the corresponding Full flag and moved forward after the NVMe write disk or camera output interface is sent.

[0020] Preferably, the DDR ring cache unit also has a variable burst length, and different burst lengths are selected according to the dynamic buffer occupancy U.

[0021] Preferably, when it is determined according to the dynamic buffer occupancy U that a larger cache is still needed, expansion can be completed by only adding new pages at the tail of the Bank rotation sequence; when it is determined that contraction is needed, the pages that have been idle are recovered from the RP end, without data moving.

[0022] Preferably, when the execution module performs multi-queue DMA write disk, the ring cache only sends the pointer of the page into the PagePtr FIFO when the page is full, and the data itself still stays in the DDR; the Scatter-Gather DMA channel polls these pointers in a Round-Robin manner, packs the corresponding Page as an NVMe write command, and carries it to the submission queue.

[0023] Preferably, the SSD returns a completion entry to the Completion Queue and feeds back through the Doorbell every time a write is successful; the CQ analysis logic on the FPGA returns the corresponding PagePtr to the cache free pool and pushes the read pointer RP as soon as the entry is captured, and the entire process does not require any memory copy.

[0024] The present application provides a high-speed camera data adaptive two-stage cache method, and the method steps are as follows:

[0025] S1: the item number data collected is deserialized, ROI cropped and gain adjusted by a preprocessing unit, and an incremental FrameID and cyclic redundancy check code are written for each frame;

[0026] S2: the camera input bandwidth R is counted by a monitoring module of the DDR ring buffer unit in , and is sent to the bandwidth prediction unit together with the fixed output bandwidth R out , and the peak duration T is calculated by using the exponential sliding maximum value algorithm burst .

[0027] S3: the instantaneous cache requirement C and the dynamic cache occupancy U are calculated by a decision module of the DDR ring buffer unit need , then the threshold refresh FSM is updated, and the dynamic allocation total page number N and the high and low thresholds U buf , U H , U L are updated.

[0028] S4: the DDR4 controller of the execution module of the DDR ring buffer unit expands or shrinks the ring buffer space within a single shot, and the Hi / Lo flag is output in real time by the occupancy rate comparator to drive the NVMe queue manager, and when U>U H , the Submission Queue is enabled at full speed to write the disk, and when U L , it is automatically shrunk to a low-speed write disk.

[0029] The computer readable medium proposed in the application stores a computer program, and the computer program is executed by a processor to realize the above-mentioned high-speed camera data adaptive two-level cache method.

[0030] The beneficial technical effects of the application are as follows:

[0031] (1) The cache method of the application makes the cache capacity and the burst traffic keep "breathing" expansion and contraction, and truly realizes on-demand allocation and instant recycling; at the same time, the row / column access granularity is adaptive to the traffic, and the peak bandwidth and low-speed efficiency are considered, the available bandwidth of DDR4 is approached to the theoretical upper limit without changing the physical device, sufficient buffering time is obtained for the back-end NVMe disk writing, thereby laying a hardware foundation for zero frame loss of the whole link.

[0032] (2) The multi-queue DMA disk writing of the application constitutes a high-speed pipeline of "full page generation → multi-channel carrying → multi-queue parallel disk writing → completion feedback → page recycling", so that the system can pull the bandwidth at full capacity when the traffic bursts, and automatically converge when the traffic is stable, and the cache is always smooth. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 It is the principle block diagram of the high-speed camera data adaptive two-level cache system proposed in the application.

[0034] Figure 2 The flow chart of the high-speed camera data adaptive two-level cache method proposed in the application is shown in the figure;

[0035] Figure 3 The DDR4 ring memory combined with Bank interleaving BurstLen is shown in the figure;

[0036] Figure 4 The multi-queue DMA combined with flexible parallelism scheduling is shown in the figure;

[0037] Figure 5 The logic diagram of the frame-level continuity detection and error log module proposed in the application is shown in the figure. DETAILED DESCRIPTION

[0038] The application will be further described below in combination with specific embodiments.

[0039] Embodiment 1

[0040] Referring to Figure 1 , the FPGA (Field-Programmable Gate Array) on-chip DDR4 is set as a first-level cache, and the entire memory is divided into equal-length Pages under ring address mapping. The FPGA logic measures the difference between the camera input bandwidth and the interface output bandwidth in real time, calculates the instantaneous cache requirement by using the bandwidth prediction and burst length model, and then refreshes the Page number and high and low thresholds at the nanosecond level, so that the DDR4 uses as much as it has and returns empty pages immediately after the peak. The NVMe zSSD acts as a second-level persistent cache. When it is detected that the DDR4 occupancy rate exceeds the high threshold, the hardware immediately increases the NVMe Submission Queue parallelism, and writes the Page into the SSD in a multi-channel DMA manner; when the occupancy rate falls below the low threshold, the queue is automatically contracted, and the write amplification and power consumption are reduced. The entire two-level architecture completes adaptive expansion and flood discharge in a hardware closed loop, and the DDR4 high-bandwidth peak absorption + NVMe large-capacity persistence ensures instantaneous zero frame loss.

[0041] Specifically, referring to Figure 1 and Figure 2 , the application proposes a high-speed camera data adaptive two-level cache system, which includes a preprocessing unit and a DDR4 ring cache unit.

[0042] The preprocessing unit is used for deserializing, ROI cropping and gain adjusting the collected item number data, and writing an incremental FrameID and a cyclic redundancy (CRC) check code into each frame, and then the pixel data is sent to the DDR4 ring cache unit.

[0043] DDR4 ring buffer unit includes monitoring module, decision module and execution module, wherein the write pointer WP fills Page sequentially, and the read pointer RP waits for release after Page is full and marked.

[0044] The monitoring module is used to count the camera input bandwidth R in , and send it into the bandwidth prediction unit together with the fixed output bandwidth R out , to calculate the peak duration T burst by using the exponential sliding maximum algorithm.

[0045] The decision module is used to calculate the instantaneous cache requirement C need and the dynamic cache occupancy U, and then refresh the FSM by threshold and update the total number of dynamically allocated pages N buf , the high and low thresholds U H , U L .

[0046] The execution module expands or shrinks the ring buffer space within a single shot through the DDR4 controller, and outputs the Hi / Lo flag in real time through the occupancy rate comparator to drive the NVMe queue manager. When U > U H , the Submission Queue is enabled at full speed to write to the disk, and when U < U L , it is automatically shrunk to a low-speed write disk. The whole start-stop process takes only microseconds.

[0047] The calculation formula of the instantaneous cache requirement C need is as follows:

[0048] C need = R cache T burst K safe

[0049] R in (t) = f frame (t) S frame

[0050] R cache (t) = max(0, R in (t) - R out )

[0051] Where f frame (t) is the real-time frame rate; S frame is the single-frame data volume; R cache (t) is the required buffer rate; and K safe is the safety factor.

[0052] The calculation formula of the dynamic cache occupancy U is as follows:

[0053]

[0054] where S buf is the single page PageBuffer cache size; WP is the write pointer; and RP is the read pointer.

[0055] Referring to Figure 3 , the address space of the DDR4 ring buffer unit is divided into equal-length PageBuffers and mapped to the corresponding eight Banks in sequence: Page0 corresponds to Bank0, Page1 corresponds to Bank1, and so on, until Bank7, and then back to Bank0; the write pointer WP is incremented in a ring manner, and every time a page is written, a Full flag is set at the beginning of the page and the next free page is jumped to; the read pointer RP clears the corresponding Full flag and moves forward after the NVMe write disk or camera output interface is sent.

[0056] In addition, the DDR4 ring buffer unit also has a variable burst length, and different burst lengths are selected according to the dynamic cache occupancy U. For example, when the input bandwidth is extremely high, the control logic selects BL64; when the bandwidth is medium, it switches to BL32; when approaching the end of the page or the bandwidth is low, it automatically converges to BL16 to avoid tail page fragmentation. BurstLen is re-evaluated and seamlessly switched every 256 clocks according to the real-time bandwidth, and is completely transparent to the upper layer data stream.

[0057] According to the dynamic cache occupancy U, when it is determined that a larger cache is still needed, only the new page at the end of the Bank rotation sequence needs to be added to complete the expansion; when it is determined that contraction is needed, the idle page is recovered from the RP end, without data moving. Through the combination of "Bank stagger + variable BurstLen", the ring buffer can maintain high bandwidth and high row hit rate at any frame rate, and at the same time, relying on the circulation mechanism of WP / RP and dynamic page number adjustment, it can realize the immediate absorption of burst bandwidth and efficient recovery of steady stream.

[0058] In principle, the ring buffer abstracts the DDR4 linear address space into a closed loop with the head and tail connected, so that the write end and the read end circulate in the same logical channel without the need for data movement or segment arrangement to continuously write and release; this naturally avoids the complex movement operation triggered by the "write tail overflow and read head idle" in the traditional linear buffer under burst scenario. And through the simple comparison of "WP never crosses RP", the relative position of the two is guaranteed, so as to completely eliminate the risk of covering the data not read due to the write tail out of bounds under the burst high flow. Further introduce Bank interleaving, evenly distribute adjacent pages to several Banks, so that continuous writing forms a multi-Bank parallel, "row consistent, column incremental" access mode, and in the ideal state, each Bank is activated rather than repeatedly precharged, so as to improve the row hit rate and compress the bus idle period to the minimum. At the same time, the variable BurstLen mechanism can switch freely according to the real-time bandwidth: use BL64 when the bandwidth is high, and thin out the row overhead to the maximum; use BL32 / BL16 when the bandwidth decreases or approaches the page tail, which avoids tail page fragmentation and controls the write delay. The three work together to bring two innovative advantages: first, the cache capacity and burst flow maintain "breathing" expansion, truly realizing on-demand allocation and immediate recycling; second, the row / column access granularity is adaptive to the flow, balancing peak bandwidth and low-speed efficiency, without changing the physical device, the available bandwidth of DDR4 is close to the theoretical limit, which provides sufficient buffering time for the back-end NVMe write disk, thereby laying the hardware foundation for zero frame loss in the whole link.

[0059] Referring to Figure 4 When the execution module performs multi-queue DMA write disk, whenever a page is full, the ring buffer only sends the pointer of the page into the PagePtr FIFO, and the data itself still stays in DDR4. The Scatter-Gather DMA channel polls these pointers in a Round-Robin manner, packages the corresponding page as an NVMe write command, and carries it to the submission queue. The queue manager located in the middle waist decides how many submission queues to activate in real time according to the cache occupancy rate: when the occupancy rate is high, SQ0-SQ3 are enabled at the same time to release the flood peak, and when the pressure decreases, the redundant queues are automatically closed to reduce write amplification.

[0060] In addition, the SSD returns a completion entry to the Completion Queue every time a write is successful and feeds back through the Doorbell; the CQ analysis logic on the FPGA returns the corresponding PagePtr to the cache free pool and pushes the read pointer RP as soon as the entry is captured, without any memory copy. This forms a high-speed pipeline of "full page generation → multi-channel carrying → multi-queue parallel write disk → completion feedback → page recycling", which makes the system pull the bandwidth to the maximum under burst flow and automatically converge under steady flow, always keeping the cache unblocked.

[0061] Finally, refer to Figure 5 To ensure zero frame loss during burst acquisition at the hardware level, this embodiment writes a self-incrementing frame number to each frame at the front end of the data path and appends a CRC16 checksum value. A "continuity detection and error module" is then added within the FPGA. This module simultaneously determines the frame number increment relationship and performs CRC verification at a single clock cycle. If a sequence number interruption or verification failure is detected, the error flag is set, and the abnormal frame number and its corresponding DDR4 physical page address are written to the error FIFO. This closed-loop hardware system allows the system to instantly locate missing frames, log them, and generate alarms while data remains in the cache or NVMe queue. This provides a reliable basis for subsequent frame replenishment, retriggering, or quality assessment, fundamentally ensuring the integrity of the entire link.

[0062] The serial number comparator uses pure combinational logic and compares FrameID every time cur With FrameID prev The CRC16 engine, which performs table lookup and expansion, verifies the frame header data in parallel. The two results are logically ORed together. Once the Error_Flag is triggered, the hardware immediately writes the abnormal frame number and its physical page address in DDR4 to the Error FIFO and generates an interrupt.

[0063] Example 2

[0064] The present invention proposes a high-speed camera data adaptive dual-level caching method, the method steps are as follows:

[0065] S1: The pre-processing unit performs deserialization, ROI cropping, and gain adjustment on the collected item data, and writes an incremental FrameID and cyclic redundancy check code to each frame;

[0066] S2: The monitoring module of the DDR4 ring buffer unit counts the camera input bandwidth R in and with a fixed output bandwidth R out The peak duration T is calculated by using the exponential sliding maximum algorithm. burst ;

[0067] S3: Calculate the instantaneous cache demand C through the decision module of the DDR4 ring cache unit need and dynamic cache occupancy U, then refresh FSM by the threshold and update the total number of dynamically allocated pages N buf and high and low thresholds U H 、U L ;

[0068] S4: DDR4 controller of execution module of DDR4 ring buffer unit expands or shrinks ring buffer space within a single shot, and drives NVMe queue manager through real-time output of Hi / Lo flag of occupancy rate comparator, and when U>U H , instantaneously enables Submission Queue full-speed write disk, and when U L , automatically shrinks to a single low-speed write disk.

[0069] Embodiment 3

[0070] The application provides a computer readable medium storing a computer program, and the computer program is executed by a processor to realize the adaptive dual-level cache method of high-speed camera data.

[0071] Although the embodiments of the present application have been shown and described, it is to be understood that various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents, and all should be included within the protection scope of the present application.

Claims

1. A high speed camera data adaptive two-level cache system, characterized by, The application comprises: A preprocessing unit for deserializing, ROI cropping and gain adjusting the collected item number data, and writing an incremental FrameID and cyclic redundancy check code for each frame; A DDR ring buffer unit comprising a monitoring module, a decision module and an execution module; The monitoring module is used for counting the camera input bandwidth R in , and is sent into a bandwidth prediction unit together with a fixed output bandwidth R out , and the peak duration T burst is calculated by using an exponential sliding maximum algorithm. The decision module is used to calculate the instantaneous cache requirement C need and the dynamic cache occupancy U, which subsequently refreshes the FSM by the threshold and updates the dynamically allocated total page number N buf and the high and low thresholds U H , U L ; The execution module expands or shrinks the ring buffer space within a single shot through the DDR4 controller, and drives the NVMe queue manager through the real-time output of the Hi / Lo flag by the occupancy rate comparator. When U>U H , the Submission Queue is enabled at full speed for a moment, and when U L , it is automatically reduced to a low-speed write disk.

2. The high speed camera data adaptive two-level cache system of claim 1, wherein, instantaneous cache requirement C need The formula for calculating this is: C need = R cache T burst K safe R in (t) = f frame (t) S frame R cache (t) = max(0, R in (t) - R out ) where f frame (t) is the real-time frame rate; S frame is the single frame data amount; R cache (t) is the required buffer net rate; K safe is the safety factor.

3. The high speed camera data adaptive two-level cache system of claim 1, wherein, The calculation formula of the dynamic cache occupancy rate U is: In the formula, S buf is the size of the single-page PageBuffer cache; WP is the write pointer; and RP is the read pointer.

4. The high speed camera data self-adapting two-level cache system of claim 1, wherein, The address space of the DDR ring buffer unit is divided into equal-length PageBuffers, which are sequentially mapped to corresponding Banks; the write pointer WP is incremented in a ring manner, and a Full flag is set at the beginning of each page when it is full, and the next free page is jumped to; the read pointer RP is cleared after the NVMe write disk or camera output interface is sent, and the corresponding Full flag is removed.

5. The high speed camera data self-adapting two-level cache system of claim 4, wherein, The DDR ring buffer unit also has a variable burst length, and different burst lengths are selected according to the dynamic cache occupancy rate U.

6. The high speed camera data self-adapting two-level cache system of claim 4, wherein, When it is determined according to the dynamic cache occupancy rate U that a larger cache is still needed, expansion can be completed by adding new pages at the end of the Bank rotation sequence; when it is determined that contraction is needed, the idle pages are recovered from the RP end, and no data movement is generated.

7. The high speed camera data self-adapting two-level cache system of claim 1, wherein, When the execution module performs multi-queue DMA write disk, whenever a page is full, the ring buffer only sends the pointer of the page into the PagePtr FIFO, and the data itself still stays in the DDR; the Scatter-Gather DMA channel polls these pointers in a Round-Robin manner, packs the corresponding Page into an NVMe write command, and moves it to the submission queue.

8. The high speed camera data self-adapting two-level cache system of claim 7, wherein, Each time the write is successful, the SSD returns a completion entry to the Completion Queue and feeds back through the Doorbell; the CQ analysis logic on the FPGA returns the corresponding PagePtr to the cache free pool and pushes the read pointer RP as soon as the entry is captured, and the whole process does not require any memory copy.

9. A method for adaptive dual-level caching of high-speed camera data, comprising: The method steps are as follows: S1: deserializing, ROI cropping and gain adjusting the collected item number data through the preprocessing unit, and writing an incremental FrameID and cyclic redundancy check code for each frame; S2: statistics of camera input bandwidth R by monitoring module of DDR ring buffer unit in , and together with fixed output bandwidth R out , the peak duration T is calculated by exponential sliding maximum algorithm in bandwidth prediction unit burst ; S3: Calculate the instantaneous cache requirement C by the decision module of the DDR ring buffer unit need and the dynamic cache occupancy U, followed by threshold refresh FSM and update the number of dynamically allocated pages N buf and high, low threshold U H , U L ; S3: DDR4 controller of execution module of DDR ring buffer unit expands and shrinks ring buffer space within a single shot, and drives NVMe queue manager through real-time output of Hi / Lo flag of occupancy rate comparator, and when U>U H , instantaneously enables Submission Queue full-speed write disk, and when U L , automatically shrinks to a low-speed write disk.

10. A computer readable medium storing a computer program, characterized in that, The computer program is executed by the processor to realize the adaptive two-level cache method for high-speed camera data according to claim 9.

Images