Large-area cmos detector data buffer heterogeneous speed matching method based on flow rate control

Abstract

The application relates to a large-array CMOS detector data buffer heterogeneous speed matching method based on flow rate control, in which the large-array CMOS detector adopts a multi-channel synchronous output mode to output image data to an FPGA, a double-port RAM in the FPGA buffers the input multi-channel image data, a DDR3-IP core controller reads the buffered data from the double-port RAM in the FPGA and writes the data into a DDR3-SDRAM, and a flow rate control method is used to perform heterogeneous speed matching on the read and write operations of the double-port RAM in the FPGA. The application realizes image data heterogeneous speed matching through a one-stage buffer double-port RAM and a flow rate control method; the resource utilization rate of the internal RAM of the FPGA is reduced when image data is buffered and output; 1 / 2 of the double-port RAM resources can be released for image processing; and the reduction of the resource utilization rate also leads to the reduction of the dynamic power consumption of the FPGA.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a method for heterogeneous speed matching of data buffers in a large-area CMOS detector based on flow rate control. Background Technology

[0002] When using FPGA to buffer the input image data stream of a large-area CMOS detector, it is necessary to match the buffer rate for image data streams with different input and output rates to ensure no data accumulation in the buffer and stable data output. In the field of space remote sensing, ping-pong storage structures are often used for buffering image data from large-area CMOS detectors to achieve row and frame image buffering. Row image buffering refers to buffering one row of image data and outputting one row of image data according to the output rate requirement; frame image buffering refers to buffering one frame of image data and outputting one frame of image data according to the output rate requirement after performing image preprocessing.

[0003] In space-based visible light remote sensing, large-area CMOS detectors are increasingly larger. Line images reach over 10K pixels, and frame images exceed 100M pixels, such as the GMAX32152 detector from Changguang Chenxin Technology Co., Ltd., which has a pixel size of 16K×9K. Due to the limitations of internal FPGA storage resources, a single FPGA cannot handle frame image buffering. Therefore, external high-speed memory, such as DDR2-SDRAM or DDR3-SDRAM, is added to the design. When writing data to SDRAM, a timed refresh operation is required on the memory cell, thus interrupting the data writing process and preventing image data from being written at a fixed rate. The DDR3-IP core user port determines whether read / write operations can be performed based on the SDRAM's execution status. When the user drive clock is 100MHz, its average image data access rate can reach approximately 80MHz. Therefore, when providing image data to be stored for the DDR3-IP core, the design must be based on the average access rate, which leads to an image data buffer rate mismatch problem (i.e., the aberration problem).

[0004] A common method to solve the data access speed problem is to use a ping-pong storage approach. Two sets of RAM storage areas are allocated within the FPGA. The first set of RAM stores odd-numbered rows of image data, and the second set stores even-numbered rows, alternating between the two. While storing data in the second set of RAM, data is read from the first set. After data integration, the data is stored in DDR3-SDRAM, and so on. This caching mode only needs to ensure that the time for image data to be read from RAM and stored in DDR3-SDRAM is less than the buffering time of the input image data to avoid data access conflicts.

[0005] The image data caching design method described above consumes a significant amount of RAM storage resources on the FPGA. For an image with a 120Kb line cache, 240Kb of RAM storage resources are required. If BlockRAM is used, the number of BlockRAM instantiations doubles, which is extremely disadvantageous for large-scale digital systems that require RAM resources to perform preprocessing algorithms. The extensive use of RAM also increases the dynamic power consumption of the FPGA, affecting the power supply system and heat dissipation. Summary of the Invention

[0006] To address the issues of high FPGA internal RAM resource occupancy and increased FPGA dynamic power consumption in existing large-area CMOS detector image data cache design methods, this invention provides a flow rate control-based method for large-area CMOS detector data cache speed matching. When implementing speed matching of image data cache based on DDR3-SDRAM, this method can reduce the FPGA internal RAM resource occupancy, thereby freeing up RAM resources to complete more image preprocessing functions, while also reducing device power consumption.

[0007] To solve the above problems, the present invention adopts the following technical solution:

[0008] The large-area CMOS detector data buffering heterogeneous speed matching method based on flow rate control is used. The large-area CMOS detector outputs image data to the FPGA in a multi-channel synchronous output mode. The FPGA internal dual-port RAM buffers the input multi-channel image data. The DDR3-IP core controller reads the buffered data from the FPGA internal dual-port RAM and writes it into DDR3-SDRAM.

[0009] The read / write operations of the FPGA's internal dual-port RAM are matched at different speeds using flow rate control methods. The specific process includes the following steps:

[0010] Calculate the image data buffer size A based on the write speed of the dual-port RAM;

[0011] Determine if the cache size A is greater than the threshold T1. If so, start the DDR3-IP core controller, read the image data cached inside the FPGA, mark the read value B, and then write the image data into the DDR3-SDRAM through the DDR3-IP core controller. During the writing process, update the value of the read value B and the storage address range in real time. If not, stop the DDR3-IP core controller from writing to the DDR3-SDRAM.

[0012] Calculate the flow difference C between cache size A and read volume B, and control the start and stop of the DDR3-IP core controller based on the relationship between the flow difference C and thresholds T1 and T2. The control rules are as follows:

[0013] When T2 < C ≤ T1, the FPGA buffer data is waiting to be stored, and the SDRAM read and write timing of the DDR3-IP core controller is executed normally.

[0014] When C≤T2, stop the SDRAM write timing of the DDR3-IP core controller;

[0015] When C = T1 < T ram ≤T max When the SDRAM write timing of the DDR3-IP core controller is activated, image data is read from the cached dual-port RAM and written to the DDR3-SDRAM;

[0016] When C > T1, the SDRAM read / write timings of the DDR3-IP core controller are executed normally, and the status flag Δ is set. When C > T1, the SDRAM read / write timings of the DDR3-IP core controller are executed normally. ram When the status flag Δ is set, the cache capacity is increased by modifying the parameters of the FPGA's internal cache RAM.

[0017] Among them, the threshold T ram The threshold T is defined as the maximum storage space of the FPGA cache, which is determined by the data rate received by the FPGA cache and the instantaneous data rate at which the DDR3-IP core controller reads image data from the FPGA's internal cache; max The value is defined as the theoretical maximum difference between cache size A and read size B, which is calculated based on the refresh time of DDR3-SDRAM.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The proposed method for data buffering and heterogeneous speed matching of a large-area CMOS detector based on flow rate control achieves heterogeneous speed matching of image data through a level-one cache dual-port RAM. When controlling image data buffering and output, it reduces the resource utilization of the FPGA's internal RAM. Compared with existing methods, it can free up 1 / 2 of the dual-port RAM resources for image processing. The reduction in resource utilization also leads to a reduction in the dynamic power consumption of the FPGA. Attached Figure Description

[0020] Figure 1 This is a block diagram illustrating the flow rate control principle in an embodiment of the present invention;

[0021] Figure 2 The present invention provides a flow rate control method and control timing sequence.

[0022] Figure 3 This is a schematic diagram of the data storage organization structure of a large-area CMOS detector in an embodiment of the present invention;

[0023] Figure 4 The image is a visible light camera image obtained using a flow rate control method in an embodiment of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and preferred embodiments.

[0025] This invention addresses the problem of excessive storage resource consumption during caching of image data streams at different rates within an FPGA. It proposes a method for different-rate matching of data caching in a large-area CMOS detector based on flow rate control. This method optimizes the timing of data caching control and achieves image data flow rate control through a flow rate chasing method during image data caching, thus achieving the goal of different-rate matching during image data caching.

[0026] The large-area CMOS detector outputs image data to the FPGA using a multi-channel synchronous output method. The FPGA's internal dual-port RAM buffers the input multi-channel image data. The DDR3-IP core controller reads the buffered data from the FPGA's internal dual-port RAM and writes it into the DDR3-SDRAM.

[0027] The heterogeneous matching method provided in this embodiment mainly includes four aspects: the data organization structure of the image buffer of a large-area CMOS detector, the principle of image buffer flow rate control, the timing of image buffer flow rate control, and the image storage space allocation strategy.

[0028] 1) Data organization structure

[0029] Large-area CMOS detectors use a multi-channel synchronous output method to output image data, with up to 100 input channels; the image data quantization bit width is mostly 12 bits. When the FPGA completes the data reception buffer, it rearranges the input multi-channel image data based on the bit width of the parallel input image data of the DDR3-IP core.

[0030] The DDR3-IP core user data bus width can be set to 2N bits (N = 7, 8, 9, 10). When the FPGA receives multi-channel synchronous input image data, it synchronously writes multi-channel image data according to the IP core settings. When the IP core user data bus width is 256 bits, it can write data corresponding to 21 image channels (252 bits) at a time.

[0031] When the detector pixel output rate is lower than the DDR3-IP core user data bus storage rate, time-sharing processing can be used to achieve data caching for more image channels. For example, if the image data input rate is 25MHz and the average storage rate is calculated at 75MHz (based on a DDR3-IP core operating clock of 100MHz and an efficiency of 75%), then caching for 63 image channels can be completed simultaneously.

[0032] 2) Flow rate control principle

[0033] The principle of flow rate control is as Figure 1 shown. The inlet flow rate V1 represents the data rate at which the FPGA cache receives data (corresponding to the RAM write operation); the outlet flow rate V2 represents the instant data rate at which the IP core of the DDR3 - SDRAM reads image data from the FPGA internal cache (corresponding to the RAM read operation). When setting the IP core, ensure that the average outlet flow rate is greater than the average inlet flow rate so that no data accumulation occurs in the buffer area; when the data volume in the FPGA internal RAM cache reaches the threshold value T1, start the data read operation.

[0034] Calculate the image data cache volume (also known as the received volume) A according to the write rate of the dual - port RAM; determine whether the cache volume A is greater than the threshold T1. When the cache volume A is greater than the threshold T1, start the DDR3 - IP core controller, read out the image data in the FPGA internal cache, and mark the read - out volume B. The initial value of the read - out volume B is 0, and write the image data into the DDR3 - SDRAM through the read - out volume B. When writing, update the value of the read - out volume B and the storage address range in real time.

[0035] The detector outputs image data at a certain rate and writes it into the FPGA at a constant speed. The inlet flow rate V1 is a constant rate; because the DDR3 - SDRAM needs to be refreshed regularly and the IP core controller is in a dynamic burst storage mode, the outlet flow rate V2 is a non - uniform output. Therefore, the input and output show a non - uniform speed mode, resulting in a dynamic change in the difference between the image data cache volume A and the read - out volume B. Under normal circumstances, the outlet flow rate V2 is greater than V1, A - B is less than T1, and no data accumulation occurs in the buffer area; however, since the SDRAM - IP core is a burst - type access logic and cannot perform read - write operations during the regular refresh period, it cannot be guaranteed that A - B < T1.

[0036] Calculate the theoretical maximum difference between the cache volume A and the read - out volume B according to the SDRAM refresh time, and set the threshold T ram (T max ≥T ram >T1), T ram is defined as the maximum storage space of the FPGA buffer area to ensure that no image data accumulation and overwriting occur. T ram is set according to V1 and V2.

[0037] Since V2 is greater than V1, during operation, the image data in the buffer area decreases to tend to 0. Set the threshold value T2. When A - B = T2 (T2 < T1), stop the write operation of the DDR3 - SDRAM through the DDR3 - IP core controller until the data in the FPGA internal cache reaches the threshold value T1, that is, when A - B > T1, start the write operation of the DDR3 - SDRAM again.

[0038] This method achieves asynchronous matching of detector image data buffering through a set of dual-port RAM inside the FPGA, reducing the utilization rate of RAM resources inside the FPGA and reducing the dynamic power consumption of the FPGA in terms of resources and speed.

[0039] 3) Flow rate control timing

[0040] The timing sequence for flow rate control of image data buffer heterogeneous matching in large-area CMOS detectors is as follows: Figure 2 As shown. When image data is input into the cache RAM, the amount of cached data A is recorded. When A > T1, the DDR3-IP core controller is started to read the image data from the cache RAM and write it into the SDRAM. The image data writing timing is executed according to the DDR3-IP core timing requirements and is completed by the DDR3-IP core controller.

[0041] When image data is read from the buffer and written to SDRAM, the amount of data read out, B, is recorded. The flow difference C = AB between the buffer amount A and the amount of data read out, and the start / stop of the DDR3-IP core controller is controlled according to the relationship between the flow difference C and the thresholds T1 and T2. The control rules are as follows:

[0042] When T2 < C ≤ T1, the FPGA buffer data is waiting to be stored, and the SDRAM read and write timing of the DDR3-IP core controller is executed normally.

[0043] When C≤T2, it means that the read rate is high at this time, and data needs to be cached to continue. The DDR3-IP core is then controlled to stop the SDRAM write timing.

[0044] When C = T1 (T1 < T) ram ≤T max When the cache has enough image data, the SDRAM write timing of the DDR3-IP core controller is activated, and the image data is read from the cache dual-port RAM and written to the DDR3-SDRAM.

[0045] When C > T1, the image data is written to the buffer at a relatively fast rate, resulting in image data accumulation. The SDRAM read / write timings of the DDR3-IP core controller execute normally. The status flag Δ (Δ is an alarm signal) is set when C > T1. ram When the status flag Δ is set, it indicates that the buffer capacity is insufficient. The buffer capacity can be increased by modifying the internal buffer RAM parameters of the FPGA.

[0046] This method employs a catch-up approach to control data flow rate, using a single blockRAM to achieve data cache speed matching. By controlling the timing of the DDR3-IP core driver, the internal cache data capacity of the FPGA is reduced, thereby lowering the dynamic power consumption of the FPGA.

[0047] 4) Storage space allocation strategy

[0048] SDRAM can achieve an average read / write efficiency of over 80%, and the RAM cache capacity T is set based on the number of row pixels per channel of a large-area CMOS detector. max This allows for reliable storage. Thresholds T1 and T2 are in (T ram / 8,T ram / 2) can be freely set within the range.

[0049] Because the DDR3-IP core controller independently controls write address and write data, SDRAM storage data is set in units of 1 frame of image data M. When the amount of data read B = M, the IP core write timing is stopped, and the write address range is controlled according to the value of M.

[0050] Once both the number of written data and the number of written addresses reach M, the storage of one frame of image data is complete. The read timing of the DDR3-IP core controller is then initiated, and the image data is output. After the read timing is completed, the FPGA internal registers are initialized, and the buffer control for the next frame of image data is initiated.

[0051] This method achieves image data speed matching by using a level-1 cache dual-port RAM and flow rate control. When caching and controlling the output of image data, it reduces the resource utilization of the FPGA's internal RAM. Compared with existing methods, it can free up 1 / 2 of the dual-port RAM resources for image processing. The reduction in resource utilization also leads to a reduction in the dynamic power consumption of the FPGA.

[0052] The following is combined with Figure 3 The specific examples shown illustrate the heterogeneous speed matching method of the present invention in detail.

[0053] In the design of a visible light large-area CMOS detector imaging system for a certain project, image data is transmitted in rows, with 36 parallel input images. The image data frequency of each channel is 25MHz, and the equivalent image data rate is 25×36=900MHz.

[0054] Data transfer is accomplished using a single DDR3-SDRAM chip, and memory control is achieved through a DDR3-IP core controller integrated within the FPGA. The IP core user port data rate is 100MHz × 128 bits, which is equivalent to 800MHz × 16 bits for the DDR3-SDRAM.

[0055] Image data is input in 12 channels × 25MHz × 10 bits, which is equivalent to 25MHz-120bit image data. Therefore, 36 channels of image data are equivalent to 120 bits, 75MHz image data input. Based on the SDRAM efficiency of 80%, it meets the SDRAM data rate requirements.

[0056] First, the 36-channel image data is cached in the FPGA's internal RAM with a capacity of 256. When the cached data reaches T1 (T1 = 128), the SDRAM write operation is initiated, generating the SDRAM write control timing. The image data is read from the FPGA's internal dual-port RAM using the DDR3-IP core and written to the SDRAM at a rate of 100MHz.

[0057] SDRAM write operations require the insertion of timed refresh timings. Therefore, the read rate of the dual-port RAM is controlled based on the real-time write data rate of the SDRAM; and the start and stop of the SDRAM write operation are controlled by the amount of RAM data read out, B.

[0058] Dual-port RAM data is written cyclically (first-in, first-out), meaning that when the cached data A reaches the maximum RAM capacity T... max (T max When the value is 256, the newly input image data will overwrite the image data written first in RAM, thus completing the data update.

[0059] Image data is read from SDRAM and transmitted to the fast-viewing device via the CameraLink interface. Verification has shown that the image data can be transmitted completely without data overwriting or loss, proving the method's practicality and effectiveness. (Exterior image data is shown below.) Figure 4 As shown.

[0060] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for heterogeneous velocity matching of data buffering in a large-area CMOS detector based on flow velocity control, characterized in that, The large-area CMOS detector outputs image data to the FPGA using a multi-channel synchronous output method. The FPGA's internal dual-port RAM buffers the input multi-channel image data. The DDR3-IP core controller reads the buffered data from the FPGA's internal dual-port RAM and writes it into the DDR3-SDRAM. The flow rate control method is used to perform speed matching for read and write operations of the FPGA's internal dual-port RAM. Specifically... Includes the following processes: Calculate the image data buffer size A based on the write speed of the dual-port RAM; Determine if the cache size A is greater than the threshold T1. If so, start the DDR3-IP core controller, read the image data cached inside the FPGA, mark the read value B, and then write the image data into the DDR3-SDRAM through the DDR3-IP core controller. During the writing process, update the value of the read value B and the storage address range in real time. If not, stop the DDR3-IP core controller from writing to the DDR3-SDRAM. Calculate the flow difference C between cache size A and read volume B, and control the start and stop of the DDR3-IP core controller based on the relationship between the flow difference C and thresholds T1 and T2. The control rules are as follows: When T2 < C ≤ T1, the FPGA buffer data is waiting to be stored, and the SDRAM read and write timing of the DDR3-IP core controller is executed normally. When C≤T2, stop the SDRAM write timing of the DDR3-IP core controller; When C = T1 < T ram ≤T max When the SDRAM write timing of the DDR3-IP core controller is activated, image data is read from the cached dual-port RAM and written to the DDR3-SDRAM; When C > T1, the SDRAM read / write timings of the DDR3-IP core controller are executed normally, and the status flag Δ is set. When C > T1, the SDRAM read / write timings of the DDR3-IP core controller are executed normally. ram When the status flag Δ is set, the cache capacity is increased by modifying the parameters of the FPGA's internal cache RAM. Among them, the threshold T ram The threshold T is defined as the maximum storage space of the FPGA cache, which is determined by the data rate received by the FPGA cache and the instantaneous data rate at which the DDR3-IP core controller reads image data from the FPGA's internal cache; max The value is defined as the theoretical maximum difference between cache size A and read size B, which is calculated based on the refresh time of DDR3-SDRAM.

2. The large-area CMOS detector data buffer heterogeneous velocity matching method based on flow velocity control according to claim 1, characterized in that, When the read quantity B equals the data quantity M of one frame of image data, the SDRAM write timing of the DDR3-IP core controller is stopped, and the write address range is controlled according to the value of M. Once the number of written data and the number of written addresses both reach M, the storage of one frame of image ends, the read timing of the DDR3-IP core controller is started, the image data is output, and after the read timing ends, the FPGA internal registers are initialized, and the buffer control of the next frame of image data is started.

3. The large-area CMOS detector data buffer heterogeneous velocity matching method based on flow velocity control according to claim 1, characterized in that, The range of values ​​for thresholds T1 and T2 is (T ram / 8,T ram / 2).

Images