A Data Caching Method for CIS Line Array Cameras

Abstract

The present invention discloses a method for caching CIS line array camera data, belonging to the field of image acquisition and processing, and particularly designs a back-end data buffering technology for CIS line array sensors. The main purpose is to cache the segmented images collected by the ADC into the FPGA and output them to the high-speed interface. By re-arranging the data order during the data writing stage, a complete row of images is written in a "Z" shape into a calculated number of RAMs, and during the reading stage, a fixed number of RAMs are sequentially selected and output to the back-end module. It realizes the data caching and image stitching processing from ADC data to the high-speed interface, and has the characteristics of high speed, large bandwidth, high real-time performance, and high development efficiency.

Description

Technical Field

[0001] The present invention relates to the field of image acquisition and processing, and particularly to a method for caching backend data for a CIS linear array sensor. Background Art

[0002] CIS cameras usually use multiple ADCs to collect front-end data. The collected data needs to be sequentially cached into the on-chip RAM of the FPGA and then sent in parallel to high-speed interfaces such as Coaxpress.

[0003] The data buffer module is the most important part of the entire image acquisition system, handling the data stream from the ADC to the high-speed interface. Currently, there are several caching schemes for CIS image acquisition systems. First, the data of each ADC channel is written into an independent RAM and then sampled with a high multiple clock to achieve the effect of parallel output. This method is limited by the maximum operating clock frequency and timing requirements of the FPGA device and is only applicable to low-speed transmission. Second, the data of each ADC channel is written into an independent RAM and then the data is cut into several blocks for parallel output, and finally the image is restored by software. This scheme has high requirements for computer software and poor real-time performance. In addition, for special input-output bandwidth ratios, FIFO can also be used for serial-to-parallel conversion. This scheme is only applicable to a small number of cameras and is difficult to meet the requirements of data acquisition for the vast majority of cameras.

[0004] Object of the Invention

[0005] Based on the prior art, the present invention provides a new data caching scheme for a CIS linear array camera, which rearranges the data order during the RAM writing stage, thereby achieving data buffering and serial-to-parallel conversion of the FPGA under a single clock. For a CIS camera sensor, it cuts a single-line image into several parts and samples them in parallel by multiple ADC channels to improve the sampling speed. Each channel sequentially receives the pixel points of each segment of the image. High-speed interfaces such as Coaxpress require parallel transmission of multiple adjacent pixel points in each clock cycle. If each RAM stores a continuous segment of the picture, read conflicts will inevitably occur during the transmission process. The present invention proposes a new data caching order to solve this problem.

[0006] The technical solution of the present invention is: a method for caching data of a CIS linear array camera, the method comprising:

[0007] Step 1: Use a FIFO to perform line synchronization and clock synchronization on the data collected by the ADC;

[0008] Step 2: Deserialize the synchronized data;

[0009] Step 3: Allocate the data of each Channel obtained by deserialization to the corresponding positions in the RAM;

[0010] Step 4: The cache system performs ping-pong operation according to the external read-write control signal to ensure data integrity;

[0011] Step 5: When reading data, read the data in the RAM in sequence and output it to the high-speed interface.

[0012] Furthermore, the specific method for allocating data to the corresponding positions in the RAM in Step 3 is as follows:

[0013] Step 3.1: Rearrange the data sequence during the RAM writing stage. Calculate that the number of RAMs for the data to be written is n, and these n RAMs are closely arranged;

[0014] Step 3.2: Store the 1st data in RAM1, the 2nd data in RAM2, the 3rd data in RAM3,..., the nth data in RAMn, the (n + 1)th data in RAM1, the (n + 2)th data in RAM2, the (n + 3)th data in RAM3,...

[0015] Write a complete row of the image into the entire column of the RAM in a "Z" shape according to the above method;

[0016] When reading in Step 5, select a fixed number of RAMs in sequence and merge the outputs during the RAM reading process.

[0017] Furthermore, the number of RAMs, the starting address and starting RAM of each channel's data, and the mapping transformation period are determined by three parameters: the number of ADC channels, the data port bit width of the high-speed interface, and the single-channel data length;

[0018] ram = (ch * sensor_length) % ram_num

[0019]

[0020] where ch is the current channel index, sensor_length is the number of pixel points in a single channel, ram_num is the required number of RAMs, ram is the starting RAM index of the current channel, and addr is the starting address of the current channel;

[0021] The number of RAMs is determined by the enumeration method. The initial value is determined as the maximum value of the number of channels and the pixel bit width of the high-speed interface. If the initial RAM index corresponding to each channel repeats, the number of RAMs is incremented by 1 until there is no conflict. % represents taking the remainder. The entire functional module is implemented parametrically. By modifying parameters such as the number of ADCs and the single-channel length, the compiler automatically calculates the starting state of the RAM read-write process, the step size of the address jump, etc., and automatically generates the hardware circuit to implement the corresponding function without reprogramming.

[0022] Further, the specific method for step 4 is as follows:

[0023] Select a 3-line buffer scheme for ping-pong operation, and the read-write control signals are generated by an external module; when the rising edge of the first write signal arrives, the first row of the image is written into RAM group A; when the next write signal arrives, the second row of the image is written into RAM group B; then the third row of the image is written into RAM group C... and so on in a cycle; on the other side, when the rising edge of the read signal arrives, if the system is writing to B at this time, read from A, write to C and read from B, write to A and read from C. Ensure that the read image is a complete image closest to the read time.

[0024] The implementation of the present invention can solve almost all data buffering problems from CIS sampling data to a high-speed interface, can well constrain the timing under a single clock, and can also save computer memory overhead and improve real-time performance when sorting images on an FPGA. By rearranging the data order during the data writing stage, writing a complete row of the image in a "Z" shape into a calculated number of RAMs, and sequentially selecting a fixed number of RAMs to output to the backend module during the reading stage. It realizes data buffering and image splicing processing from ADC data to a high-speed interface, and has the characteristics of high speed, large bandwidth, high real-time performance, and high development efficiency. Description of the Drawings

[0025] Figure 1 It is a schematic diagram of image buffering.

[0026] Figure 2 It is a system block diagram of the data buffer module. Specific Embodiments

[0027] The embodiment of the present invention provides a method for buffering CIS linear array camera data to facilitate those of ordinary skill in the art to understand and implement the present invention. The present invention is further described below, and the steps are as follows:

[0028] Step 1: Select a model of CIS camera, determine its number of channels, number of pixel points per single channel, and data bit width of the high-speed interface; build an image front-end acquisition system except for the buffer module; instantiate the buffer module at the top layer and modify parameters such as the number of channels, number of pixel points per single channel, and data bit width of the high-speed interface. After completing system simulation verification, CIS images can be collected and transmitted to the acquisition card. Synchronize the data collected by the ADC in rows and in clock using a FIFO;

[0029] Step 2: Deserialize the synchronized data;

[0030] During deserialization, a single ADC with 9 parallel acquisition channels and 10-bit precision is used. In the first clock cycle, channels 1, 2, and 3 are output in parallel. In the second clock cycle, channels 4, 5, and 6 are output. In the third clock cycle, channels 7, 8, and 9 are output, and so on. In the (3m + 1)-th clock cycle, channels 1, 2, and 3 are output. In the (3m + 2)-th clock cycle, channels 4, 5, and 6 are output. In the (3m + 3)-th clock cycle, channels 7, 8, and 9 are output, and so on, cycling in sequence.

[0031] Specifically, a state machine is used to control the deserialization process of the ADC. The state cycles and jumps between the (3m + 1)-th clock cycle, the (3m + 2)-th clock cycle, and the (3m + 3)-th clock cycle, with the state changing once per clock cycle, where m = 0, 1, 2, 3,.... To ensure the data phase synchronization of each channel, the data obtained in the states of the (3m + 1)-th clock cycle and the (3m + 2)-th clock cycle are locked in registers and finally synchronously output in the state of the (3m + 3)-th clock cycle.

[0032] Step 3: Allocate the data of each Channel obtained from deserialization to the corresponding positions in the RAM.

[0033] Step 4: The cache system performs ping-pong operations according to external read and write control signals to ensure data integrity.

[0034] Adjust the number of buffered image rows through parameters according to the size of the on-chip resources of the FPGA, and select a 3-row buffering scheme. Compared with the traditional 2-row buffering scheme, buffering 3 rows provides more margin time during the buffering process, avoiding data read-write conflict problems to the greatest extent. In addition, it improves the image quality and reduces the occurrence of image line dropout in the case of mismatched read and write line frequencies. The specific implementation method of the cache system performing ping-pong operations according to external read and write control signals is as follows:

[0035] The read and write control signals are generated by an external module. When the rising edge of the first write signal arrives, the first row of the image is written into RAM group A. When the next write signal arrives, the second row of the image is written into RAM group B. Then, the third row of the image is written into RAM group C, and so on, cycling repeatedly. On the other side, when the rising edge of the read signal arrives, if the system is writing to B at this time, it reads from A; if writing to C, it reads from B; if writing to A, it reads from C. Ensure that the read image is the most recent complete image from the read moment.

[0036] Step 5: When reading data, read the data in the RAM in sequence and output it to the high-speed interface.

Claims

1. A method for caching data of a CIS linear array camera, the method comprising: Step 1: Synchronize the data collected by the ADC row-by-row and clock-by-clock using a FIFO; Step 2: Deserialize the synchronized data; Step 3: Allocate the data of each Channel obtained by deserialization to the corresponding positions in the RAM; Step 3.1: Rearrange the data order during the RAM writing phase. Calculate that the number of RAMs for the data to be written is n, and these n RAMs are arranged closely; Step 3.2: Store the first data in RAM1, the second data in RAM2, the third data in RAM3,..., the nth data in RAMn, the (n + 1)th data in RAM1, the (n + 2)th data in RAM2, the (n + 3)th data in RAM3,... Write a complete row of the image into the entire column of the RAM in a "Z" shape according to the above method; Step 4: The caching system performs a ping-pong operation according to the external read-write control signal to ensure data integrity; Step 5: When reading data, select a fixed number of RAMs in sequence and merge and output them to the high-speed interface; When deserializing in Step 2, a single ADC with 9 parallel acquisition channels and 10-bit precision is used. In the first clock cycle, channels 1, 2, and 3 are output in parallel, in the second clock cycle, channels 4, 5, and 6 are output, in the third clock cycle, channels 7, 8, and 9 are output,..., in the (3m + 1)th clock cycle, channels 1, 2, and 3 are output, in the (3m + 2)th clock cycle, channels 4, 5, and 6 are output, in the (3m + 3)th clock cycle, channels 7, 8, and 9 are output,... and so on in a cycle; During the process of controlling the ADC deserialization with a state machine, the state jumps cyclically between the (3m + 1)th clock cycle, the (3m + 2)th clock cycle, and the (3m + 3)th clock cycle, and the state changes once per clock cycle, where m = 0, 1, 2, 3,...; To ensure the data phase synchronization of each channel, the data obtained in the states of the (3m + 1)th clock cycle and the (3m + 2)th clock cycle will be locked in a register and finally synchronously output in the state of the (3m + 3)th clock cycle.

2. The data caching method of a CIS linear array camera according to claim 1, characterized in that, The number of RAMs, the starting address and starting RAM of the data of each channel, and the mapping transformation period are determined by three parameters: the number of ADC channels, the data port width of the high-speed interface, and the length of the data of a single channel; ram = (ch * sensor_length) % ram_num where ch is the current channel index, sensor_length is the number of pixel points in a single channel, ram_num is the number of required rams, ram is the starting RAM index of the current channel, and addr is the starting address of the current channel; The number of RAMs is determined by the enumeration method. The initial value is determined as the maximum value of the number of channels and the pixel width of the high-speed interface. If the starting RAM index corresponding to each channel repeats, the number of RAMs is incremented by 1 until no conflict occurs. % represents the remainder operation.

3. A CIS linear array camera data caching method according to claim 1, characterized in that, The specific method of Step 4 is: Select the 3-line buffer scheme for ping-pong operation, and the read and write control signals are generated by an external module; when the rising edge of the first write signal arrives, the first line of the image is written into RAM bank A; when the next write signal arrives, the second line of the image is written into RAM bank B; then the third line of the image is written into RAM bank C... and so on in a loop; on the other hand, when the rising edge of the read signal arrives, if the system is writing to B at this time, read from A, write to C and read from B, write to A and read from C; ensure that the image read is the complete image closest to the read time.

Images