A high-resolution CMOS imaging and testing integrated system

Abstract

The present invention provides a high-resolution CMOS imaging and testing integrated system. Through a self-made CMOS imaging and testing system, the imaging quality of an optical imaging system to be tested is evaluated. The testing part of this system includes: creating an ahamming window function centered on an image, finding the image center according to the two-step centroid method, extracting and optimizing the ESF curve by column superposition and SG according to the non-parametric method, and finally extracting and optimizing the LSF curve according to the convolution kernel and Hamming window. Finally, the MTF value is obtained through Fourier transform. The present invention can judge the imaging quality of the optical imaging system to be tested according to this index.

Description

Technical Field

[0001] The present invention belongs to the field of high-speed and high-definition imaging, and specifically relates to a test system for an integrated system of high-resolution CMOS camera imaging and testing. Background Art

[0002] Since the development of the imaging test system to date, it has entered the stage of new computer imaging and computational photography. Through the high-performance computing power of the computer and the globalized data signal processing method, various difficult problems that are difficult to solve in traditional imaging tests have been broken through, enabling breakthroughs in tests such as lensless imaging and high-resolution imaging.

[0003] In the test imaging evaluation method, it has experienced the discrimination method, the star point test measurement method, and the current modulation transfer method. The modulation transfer function (MTF) is an objective index of the performance of the CMOS optoelectronic imaging system. Optimizing the MTF can effectively improve the resolution of the imaging, which plays a guiding role in production and research and development. The resolution of the CMOS imaging system has developed rapidly in recent years and plays an important role in various industries. Therefore, researching and optimizing the MTF of the imaging system will promote the development of many fields. Currently, the modulation transfer function method is still the mainstream evaluation method. In the modulation transfer method, it is divided into three categories: the slit method, the contrast method, and the edge method. The light source of the slit method needs to be a line light source, so the slit needs to be small enough, and in order to prevent diffraction from occurring, the slit cannot be too small, so there are inherent defects; the process difficulty of the sine grating in the contrast method makes it less used in engineering; and the multi-scene usability and low cost of the edge method itself have become the main measurement means.

[0004] However, nowadays, internationally, in the high-end market, the cost of each automatic test equipment developed by well-known equipment manufacturers is as high as hundreds of thousands of US dollars, and in the low-end market, a specific evaluation board is used to measure limited functions. Therefore, in China, the performance parameter technologies of CMOS image sensors have gradually increased in recent years. Mao Cheng, Hu Xinwei et al. from Nanjing University developed a high-speed image sensing general test platform based on the PXIe platform as the core and using NI LABVIEW. Summary of the Invention

[0005] The purpose of the present invention is to provide an integrated system for high-resolution CMOS imaging and testing.

[0006] The technical solution for achieving the object of the present invention is as follows: A high-resolution CMOS imaging and testing integrated system, including a CMOS sensor, a sensor interface baseboard, a power supply unit, an FPGA data processing unit, and a Labview image processing module. The CMOS sensor is used to collect visible light images and, after receiving the driving signal sent by the FPGA, transmit the collected image data to the data processing unit; the sensor interface baseboard is used to fix the CMOS sensor and achieve signal data transmission between the FPGA and the sensor through a differential signal connector and a test base; the power supply unit is used to provide the 3.3V analog voltage, 1.8V interface power supply, and 1.2V digital power supply required by the CMOS sensor; the FPGA data processing unit is used to generate sensor driving signals and perform data restoration through cameralink encoding and decoding; the Labview image processing module is used to convert the collected data from BAYER to RGB, create an ahamming window function centered on the image, find the image center according to the two-center-of-gravity method, extract and optimize the ESF curve according to the non-parametric method by column superposition and SG, extract and optimize the LSF curve according to the convolution kernel and the Hamming window, and perform Fourier transform on the LSF curve to obtain the MTF value.

[0007] Preferably, the specific steps for the Labview image processing module to process the collected images are as follows:

[0008] S1: Convert the collected image from the BAYER space to the RGB space;

[0009] S2: First, filter the grayscale image with a Hamming window centered on the image center, then use the center-of-gravity method to extract the edges to find the image center of gravity, and fit the center of gravity of each row into an inclined straight line;

[0010] S3: Based on the fitted straight line, extract and optimize the ESF curve along the inclination angle of the fitted straight line;

[0011] S4: Use convolution on the ESF curve to obtain the LSF curve, and optimize the LSF using the Hamming window;

[0012] S5: Perform Fourier transform on the LSF curve to obtain the MTF, and display the curve on the Labview host computer interface.

[0013] Preferably, the specific formula for converting the collected image from the BAYER space to the RGB space is as follows:

[0014] If the pixel color is G, then insert the other two colors R and B. The determination formulas for R and B are as follows:

[0015] R X =(R1 + R2) / 2

[0016] B X = (B1 + B2) / 2

[0017] Wherein, R1 represents the red point on the left side of the pixel point, R2 represents the red point on the right side of the pixel point, B1 represents the blue point on the upper side of the pixel point, and R2 represents the blue point on the lower side of the pixel point;

[0018] If the pixel color is R, then insert the other two colors G and B, and the determination formulas for G and B are as follows:

[0019] G X = (G1 + G2 + G3 + G4) / 4

[0020] B X = (B1 + B2 + B3 + B4) / 4

[0021] Wherein, G1 represents the green point above the pixel point, G2 represents the green point below the pixel point, G3 represents the green point on the left side of the pixel point, G4 represents the green point on the right side of the pixel point, B1 represents the blue point in the upper left of the pixel point, B2 represents the blue point in the upper right of the pixel point, B3 represents the blue point in the lower left of the pixel point, and B4 represents the blue point in the lower right of the pixel point;

[0022] If the pixel color is B, then insert the other two colors G and R, and the determination formulas for G and R are as follows:

[0023] G X = (G1 + G2 + G3 + G4) / 4

[0024] R X = (R1 + R2 + R3 + R4) / 4

[0025] Wherein, G1 represents the green point above, G2 represents the green point below, G3 represents the green point on the left side, G4 represents the green point on the right side, R1 represents the red point in the upper left, R2 represents the red point in the upper right, R3 represents the red point in the lower left, and R4 represents the red point in the lower right.

[0026] Preferably, the specific formula for Hamming window filtering is:

[0027] data(i, j) = 0.54 + 0.46 * cos(pi * arg / wid)

[0028] c(i, j) = b(i, j) * data(j)

[0029] Wherein, c(i, j) represents the pixel value filtered by the window function, b(i, j) represents the original pixel value, and data(j) represents the value corresponding to the window function; data(i, j) represents the value of the window function in the i-th row and j-th column, arg represents the number of pixel points of the pixel from the center pixel point, and wid represents the farthest distance of the pixel from the center point.

[0030] Preferably, based on the fitting line, the specific formula for extracting the ESF curve along the inclination angle of the fitting line is as follows:

[0031] Along the fitting line, the data points in each row within the same phase shift period are stacked column by column to obtain an ESF curve. An ESF is obtained for each phase shift period, and the ESF curves are superimposed and averaged to obtain the final ESF curve.

[0032] Preferably, the ESF curve is optimized using a fifth-order SG filter.

[0033] Preferably, the specific formula for obtaining the MTF by performing a Fourier transform on the LSF curve is as follows:

[0034]

[0035] In the formula, MTF M represents the modulation transfer function of the system, LSF(x) represents the line spread function, and f is the frequency.

[0036] Compared with the prior art, the present invention has the following remarkable advantages: 1) In the method of the present invention, all hardware selections use the PXle industrial bus framework. The sensor is driven and data is collected by the FPGA, and the host computer saves and processes the image data and conducts tests, realizing the integration of sensor imaging and testing. After the sensor collects an image, a noise-removed MTF curve can be obtained and measured. The operation interface written in LabVIEW is simple to operate;

[0037] 2) The present invention calculates the MTF curve of the high-resolution imaging test system after removing random noise based on the collected digital images, and judges the performance quality of the imaging system to be tested and whether the quality is qualified according to the above indicators, with high evaluation accuracy;

[0038] 3) The present invention designs a sensor interface board and a sensor test base for the high-resolution CMOS imaging and testing integration method. According to this interface board and test base, the hardware framework of the test can be greatly simplified and the cost can be reduced.

[0039] The following further describes the present invention in detail with reference to the accompanying drawings. Description of the Drawings

[0040] Figure 1 is the structural block diagram of the high-resolution CMOS imaging and testing integration method of the present invention.

[0041] Figure 2 is the structural diagram of the high-resolution CMOS imaging and testing image processing evaluation test method of the present invention.

[0042] Figure 3Design diagram of the sensor interface board for the high-resolution CMOS imaging and testing integrated method of the present invention.

[0043] Figure 4 Sensor test base diagram for the high-resolution CMOS imaging and testing integrated method of the present invention.

[0044] Figure 5 Schematic diagram of the process for extracting the ESF curve in the edge method MTF algorithm of the present invention. Detailed implementation manner

[0045] As Figures 1 to 5 shown, a high-resolution CMOS imaging and testing integrated system includes a CMOS sensor, a sensor interface base board, a power supply unit, an FPGA data processing unit, and a Labview image processing module. The CMOS sensor is used to collect visible light images and, after receiving the drive signal sent by the FPGA, transmit the collected image data to the data processing unit; the sensor interface base board is used to fix the CMOS sensor and realize signal data transmission between the FPGA and the sensor through a differential signal connector and a test base; the power supply unit is used to provide the 3.3V analog voltage, 1.8V interface power supply, and 1.2V digital power supply required by the CMOS sensor; the FPGA data processing unit is used to generate sensor drive signals and perform data restoration through cameralink encoding and decoding; the Labview image processing module is used to convert the collected data from BAYER to RGB, create an ahamming window function centered on the image, find the image center according to the two-center-of-gravity method, extract and optimize the ESF curve according to the non-parametric method by column superposition and SG, extract and optimize the LSF curve according to the convolution kernel and the Hamming window, and perform Fourier transform on the LSF curve to obtain the MTF value.

[0046] The working process of the system of the present invention is as follows:

[0047] Collect test pictures:

[0048] (1a) Arrange a bromine-tungsten light source, an integrating sphere, an object-side telecentric lens, an image-side telecentric lens, and a target. In the designed optical path, the bromine-tungsten lamp generates natural light in random directions, which is modulated into a parallel light beam through the integrating sphere and the object-side telecentric lens. The target will block part of the parallel light, and the remaining parallel light is collimated again through the image-side telecentric lens, and finally the light is incident into the sensor;

[0049] (1b) The FPGA drive unit sends timing control signals and uses SPI to send configuration register signals to drive the CMOS sensor: Write the SPI protocol instruction configuration driver using Verilog code. Implement the transceiver process of the SPI chip select signal, serial clock signal, master transmit slave receive signal, and master receive slave transmit signal through the state machine principle, and write the configuration information into the self-built lookup table and write it sequentially. Convert the RTL code of this function into an edf netlist file. Call the edf netlist file at the integrated node end of LabVIEW on the host computer to generate a sensor configuration drive IP core. Write this function program into the FPGA. Use the sensor configuration drive IP core in the FPGA to drive the sensor, and transmit drive signals to the sensor through the sensor interface board: The FPGA used in the present invention is a Kintex-7 FPGA chip, and a differential signal connector needs to be inserted at its interface to achieve data input and output. Output the corresponding initialization configuration information to the sensor through the written SPI register configuration IP core to complete the drive of the detection container; Control the power supply module by the host computer to provide the 3.3V analog voltage, 1.8V interface power supply, and 1.2V digital power supply required by the sensor for the sensor interface board;

[0050] (1c) After receiving the drive signal, the high-resolution CMOS sensor starts to collect high-definition images and generates digital image data to be sent to the FPGA image data processing unit: The high-resolution CMOS sensor used in the present invention is a high-resolution CMOS sensor of Sony Corporation, which can provide a working speed of 25.8FPS at a resolution of 6646*4852. This CMOS high-resolution digital sensor camera has a high degree of internal integration. The frequency of the accompanying signal of the input pixel drive signal is 13.5MHZ. It internally inherits a low-noise 12Bit ADC conversion module. After ADC conversion, the data is serially output as 1Bit data through the SLVS driver and sent out. What is adopted is 8 data transmission lines plus 1 clock transmission line, and the transmission rate of the clock transmission line is 297MHZ.

[0051] (1d) After receiving the original image signal sent by the high-resolution CMOS sensor, the FPGA image processing unit performs serial-to-parallel conversion processing and signal stability processing: The LVDS transmission mode is used. The signal output by the sensor is a serial signal. A stable data signal is generated through the IDELAYE2 primitive. The stable data signal is serially converted to parallel using ISERDESE2 (DDR mode), generating 6 data signals from the data on one data line, putting the data into the fifo for caching, and then taking the signal out of the fifo and sending it to the protocol decoding module.

[0052] (1f) Decoding module, which mainly uses the transmission format of SLVS. Each frame of the SLVS format includes line blanking, serial communication information frame, invalid image signal line, boundary image signal line, and each line has sync codes (SAV and EAV) and line blanking; when the SAV and EAV bits are ABh and B6hd, the valid image signal between SAV and EAV is intercepted and retained. It is transmitted to the host computer through the FIFO cache, and under the action of the drive signal, high-speed data is generated and sent to the FPGA data processing unit. The FPGA data processing unit receives the data signal, uses the IP core to stabilize the data and perform serial-to-parallel conversion and image decoding to generate a BAYER space picture.

[0053] (2a) Through the previous module, a 12-bit pixel and line-field synchronization signal are obtained. Since the influence of the pixel data signal at the lower bits of the image is weak, the first eight valid data signals are taken. Through the line-field synchronization signal, the coordinates of the pixels are obtained. Since a 9*9 matrix is used, the signals of the first two rows need to be cached in the FIFO. At this time, it is divided into four cases according to the parity of the coordinates. For example, the center of an image is R, the upper, lower, left, and right are G, and the four corners are B. Add the Gs and divide by 4, and similarly get B, and calculate the RGB values of each pixel point. Convert the collected image from the BAYER space to the RGB space. For the nine-square grid centered on this pixel, interpolate it with the surrounding colors. If the middle point is G, then the other two colors are R and B. For inserting R and B, the specific formulas are as follows:

[0054] R X = (R1 + R2) / 2

[0055] B X = (B1 + B2) / 2

[0056] In the formula, R1 represents the left red point, R2 represents the right red point, B1 represents the upper blue point, and R2 represents the lower blue point;

[0057] If the center point is an R point, then the other two colors are G and B. For inserting G and B, the specific formulas are as follows:

[0058] G X = (G1 + G2 + G3 + G4) / 4

[0059] B X = (B1 + B2 + B3 + B4) / 4

[0060] In the formula, G1 represents the upper green point, G2 represents the lower green point, G3 represents the left green point, G4 represents the right green point, B1 represents the upper left blue point, B2 represents the upper right blue point, B3 represents the lower left blue point, and B4 represents the lower right blue point;

[0061] If the center point is point B, the other two colors are G and R. For the inserted G and R, the specific formulas are as follows:

[0062] G X =(G1 + G2 + G3 + G4) / 4

[0063] R X =(R1 + R2 + R3 + R4) / 4

[0064] In the formula, G1 represents the upper green point, G2 represents the lower green point, G3 represents the left green point, G4 represents the right green point, R1 represents the upper left red point, R2 represents the upper right red point, R3 represents the lower left red point, and R4 represents the lower right red point;

[0065] (2b) First, filter the grayscale image with a Hamming window centered on the image, and then use the centroid method to extract the edge. The calculation formula is as shown in Equation

[0066] data(i,j)=0.54 + 0.46*cos(pi*arg / wid)

[0067] c(i,j)=b(i,j)*data(j)

[0068] In the formula, data(i,j) represents the filtered value of the window function, arg represents the number of pixel points from the center of the pixel, and wid represents the farthest distance of the pixel from the center point. c(i,j) represents the pixel value filtered by the window function, and b(i,j) represents the original pixel value;

[0069] (2c) Along the found edge, stack the data of each row in the same period column by column. The obtained data is the ESF, and draw a curve. The translation process is as Figure 5 shown. To ensure the uniformity of the stacked data, it is necessary to delete some positions on the left and right sides, and its length is equal to the size of the inclined line. The formula for calculating the length of deleting the left and right sides is:

[0070] Δx = p tanα

[0071] In the formula, p represents the length of the picture, α represents the inclination angle, and Δx represents the length of deleting both sides of the picture;

[0072] Accumulate the remaining data, and the obtained data is filtered by the fifth-order SG (Savitzky-Golay) filter. Its fitting polynomial is:

[0073] y(x)=a5x 5 +a4x 4 +a3x 3 +a2x 2 +a1x + a0 (1)

[0074] where x represents the pixel value;

[0075] (2d) In the process of extracting the LSF line spread function, common methods include differential processing of the ESF, and obtaining the LSF curve by convolving the ESF(x) curve with the convolution kernel [-0.5, 0, 0.5]. The calculation formula is as follows:

[0076] LSF(x) = ESF(x) * [-0.5, 0, 0.5]

[0077] LSF(x) represents the line spread function, and ESF(x) represents the edge spread function;

[0078] (2f) Fourier transform the LSF curve to obtain the MTF. The processing calculation formula is as in Equation

[0079]

[0080] where MTF M represents the modulation transfer function of the system, and LSF(x) represents the line spread function.

[0081] The modulation transfer function represents the standard of the CMOS camera in image transfer. The vertical axis represents the quality of the contrast, and the horizontal axis represents the distance from the imaging center. The slower the change of this curve, the better the device performance; the steeper the change, the worse the device performance.

Claims

1. A high-resolution CMOS imaging and testing integrated system, characterized in that, It includes a CMOS sensor, a sensor interface base board, a power supply unit, an FPGA data processing unit, and a Labview image processing module. The CMOS sensor is used to collect visible light images and, after receiving the driving signal sent by the FPGA, transmit the collected image data to the data processing unit. The sensor interface base board is used to fix the CMOS sensor and realize the signal data transmission between the FPGA and the sensor through a differential signal connector and a test base. The power supply unit is used to provide the 3.3V analog voltage, 1.8V interface power supply, and 1.2V digital power supply required by the CMOS sensor. The FPGA data processing unit is used to generate the sensor driving signal and perform data restoration through cameralink encoding and decoding. The Labview image processing module is used to convert the collected data from BAYER to RGB, create an ahamming window function centered on the image, find the image center according to the two - time centroid method, extract and optimize the ESF curve by column superposition and SG according to the non - parametric method, extract and optimize the LSF curve according to the convolution kernel and the Hamming window, and perform Fourier transform on the LSF curve to obtain the MTF value.

2. The high-resolution CMOS imaging and testing integrated system according to claim 1, characterized in that, The specific steps for the Labview image processing module to process the collected images are as follows: S1: Convert the collected image from the BAYER space to the RGB space; S2: First, filter the grayscale image with a Hamming window centered on the image center, then use the centroid method to extract the edge to find the image centroid, and fit the centroids of each row into an inclined straight line; S3: Based on the fitted straight line, extract and optimize the ESF curve along the inclination angle of the fitted straight line; S4: Use convolution on the ESF curve to obtain the LSF curve, and optimize the LSF using the Hamming window; S5: Perform Fourier transform on the LSF curve to obtain the MTF and display the curve on the Labview upper computer interface.

3. The high-resolution CMOS imaging and testing integrated system according to claim 2, characterized in that, The specific formula for converting the collected image from the BAYER space to the RGB space is as follows: If the pixel color is G, insert the other two colors R and B. The determination formulas for R and B are as follows: R X = (R1 + R2) / 2 B X = (B1 + B2) / 2 In the formula, R1 represents the red point on the left side of the pixel point, R2 represents the red point on the right side of the pixel point, B1 represents the blue point above the pixel point, and R2 pixel point represents the blue point below; If the pixel color is R, insert the other two colors G and B. The determination formulas for G and B are as follows: G X = (G1 + G2 + G3 + G4) / 4 B X = (B1 + B2 + B3 + B4) / 4 In the formula, G1 represents the green point above the pixel point, G2 represents the green point below the pixel point, G3 represents the green point on the left side of the pixel point, G4 represents the green point on the right side of the pixel point, B1 represents the blue point in the upper left of the pixel point, B2 represents the blue point in the upper right of the pixel point, B3 represents the blue point in the lower left of the pixel point, and B4 represents the blue point in the lower right of the pixel point; If the pixel color is B, insert the other two colors G and R. The determination formulas for G and R are as follows: G X = (G1 + G2 + G3 + G4) / 4 R X = (R1 + R2 + R3 + R4) / 4 In the formula, G1 represents the green point above, G2 represents the green point below, G3 represents the green point on the left, G4 represents the green point on the right, R1 represents the red point in the upper left, R2 represents the red point in the upper right, R3 represents the red point in the lower left, and R4 represents the red point in the lower right.

4. The high-resolution CMOS imaging and testing integrated system according to claim 2, characterized in that The specific formula for Hamming window filtering is as follows: data(i,j) = 0.54 + 0.46 * cos(pi * arg / wid) c(i,j) = b(i,j) * data(j) Where, c(i,j) represents the pixel value filtered by the window function, b(i,j) represents the original pixel value, and data(j) represents the value corresponding to the window function; data(i,j) represents the value of the window function at the i-th row and j-th column, arg represents the number of pixel points of the pixel from the center, and wid represents the farthest distance of the pixel from the center point.

5. The high-resolution CMOS imaging and testing integrated system according to claim 2, characterized in that, Based on the fitted straight line, the specific formula for extracting the ESF curve along the inclination angle of the fitted straight line is as follows: Along the fitted straight line, the data points of each row within the same phase shift period are superimposed column by column to obtain an ESF curve. An ESF is obtained for each phase shift period, and the final ESF curve is obtained by superimposing and averaging the ESF curves.

6. The high-resolution CMOS imaging and testing integrated system according to claim 2, wherein Optimize the ESF curve using fifth-order SG filtering.

7. The high-resolution CMOS imaging and testing integrated system according to claim 2, wherein The specific formula for obtaining the MTF by performing Fourier transform on the LSF curve is as follows: where MTF M represents the modulation transfer function of the system, LSF(x) represents the line spread function, and f is the frequency.

Images