Column-level fixed-mode noise correction method for large-area-array CMOS sensor

Abstract

The invention discloses a column-level fixed-mode noise correction method for a large-area-array CMOS sensor, and belongs to the technical field of CMOS image sensors. The method comprises the steps of collecting an original image, collecting the original image of the large-area-array CMOS sensor under uniform illumination, calculating the mean ratio of each column of the upper part and the lower part of the image, fitting a correction curve method, fitting a smooth curve according to the change trend of the mean ratio of each column, and correcting the lower half part of the image column by column by utilizing the fitted curve. According to the method, the ratio fitting curve is used for correcting the column-level noise of the large-area-array CMOS, so that when a large-area-array CMOS chip is purchased, the chip does not need to be screened, all the column-level noise caused by slope errors can be corrected, the production cost is greatly saved, after correction through the method, the column-level fixed noise of the CMOS is corrected, and the product quality is improved. Pixels of the whole area array are consistent in response under the same illumination, brightness mutation of adjacent columns is avoided through fitting curve smoothing processing, and the image quality is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of CMOS image sensors, and in particular to a column-level fixed pattern noise correction method for large-array CMOS sensors. Background Art

[0002] CMOS image sensors are now widely used in aerospace, industrial inspection, and other fields. With the advent of the big data era, the market has placed higher demands on CMOS resolution and integration. To improve resolution, CMOS image sensors need to increase the size of the pixel array. Due to the limitations of current processing technology, increasing the pixel array inevitably leads to inconsistent ramp signals during the CMOS chip manufacturing process. This problem generates column-level fixed noise, which seriously affects image quality.

[0003] In the past, to address this issue, chip manufacturers started with chip hardware, optimizing the ramp generation circuit to improve the gain error of the ramp signal. However, when these methods were applied to ultra-large array CMOS image sensors, slope consistency errors still existed, resulting in column-level fixed noise in the final CMOS image. For scenes with high imaging quality requirements, the only way to ensure the image quality was to screen CMOS chips with smaller slope consistency errors. This greatly increased production costs and wasted a lot of time on chip screening. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for correcting column-level fixed pattern noise of large-array CMOS sensors. The method uses a ratio fitting curve to correct the column-level noise of large-array CMOS sensors, thereby solving the problem of increasing production costs by screening CMOS chips with smaller slope consistency errors to ensure image output quality.

[0005] The present invention is achieved through the following technical solutions:

[0006] The present invention provides a method for correcting column-level fixed pattern noise of a large-array CMOS sensor, comprising the following steps:

[0007] Collect original images and collect original images of large-array CMOS sensors under uniform illumination.

[0008] Calculate the ratio of the mean values ​​of each column in the upper and lower parts of the image. Divide the original image into two parts, and calculate the mean value of the upper half (ave_up) and the lower half (ave_down) of each column respectively.

[0009] The calibration curve fitting method is to fit a smooth curve according to the changing trend of the mean ratio of each column.

[0010] The lower half of the image is corrected column by column using the fitting curve, and each column of pixels in the lower half of the original image is multiplied by the corresponding fitting ratio to obtain the corrected image.

[0011] Furthermore, the ratio of the mean of the upper half of each column of the original image to the mean of the lower half is calculated. The ratio ave_ratio_up_down is calculated as follows:

[0012] ave_ratio_up_down(1,j)=ave_up(1,j) / ave_down(1,j)

[0013] Where ave_up is the mean of each column in the upper half, ave_down is the mean of each column in the lower half, and ave_ratio_up_down is the final ratio.

[0014] Furthermore, if the theoretical image is uniform up and down, the ratio ave_ratio_up_down is 1. Multiply each column of pixels in the lower half by the corresponding ratio to obtain a new image. The formula is as follows:

[0015] New_image(i,j)=Original_image(i,j)*ave_ratio_up_down(1,j)

[0016] Where New_image(i,j) is the processed lower half image, Original_image(i,j) is the original image, and ave_ratio_up_down(1,j) is the ratio of each column multiplication.

[0017] Furthermore, 12 CMOS chips were sampled and the upper and lower mean ratios of each column were calculated.

[0018] Furthermore, by fitting a correction curve method, a curve is fitted according to the change trend of ave_ratio_up_down, so that the change ratio between each column of pixels changes linearly.

[0019] Furthermore, the calibration curve fitting method uses quadratic polynomial regression, and the fitting formula is:

[0020] ave_ratio_up_down_new(j)=a*j2+b*j+c

[0021] Among them, a, b, and c are fitting coefficients, which are determined by the least squares method.

[0022] Furthermore, the fitting curve is used to recalculate the lower half of the image. The calculation formula is as follows:

[0023] New_image_curve(i,j)=Original_image(i,j)*

[0024] ave_ratio_up_down_new(1,j)

[0025] Where New_image_curve(i,j) is the processed lower half image curve, Original_image(i,j) is the original image, and ave_ratio_up_down_new(1,j) is the fitted curve.

[0026] Furthermore, the image uniformity is evaluated by calculating the standard deviation (SD) of the grayscale values ​​of the entire image, using the formula:

[0027]

[0028] Where Ii is the pixel value, μ is the mean, and N is the total number of pixels.

[0029] The present invention has the following beneficial effects:

[0030] The present invention uses a ratio fitting curve to correct large-array CMOS column-level noise, eliminating the need to screen chips when purchasing large-array CMOS chips. All column-level noise caused by slope errors can be corrected, significantly saving production costs. After correction using this method, the column-level fixed noise of the CMOS is corrected, ensuring consistent pixel response across the entire array under the same illumination. Smoothing through the fitting curve avoids sudden brightness changes in adjacent columns, significantly improving image quality.

[0031] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The column-level fixed noise diagram for the lower half;

[0033] Figure 2 is the ratio of the mean value of each column in the upper and lower halves of the original image;

[0034] Figure 3 The image after each column is processed separately;

[0035] Figure 4 For CMOS chips, the upper and lower mean values ​​of each column are compared;

[0036] Figure 5 It is the absolute value table of the maximum difference between the ratios of two adjacent columns of pixels on the CMOS chip;

[0037] Figure 6 is the ratio fitting curve graph;

[0038] Figure 7 Processing diagram for fitting curve. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] See also Figure 1-7 The present invention provides a technical solution: a method for correcting column-level fixed pattern noise of a large-array CMOS sensor, comprising the following steps:

[0041] Collect original images from a large-array CMOS sensor under uniform illumination. The test environment includes standard D65 light source, a temperature of 25±2°C, and a humidity of 50%±5%.

[0042] Calculate the ratio of the mean values ​​of each column in the upper and lower parts of the image. Divide the original image into two parts, and calculate the mean value of the upper half (ave_up) and the lower half (ave_down) of each column respectively.

[0043] Calculate the ratio of the mean of the upper half of each column of the original image to the mean of the lower half. The ratio ave_ratio_up_down is calculated as follows:

[0044] ave_ratio_up_down(1,j)=ave_up(1,j) / ave_down(1,j)

[0045] Where ave_up is the mean of each column in the upper half, ave_down is the mean of each column in the lower half, and ave_ratio_up_down is the final ratio.

[0046] The result of ave_ratio_up_down is as follows Figure 2 As shown in the figure, it can be seen that the ratio of each column is different and all are greater than 1, indicating that the mean value of the upper half is larger, that is, the image in the upper half is brighter. Since the columns in the upper half respond uniformly, the entire ratio tends to gradually increase, indicating that the lower half of the image becomes darker from left to right.

[0047] Theoretically, if the image is uniform up and down, the ratio ave_ratio_up_down is 1, so it is necessary to multiply each column of pixels in the lower half by the corresponding ratio to obtain a new image. The formula is as follows:

[0048] New_image(i,j)=Original_image(i,j)*ave_ratio_up_down(1,j)

[0049] Where New_image(i,j) is the processed lower half of the image, Original_image(i,j) is the original image, and ave_ratio_up_down(1,j) is the ratio of each column multiplication. The new image is as follows Figure 3 As shown, each column in the lower half of the new image is uneven, with vertical stripes, and there is a deviation of 1 to 2 brightness values ​​between two adjacent columns.

[0050] Figure 3 It can be seen that the image has no upper and lower dividing lines, and the overall appearance is more uniform. The small picture on the left is the local detail picture after the red frame is magnified, that is, the detail image. From the detail picture, it can be seen that each column in the lower half of the processed image is uneven, with obvious vertical stripes. Because the coefficients of each column image are multiplied by different factors, there may be a deviation of 1 to 2 brightness values ​​between adjacent columns. After the subsequent image enhancement method, the stripes in such an original image will become more serious, and the image quality will become worse.

[0051] The calibration curve fitting method is to fit a smooth curve according to the changing trend of the mean ratio of each column.

[0052] Randomly sample 12 CMOS chips for sampling, calculate the ratio of the upper and lower mean values ​​of each column, and based on the test data of the 12 chips, the root mean square error (RMSE) of the fitting is less than 0.005, ensuring the smoothness of the curve. The absolute value of the maximum difference in the ratio of pixels in two adjacent columns of the 12 chips ranges from 0.0046 to 0.0107, indicating that the original data has little fluctuation and is suitable for smoothing by fitting the curve. Figure 5 As shown in the figure, it can be seen that the change between each column of image data is small, so the ratio of each column will not change drastically. Based on this rule, the present invention proposes a fitting correction curve method for the first time. Through the fitting correction curve method, a curve is fitted according to the change trend of ave_ratio_up_down, so that the change ratio between each column of pixels changes linearly, thereby avoiding the vertical stripe phenomenon. The fitted curve is shown in FIG. Figure 6 As shown, Figure 6 The red color is the original curve ave_ratio_up_down, and the yellow color is the fitted curve ave_ratio_up_down_new.

[0053] The calibration curve fitting method uses quadratic polynomial regression, and the fitting formula is:

[0054] ave_ratio_up_down_new(j)=a*j2+b*j+c

[0055] Among them, a, b, and c are fitting coefficients, which are determined by the least squares method.

[0056] The lower half of the image is corrected column by column using the fitting curve, and each column of pixels in the lower half of the original image is multiplied by the corresponding fitting ratio to obtain the corrected image.

[0057] Use the fitting curve to recalculate the lower half of the image. The calculation formula is as follows:

[0058] New_image_curve(i,j)=Original_image(i,j)*

[0059] ave_ratio_up_down_new(1,j)

[0060] Where New_image_curve(i,j) is the processed lower half image curve, Original_image(i,j) is the original image, and ave_ratio_up_down_new(1,j) is the fitted curve.

[0061] The processed image is as follows Figure 7 As shown, the small picture on the left is the local detail picture after the red frame is enlarged, that is, the detail image, and Figure 3 Zooming in on the details at the same position, we can see from the detail image that the image is uniform, the columns are uniform and consistent, and there is no dividing line between the upper and lower parts of the entire image, which greatly improves the uniformity of the entire image.

[0062] Specifically, image uniformity is evaluated by calculating the standard deviation (SD) of the grayscale values ​​of the entire image, using the formula:

[0063]

[0064] Where Ii is the pixel value, μ is the mean, and N is the total number of pixels.

[0065] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A column-level fixed pattern noise correction method for large-array CMOS sensors, characterized in that: The following steps are involved: Collect original images, collect original images of large-array CMOS sensors under uniform illumination; Calculate the mean ratio of each column in the upper and lower parts of the image. Divide the original image into two parts, and calculate the mean of the upper half (ave_up) and the mean of the lower half (ave_down) of each column respectively. The calibration curve fitting method is to fit a smooth curve according to the changing trend of the mean ratio of each column; The lower half of the image is corrected column by column using the fitting curve, and each column of pixels in the lower half of the original image is multiplied by the corresponding fitting ratio to obtain the corrected image.

2. The method for correcting column-level fixed pattern noise for a large-array CMOS sensor according to claim 1, wherein: Calculate the ratio of the mean of the upper half of each column of the original image to the mean of the lower half. The ratio ave_ratio_up_down is calculated as follows: ave_ratio_up_down(1,j)=ave_up(1,j) / ave_down(1,j) Where ave_up is the mean of each column in the upper half, ave_down is the mean of each column in the lower half, ave_ratio_up_down is the final ratio, and j is the column index.

3. The method for correcting column-level fixed pattern noise for a large-array CMOS sensor according to claim 1, wherein: If the theoretical image is uniform up and down, the ratio ave_ratio_up_down is 1. Multiply each column of pixels in the lower half by the corresponding ratio to get a new image. The formula is as follows: New_image(i,j)=Original_image(i,j)*ave_ratio_up_down(1,j) where New_image(i,j) is the processed lower half image, Original_image(i,j) is the original image, and ave_ratio_up_down(1,j) is the ratio of each column multiplication.

4. The method for correcting column-level fixed pattern noise for a large-array CMOS sensor according to claim 1, wherein: Take 12 CMOS chips and calculate the ratio of the upper and lower mean values ​​in each column.

5. The method for correcting column-level fixed pattern noise for a large-array CMOS sensor according to claim 4, wherein: By fitting the correction curve method, a curve is fitted according to the change trend of ave_ratio_up_down, so that the change ratio between pixels in each column changes linearly.

6. The method for correcting column-level fixed pattern noise for a large-array CMOS sensor according to claim 5, wherein: The calibration curve fitting method uses quadratic polynomial regression, and the fitting formula is: ave_ratio_up_down_new(j)=a*j2+b*j+c Among them, a, b, and c are fitting coefficients, which are determined by the least squares method.

7. The method for correcting column-level fixed pattern noise for a large-array CMOS sensor according to claim 1, wherein: Use the fitting curve to recalculate the lower half of the image. The calculation formula is as follows: New_image_curve(i,j)=Original_image(i,j)* ave_ratio_up_down_new(1,j) Where New_image_curve(i,j) is the processed lower half image curve, Original_image(i,j) is the original image, and ave_ratio_up_down_new(1,j) is the fitted curve.

8. The method for correcting column-level fixed pattern noise for a large-array CMOS sensor according to claim 1, wherein: Image uniformity is evaluated by calculating the standard deviation (SD) of the grayscale values ​​of the entire image using the following formula: Where Ii is the pixel value, μ is the mean, and N is the total number of pixels.

Images