A method for correcting photoresponse non-uniformity on CMOS image sensors

Abstract

The method and device for correcting the non-uniformity of light response on a COMS image sensor provided in an embodiment of the present invention can calculate the offset of the image at different saturations offline during the non-uniformity correction process. During online calculation, only a standard correction matrix needs to be known. The correction matrix parameters at the saturation can be obtained through the offset at different saturations. In the correction calculation process, linear interpolation, nonlinear interpolation, and the nearest neighbor principle are introduced to calibrate the standard parameter matrix to increase the correction accuracy. It not only effectively solves the problem of horizontal and vertical stripes in CMOS images and improves image quality, but also reduces the storage capacity of correction parameters and other problems, greatly improving the accuracy of non-uniformity correction. The present invention has the advantages of simple calculation process, low memory overhead, good software migration, and high hardware portability for the image sensor chip imaging system.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to a method for correcting light response non-uniformity on a COMS image sensor. Background Art

[0002] Due to the limitation of integrated circuit technology and other objective factors, the CMOS image sensor itself also has nonlinear characteristics. Under the combined influence of these factors, the light response of the image sensor is non-uniform, and cross lines formed by the intersection of a horizontal stripe and a vertical stripe appear in the CMOS image, which seriously affects the quality of the acquired image. In order to improve the image quality, the software non-uniformity correction algorithm is an important part, and the hardware calculation overhead is an important consideration. Therefore, if a balance can be achieved between the correction effect and the calculation overhead, it can not only ensure the correction accuracy but also reduce the cost, which is more in line with the actual engineering needs.

[0003] Traditional non-uniform correction algorithms based on calibration methods mainly include single-point correction method and multi-point correction method. Although the single-point correction method is easy to implement and has a small amount of calculation, the correction effect is poor. The correction accuracy of the multi-point correction algorithm is related to the length of its segmented intervals. The more segmented intervals, the better the correction effect, but the more correction parameters are required, which greatly increases the hardware storage capacity and is very difficult to implement in engineering applications.

[0004] The existing technology has high hardware cost and consumes huge memory. Taking the traditional multi-point correction method as an example, if there are five intervals, five sets of correction parameters are required. The more segmented intervals the multi-point correction algorithm has, the more correction parameters are required, which consumes huge memory, requires more calculations, and has huge calculation overhead. Summary of the invention

[0005] In view of this, embodiments of the present invention provide a method, an apparatus, a computer device, and a readable storage medium for correcting light response non-uniformity on a CMOS image sensor.

[0006] In a first aspect, an embodiment of the present invention provides a method for correcting light response non-uniformity on a CMOS image sensor, comprising:

[0007] The original image of the CMOS image sensor is collected to obtain n groups of saturation values ​​of φ1, φ2, φ3, ... φ n The image of , and the dark field image of the corresponding image, n is a positive integer;

[0008] Superimposing n groups of dark field images, removing singular values ​​in the time domain, and then performing mean calculation to obtain a dark field mean matrix;

[0009] The n groups of collected images with different saturation are superimposed respectively, and after removing the singular values ​​in the time domain, the mean value is calculated to obtain the mean matrix of different saturation;

[0010] Subtracting the dark field mean matrix from the different saturation mean matrices to obtain a full-frame matrix and the grayscale value of each pixel of the full-frame matrix;

[0011] Performing mean processing on the full-frame matrix and dividing the mean value by the grayscale value of each pixel in the full-frame matrix to obtain n groups of first correction matrices, and selecting any one of the n groups of first correction matrices as a reference correction matrix;

[0012] For the n groups of images with different saturation levels, respectively determine the pixel means of all rows of pixels above the intersection line and all rows of pixels below the intersection line of the first correction matrix;

[0013] The saturation φ1 group and other saturation pixel means of the k rows on the upper side of the intersection line are respectively subtracted to obtain the corresponding n-1 groups of first offsets offset1, and the saturation φ1 group and other saturation pixel means of the k rows on the lower side of the intersection line are respectively subtracted to obtain the n-1 groups of second offsets offset2;

[0014] A first offset offset1 is added to each pixel value of the matrix on the side of the cross line of the reference correction matrix, and a second offset offset2 is added to each pixel value of the matrix on the side of the cross line to obtain φ2, φ3, ... φ n A second correction matrix for saturation;

[0015] Based on φ2, φ3, ... φ n The first correction matrix and the second correction matrix of saturation respectively calculate the average of the k columns of pixels on the left side of the intersection line and the average of the k columns of pixels on the right side of the intersection line. k is a positive integer;

[0016] Determine the third offset offset3 and the fourth offset offset4 by using the average value of the k columns of pixels on the left side of the crossing line and the average value of the k columns of pixels on the right side of the crossing line;

[0017] Determine a dark field image of an input image, and determine to perform a subtraction operation between the input image and the dark field image of the input image;

[0018] Traversing each pixel point, using the introduction of linear interpolation method, nonlinear interpolation method, nearest neighbor principle, and calibration of standard parameter matrix to judge the area where the pixel is located, the input image is divided into four areas of upper left, upper right, lower left, and lower right according to the cross dividing line;

[0019] Obtaining a parameter param of a reference correction matrix of the area where the pixel is located, and determining pixel correction parameters of the four areas respectively using a first offset, a second offset, a third offset, and a fourth offset;

[0020] The correction pixel is determined by using the pixel and the pixel correction parameters of the four areas to complete the correction.

[0021] As an optional solution, obtaining the parameter param of the reference correction matrix of the area where the pixel is located, and using the first offset, the second offset, the third offset and the fourth offset to respectively determine the pixel correction parameters of the four areas include:

[0022] The parameter param of the reference correction matrix of the area where the pixel is located is known, and the four offsets under the saturation are a first offset offset1, a second offset offset2, a third offset offset3, and a fourth offset offset4;

[0023] The pixel correction parameters for the upper left area are param-offset1-offset3;

[0024] The pixel correction parameters in the upper right area are param-offset1–offset4;

[0025] The pixel correction parameter in the lower left area is param–offset2-offset3;

[0026] The pixel correction parameters in the lower right area are param–offset2–offset4.

[0027] As an optional solution, the determining the correction pixel by using the pixel and the pixel correction parameters of the four regions includes:

[0028] A multiplication operation is performed using the pixel and the pixel correction parameters of the four regions to obtain a corrected pixel.

[0029] In a second aspect, an embodiment of the present invention provides a device for correcting non-uniformity of light response on a COMS image sensor, including a correction parameter acquisition unit and a correction calculation unit;

[0030] The correction parameter acquisition unit is used to perform the following steps:

[0031] The original image of the CMOS image sensor is collected to obtain n groups of saturation values ​​of φ1, φ2, φ3, ... φ n The image of , and the dark field image of the corresponding image, n is a positive integer;

[0032] Superimposing n groups of dark field images, removing singular values ​​in the time domain, and then performing mean calculation to obtain a dark field mean matrix;

[0033] The n groups of collected images with different saturation are superimposed respectively, and after removing the singular values ​​in the time domain, the mean value is calculated to obtain the mean matrix of different saturation;

[0034] Subtracting the dark field mean matrix from the different saturation mean matrices to obtain a full-frame matrix and the grayscale value of each pixel of the full-frame matrix;

[0035] Performing mean processing on the full-frame matrix and dividing the mean value by the grayscale value of each pixel in the full-frame matrix to obtain n groups of first correction matrices, and selecting any one of the n groups of first correction matrices as a reference correction matrix;

[0036] For the n groups of images with different saturation levels, respectively determine the pixel means of all rows of pixels above the intersection line and all rows of pixels below the intersection line of the first correction matrix;

[0037] The saturation φ1 group and other saturation pixel means of the k rows on the upper side of the intersection line are respectively subtracted to obtain the corresponding n-1 groups of first offsets offset1, and the saturation φ1 group and other saturation pixel means of the k rows on the lower side of the intersection line are respectively subtracted to obtain the n-1 groups of second offsets offset2;

[0038] A first offset offset1 is added to each pixel value of the matrix on the side of the cross line of the reference correction matrix, and a second offset offset2 is added to each pixel value of the matrix on the side of the cross line to obtain φ2, φ3, ... φ n A second correction matrix for saturation;

[0039] Based on φ2, φ3, ... φ n The first correction matrix and the second correction matrix of saturation respectively calculate the average of the k columns of pixels on the left side of the intersection line and the average of the k columns of pixels on the right side of the intersection line. k is a positive integer;

[0040] Determine the third offset offset3 and the fourth offset offset4 by using the average value of the k columns of pixels on the left side of the crossing line and the average value of the k columns of pixels on the right side of the crossing line;

[0041] The correction calculation unit is used to perform the following steps:

[0042] Determine a dark field image of an input image, and determine to perform a subtraction operation between the input image and the dark field image of the input image;

[0043] Traversing each pixel point, using the introduction of linear interpolation method, nonlinear interpolation method, nearest neighbor principle, and calibration of standard parameter matrix to judge the area where the pixel is located, the input image is divided into four areas of upper left, upper right, lower left, and lower right according to the cross dividing line;

[0044] Obtaining a parameter param of a reference correction matrix of the area where the pixel is located, and determining pixel correction parameters of the four areas respectively using a first offset, a second offset, a third offset, and a fourth offset;

[0045] The correction pixel is determined by using the pixel and the pixel correction parameters of the four areas to complete the correction.

[0046] As an optional solution, the correction calculation unit is specifically used to perform the following steps:

[0047] The parameter param of the reference correction matrix of the area where the pixel is located is known, and the four offsets under the saturation are a first offset offset1, a second offset offset2, a third offset offset3, and a fourth offset offset4;

[0048] The pixel correction parameters for the upper left area are param-offset1-offset3;

[0049] The pixel correction parameters in the upper right area are param-offset1–offset4;

[0050] The pixel correction parameter in the lower left area is param–offset2-offset3;

[0051] The pixel correction parameters in the lower right area are param–offset2–offset4.

[0052] As an optional solution, the correction calculation unit is specifically used to perform the following steps:

[0053] A multiplication operation is performed using the pixel and the pixel correction parameters of the four regions to obtain a corrected pixel.

[0054] In a third aspect, an embodiment of the present invention provides a computer device, including:

[0055] at least one processor; and

[0056] a memory communicatively connected to the at least one processor; wherein,

[0057] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned method for correcting the non-uniformity of light response on a CMOS image sensor.

[0058] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the above-mentioned method for correcting the non-uniformity of light response on a COMS image sensor.

[0059] The method and device for correcting the non-uniformity of light response on a COMS image sensor provided in an embodiment of the present invention can calculate the offset of the image at different saturations offline during the non-uniformity correction process. During online calculation, only a standard correction matrix needs to be known. The correction matrix parameters at the saturation can be obtained through the offset at different saturations. In the correction calculation process, linear interpolation, nonlinear interpolation, and the nearest neighbor principle are introduced to calibrate the standard parameter matrix to increase the correction accuracy. It not only effectively solves the problem of horizontal and vertical stripes in CMOS images and improves image quality, but also reduces the storage capacity of correction parameters and other problems, greatly improving the accuracy of non-uniformity correction. The present invention has the advantages of simple calculation process, low memory overhead, good software migration, and high hardware portability for the image sensor chip imaging system. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A flowchart of a method for correcting light response non-uniformity on a COMS image sensor is provided in an embodiment of the present invention;

[0061] Figure 2 A schematic flow chart of a correction parameter acquisition process in a method for correcting light response non-uniformity on a COMS image sensor is provided in an embodiment of the present invention;

[0062] Figure 3 A schematic flow chart of a correction calculation process in a method for correcting non-uniformity of light response on a CMOS image sensor is provided in an embodiment of the present invention;

[0063] Figure 4 A schematic structural diagram of a device for correcting light response non-uniformity on a CMOS image sensor is provided in an embodiment of the present invention;

[0064] Figure 5 A structural block diagram of a computer device is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0066] The terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0067] Combination Figure 1 As shown, an embodiment of the present invention provides a method for correcting light response non-uniformity on a CMOS image sensor, comprising:

[0068] S101, collect original images from the CMOS image sensor to obtain n groups of saturation values ​​of φ1, φ2, φ3, ... φ n The image of , and the dark field image of the corresponding image, n is a positive integer;

[0069] S102, superimposing n groups of the dark field images, removing singular values ​​in the time domain, and then performing mean calculation to obtain a dark field mean matrix;

[0070] S103, respectively superimposing the n groups of images with different saturations collected, removing the singular values ​​in the time domain, and then performing mean calculation to obtain mean matrices of different saturations;

[0071] S104, using the different saturation mean matrices to subtract the dark field mean matrix to obtain a full-frame matrix and the grayscale value of each pixel in the full-frame matrix;

[0072] S105, performing mean processing on the full-frame matrix and dividing the mean value with the grayscale value of each pixel in the full-frame matrix to obtain n groups of first correction matrices, and selecting any one of the n groups of first correction matrices as a reference correction matrix;

[0073] S106, for the n groups of images with different saturation, respectively determining the pixel means of all rows of pixels above the intersection line and all rows of pixels below the intersection line of the first correction matrix;

[0074] S107: the saturation φ1 group and other saturation pixel means of the k rows on the upper side of the intersection line are respectively subtracted to obtain the corresponding n-1 groups of first offsets offset1, and the saturation φ1 group and other saturation pixel means of the k rows on the lower side of the intersection line are respectively subtracted to obtain the n-1 groups of second offsets offset2;

[0075] S108, add a first offset offset1 to each pixel value on the upper side of the cross line of the reference correction matrix, and add a second offset offset2 to each pixel value on the lower side of the cross line, to obtain φ2, φ3, ... φ n A second correction matrix for saturation;

[0076] S109, based on φ2, φ3, ... φ n The first correction matrix and the second correction matrix of saturation respectively calculate the average of the k columns of pixels on the left side of the intersection line and the average of the k columns of pixels on the right side of the intersection line. k is a positive integer;

[0077] S110, determining a third offset offset3 and a fourth offset offset4 by using the average value of the k columns of pixels on the left side of the crossing line and the average value of the k columns of pixels on the right side of the crossing line;

[0078] S111, determining a dark field image of an input image, and performing a subtraction operation between the input image and the dark field image of the input image;

[0079] S112, traversing each pixel point, using the introduction of linear interpolation method, nonlinear interpolation method, nearest neighbor principle, and calibration standard parameter matrix to judge the area where the pixel is located, and dividing the input image into four areas of upper left, upper right, lower left, and lower right according to the cross dividing line;

[0080] S113, obtaining a parameter param of a reference correction matrix of the area where the pixel is located, and determining pixel correction parameters of the four areas respectively using a first offset, a second offset, a third offset and a fourth offset;

[0081] S114. Determine a correction pixel using the pixel and the pixel correction parameters of the four regions to complete the correction.

[0082] Among them, steps S101 to S110 are the correction parameter acquisition process S1, and steps S111 to S114 are the correction calculation process S2. In the process of correction parameter acquisition, the offset of the image at different saturations is calculated offline. In the process of correction calculation, we introduce linear interpolation, nonlinear interpolation, and the nearest neighbor principle to calibrate the standard parameter matrix and increase the correction accuracy.

[0083] In some possible embodiments, in S113, obtaining the parameter param of the reference correction matrix of the area where the pixel is located, and using the first offset, the second offset, the third offset and the fourth offset to respectively determine the pixel correction parameters of the four areas include:

[0084] The parameter param of the reference correction matrix of the area where the pixel is located is known, and the four offsets under the saturation are a first offset offset1, a second offset offset2, a third offset offset3, and a fourth offset offset4;

[0085] The pixel correction parameters for the upper left area are param-offset1-offset3;

[0086] The pixel correction parameters in the upper right area are param-offset1–offset4;

[0087] The pixel correction parameter in the lower left area is param–offset2-offset3;

[0088] The pixel correction parameters in the lower right area are param–offset2–offset4.

[0089] In some possible embodiments, in S114, determining the corrected pixel by using the pixel and the pixel correction parameters of the four regions includes:

[0090] A multiplication operation is performed using the pixel and the pixel correction parameters of the four regions to obtain a corrected pixel.

[0091] The method for correcting the non-uniformity of the light response on the COMS image sensor provided by the present invention is based on multi-point correction as a whole. The essential difference is that the method of the present invention requires fewer correction parameters and less memory overhead. For example, under the same segmentation conditions, multi-point correction requires n correction parameter matrices, while our method only requires 1 correction parameter matrix and n-1 offset matrices. The effect is more obvious in scenes with large image resolution and many segments.

[0092] The advantage of the present invention is that during the process of non-uniformity correction, we can calculate the offset of the image at different saturations offline. During online calculation, we only need to know a standard correction matrix, and the correction matrix parameters at the saturation can be obtained through the offset at different saturations.

[0093] The present invention includes two processes: a correction parameter acquisition process and a correction calculation process. In the correction parameter acquisition process, the offset of the image at different saturations is calculated offline. In the correction calculation process, we introduce linear interpolation, nonlinear interpolation, and the nearest neighbor principle to calibrate the standard parameter matrix and increase the correction accuracy.

[0094] In order to facilitate understanding of the present solution, the following introduces two processes, namely, a correction parameter acquisition process and a correction calculation process, for the light response non-uniformity correction method on a COMS image sensor provided by the present invention.

[0095] Combination Figure 2 As shown, the process of obtaining the correction parameters includes the following steps:

[0096] Step 1: Collect the original image of the CMOS image sensor to obtain n groups of saturation φ1, φ2, φ3, ... φ n , and the dark field image of the corresponding image;

[0097] Step 2: Superimpose the collected dark field images, remove the singular values ​​in the time domain, and then perform mean calculation;

[0098] Step 3: Superimpose the n groups of images with different saturation values ​​collected, remove the singular values ​​in the time domain, and then calculate the average value;

[0099] Step 4: Use the different saturation mean matrices of step 3 to subtract the dark field mean matrix of step 2 to obtain the full-frame matrix;

[0100] Step 5: After taking the average value of the full-frame matrix in step 4, divide it by the grayscale value of each pixel in the full-frame matrix in step 4 to obtain n sets of correction matrices;

[0101] Step 6: Among the n sets of matrices in step 5, select one set of correction matrices as the standard correction matrix;

[0102] Step 7: Calculate the pixel mean of all rows above and below the intersection line of the correction matrix for the n groups of images with different saturations in step 5;

[0103] Step 8: The saturation φ1 group of k rows on the upper side of the intersection line and the other saturation pixel means are subtracted respectively to obtain the corresponding n-1 group offsets offset1, and the saturation φ1 group of k rows on the lower side of the intersection line and the other saturation pixel means are subtracted respectively to obtain the n-1 group offsets offset2;

[0104] Step 9: Add offset1 to each pixel on the upper side of the cross line of the reference correction matrix, and add offset2 to each pixel on the lower side of the cross line to obtain φ2, φ3, ... φ n Saturation correction matrix;

[0105] Step 10: Based on saturation φ2, φ3, ... φ n The correction matrix and the correction matrix obtained in step nine are used to calculate the mean values ​​of the pixels in the k columns on the left side of the intersection line and the k columns on the right side of the intersection line respectively;

[0106] Step 11: Subtract the calculation results obtained in step 10 to obtain offset offset3 and offset offset4.

[0107] Combination Figure 3 As shown, the correction calculation process includes the following steps:

[0108] Step 1: Find the dark field image of the input image;

[0109] Step 2: Subtract the input image from the dark field image;

[0110] Step 3: Traverse each pixel point, introduce linear interpolation, nonlinear interpolation, nearest neighbor principle, and calibrate the standard parameter matrix;

[0111] Step 4: According to the pixel area, the image is divided into four areas: upper left, upper right, lower left, and lower right according to the cross-border line;

[0112] Step 6: Given the reference correction matrix parameter param of the pixel position, the four offsets under the saturation are offset1, offset2, offset3, and offset4;

[0113] Step 5: The pixel correction parameters in the upper left area are param-offset1-offset3;

[0114] Step 6: The pixel correction parameters in the upper right area are param-offset1–offset4;

[0115] Step 7: The pixel correction parameter of the lower left area is param–offset2-offset3;

[0116] Step 8: The pixel correction parameters in the lower right area are param–offset2–offset4;

[0117] Step 9: Multiply the input pixel by the pixel correction parameter of each area to obtain the corrected pixel.

[0118] The method for correcting the non-uniformity of light response on a CMOS image sensor provided by the embodiment of the present invention has the following beneficial effects:

[0119] (1) The algorithm of the present invention is improved and accelerated based on the multi-point correction method, which not only effectively solves the horizontal and vertical stripes problem of CMOS images and improves the image quality, but also reduces the storage amount of correction parameters and other problems, and greatly improves the accuracy of non-uniformity correction;

[0120] (2) In the process of non-uniformity correction, the present invention can calculate the offset of the image at different saturations offline. During online calculation, only a standard correction matrix needs to be known, and the correction matrix parameters at the saturation can be obtained through the offset at different saturations.

[0121] (3) In the process of correction calculation, linear interpolation, nonlinear interpolation and nearest neighbor principle are introduced to calibrate the standard parameter matrix and increase the correction accuracy.

[0122] Compared with other methods, the biggest advantage of the present invention is that it not only effectively solves the problem of horizontal and vertical stripes in CMOS images and improves image quality, but also reduces the storage capacity of correction parameters and other issues. The present invention achieves a balance between correction effect and computational overhead, ensuring correction accuracy and reducing costs, which is more in line with actual engineering needs.

[0123] Combination Figure 4 As shown, accordingly, an embodiment of the present invention provides a device for correcting non-uniformity of light response on a COMS image sensor, comprising a correction parameter acquisition unit 401 and a correction calculation unit 402;

[0124] The calibration parameter acquisition unit 401 is used to perform the following steps:

[0125] The original image of the CMOS image sensor is collected to obtain n groups of saturation values ​​of φ1, φ2, φ3, ... φ n The image of , and the dark field image of the corresponding image, n is a positive integer;

[0126] Superimposing n groups of dark field images, removing singular values ​​in the time domain, and then performing mean calculation to obtain a dark field mean matrix;

[0127] The n groups of collected images with different saturation are superimposed respectively, and after removing the singular values ​​in the time domain, the mean value is calculated to obtain the mean matrix of different saturation;

[0128] Subtracting the dark field mean matrix from the different saturation mean matrices to obtain a full-frame matrix and the grayscale value of each pixel of the full-frame matrix;

[0129] Performing mean processing on the full-frame matrix and dividing the mean value by the grayscale value of each pixel in the full-frame matrix to obtain n groups of first correction matrices, and selecting any one of the n groups of first correction matrices as a reference correction matrix;

[0130] For the n groups of images with different saturation levels, respectively determine the pixel means of all rows of pixels above the intersection line and all rows of pixels below the intersection line of the first correction matrix;

[0131] The saturation φ1 group and other saturation pixel means of the k rows on the upper side of the intersection line are respectively subtracted to obtain the corresponding n-1 groups of first offsets offset1, and the saturation φ1 group and other saturation pixel means of the k rows on the lower side of the intersection line are respectively subtracted to obtain the n-1 groups of second offsets offset2;

[0132] A first offset offset1 is added to each pixel value of the matrix on the side of the cross line of the reference correction matrix, and a second offset offset2 is added to each pixel value of the matrix on the side of the cross line to obtain φ2, φ3, ... φ n A second correction matrix for saturation;

[0133] Based on φ2, φ3, ... φ n The first correction matrix and the second correction matrix of saturation respectively calculate the average of the k columns of pixels on the left side of the intersection line and the average of the k columns of pixels on the right side of the intersection line. k is a positive integer;

[0134] Determine the third offset offset3 and the fourth offset offset4 by using the average value of the k columns of pixels on the left side of the crossing line and the average value of the k columns of pixels on the right side of the crossing line;

[0135] The correction calculation unit 402 is used to perform the following steps:

[0136] Determine a dark field image of an input image, and determine to perform a subtraction operation between the input image and the dark field image of the input image;

[0137] Traversing each pixel point, using the introduction of linear interpolation method, nonlinear interpolation method, nearest neighbor principle, and calibration of standard parameter matrix to judge the area where the pixel is located, the input image is divided into four areas of upper left, upper right, lower left, and lower right according to the cross dividing line;

[0138] Obtaining a parameter param of a reference correction matrix of the area where the pixel is located, and determining pixel correction parameters of the four areas respectively using a first offset, a second offset, a third offset, and a fourth offset;

[0139] The correction pixel is determined by using the pixel and the pixel correction parameters of the four areas to complete the correction.

[0140] In some possible embodiments, the correction calculation unit 402 is specifically configured to perform the following steps:

[0141] The parameter param of the reference correction matrix of the area where the pixel is located is known, and the four offsets under the saturation are a first offset offset1, a second offset offset2, a third offset offset3, and a fourth offset offset4;

[0142] The pixel correction parameters for the upper left area are param-offset1-offset3;

[0143] The pixel correction parameters in the upper right area are param-offset1–offset4;

[0144] The pixel correction parameter in the lower left area is param–offset2-offset3;

[0145] The pixel correction parameters in the lower right area are param–offset2–offset4.

[0146] In some possible embodiments, the correction calculation unit 402 is specifically configured to perform the following steps:

[0147] A multiplication operation is performed using the pixel and the pixel correction parameters of the four regions to obtain a corrected pixel.

[0148] The method and device for correcting the non-uniformity of light response on a COMS image sensor provided in an embodiment of the present invention can calculate the offset of the image at different saturations offline during the non-uniformity correction process. During online calculation, only a standard correction matrix needs to be known. The correction matrix parameters at the saturation can be obtained through the offset at different saturations. In the correction calculation process, linear interpolation, nonlinear interpolation, and the nearest neighbor principle are introduced to calibrate the standard parameter matrix to increase the correction accuracy. It not only effectively solves the problem of horizontal and vertical stripes in CMOS images and improves image quality, but also reduces the storage capacity of correction parameters and other problems, greatly improving the accuracy of non-uniformity correction. The present invention has the advantages of simple calculation process, low memory overhead, good software migration, and high hardware portability for the image sensor chip imaging system.

[0149] Accordingly, according to an embodiment of the present invention, the present invention also provides a computer device, a readable storage medium and a computer program product.

[0150] Figure 5 FIG. 1 is a schematic diagram of the structure of a computer device 12 provided in an embodiment of the present invention. Figure 5 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 5 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0151] like Figure 5 As shown, computer device 12 is in the form of a general-purpose computing device. Computer device 12 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0152] Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16 , a system memory 28 , and a bus 18 that connects various system components including system memory 28 and processing unit 16 .

[0153] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0154] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0155] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the embodiments of the present invention.

[0156] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0157] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface 22. In addition, the computer device 12 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0158] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the light response non-uniformity correction method on the CMOS image sensor provided by the embodiment of the present invention.

[0159] An embodiment of the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, on which a computer program is stored, wherein when the program is executed by a processor, the method for correcting the light response non-uniformity on the COMS image sensor provided in all the inventive embodiments of the present application is implemented.

[0160] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, an apparatus, or a device.

[0161] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0162] The program code included in the computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages ​​such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer by any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect to the Internet).

[0163] An embodiment of the present invention further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned method for correcting the non-uniformity of light response on a CMOS image sensor.

[0164] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document is not limited here.

[0165] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for correcting the non-uniformity of light response on a COMS image sensor, characterized in that: include: The original image of the CMOS image sensor is collected to obtain n groups of saturation Superimposing n groups of dark field images, removing singular values ​​in the time domain, and then performing mean calculation to obtain a dark field mean matrix; The n groups of collected images with different saturation are superimposed respectively, and after removing the singular values ​​in the time domain, the mean value is calculated to obtain the mean matrix of different saturation; Subtracting the dark field mean matrix from the different saturation mean matrices to obtain a full-frame matrix and the grayscale value of each pixel of the full-frame matrix; Performing mean processing on the full-frame matrix and dividing the mean value by the grayscale value of each pixel in the full-frame matrix to obtain n groups of first correction matrices, and selecting any one of the n groups of first correction matrices as a reference correction matrix; For the n groups of images with different saturation levels, respectively determining the mean values ​​of all rows of pixels above the intersection line and all rows of pixels below the intersection line of the first correction matrix; The saturation of the k rows on the upper side of the intersection line Each pixel value of the matrix on the side of the cross line of the reference correction matrix is ​​respectively added with the first offset offset1, and each pixel value of the matrix on the side of the cross line is respectively added with the second offset offset2, so as to obtain based on Determine a third offset offset3 and a fourth offset offset4 by using the average value of the k columns of pixels on the left side of the crossing line and the average value of the k columns of pixels on the right side of the crossing line; Determine a dark field image of an input image, and perform a subtraction operation on the input image and the dark field image of the input image; Traversing each pixel point, updating the reference correction matrix by introducing a linear interpolation method, a nonlinear interpolation method or a nearest neighbor principle and judging the area where the pixel is located, dividing the input image into four areas of upper left, upper right, lower left and lower right according to the cross dividing line; Obtaining a parameter param of a reference correction matrix of the area where the pixel is located, and determining pixel correction parameters of the four areas respectively using a first offset offset1, a second offset offset2, a third offset offset3 and a fourth offset offset4; The correction pixel is determined by using the pixel and the pixel correction parameters of the four areas to complete the correction.

2. The method for correcting the non-uniformity of light response on a COMS image sensor according to claim 1, characterized in that: The step of obtaining a parameter param of a reference correction matrix of a region where the pixel is located, and respectively determining pixel correction parameters of the four regions using a first offset offset1, a second offset offset2, a third offset offset3, and a fourth offset offset4 includes: The parameter param of the reference correction matrix of the area where the pixel is located is known, and the corresponding four offsets are found by subtracting the pixel intensity of the dark field image from the input image. The four offsets are a first offset offset1, a second offset offset2, a third offset offset3, and a fourth offset offset4; The pixel correction parameters for the upper left area are param - offset1 - offset3; The pixel correction parameters for the upper right area are param - offset1 - offset4; The pixel correction parameters for the lower left area are param – offset2 – offset3; The pixel correction parameters for the lower right area are param – offset2 – offset4.

3. The method for correcting the non-uniformity of light response on a COMS image sensor according to claim 1 or 2, characterized in that: The determining the correction pixel by using the pixel and the pixel correction parameters of the four regions comprises: A multiplication operation is performed using the pixel and the pixel correction parameters of the four regions to obtain a corrected pixel.

4. A device for correcting the non-uniformity of light response on a COMS image sensor, characterized in that: It includes a correction parameter acquisition unit and a correction calculation unit; The correction parameter acquisition unit is used to perform the following steps: The original image of the CMOS image sensor is collected to obtain n groups of saturation Superimposing n groups of dark field images, removing singular values ​​in the time domain, and then performing mean calculation to obtain a dark field mean matrix; The n groups of collected images with different saturation are superimposed respectively, and after removing the singular values ​​in the time domain, the mean value is calculated to obtain the mean matrix of different saturation; Subtracting the dark field mean matrix from the different saturation mean matrices to obtain a full-frame matrix and the grayscale value of each pixel of the full-frame matrix; Performing mean processing on the full-frame matrix and dividing the mean value by the grayscale value of each pixel in the full-frame matrix to obtain n groups of first correction matrices, and selecting any one of the n groups of first correction matrices as a reference correction matrix; For the n groups of images with different saturation levels, respectively determining the mean values ​​of all rows of pixels above the intersection line and all rows of pixels below the intersection line of the first correction matrix; The saturation of the k rows on the upper side of the intersection line Each pixel value of the matrix on the side of the cross line of the reference correction matrix is ​​respectively added with the first offset offset1, and each pixel value of the matrix on the side of the cross line is respectively added with the second offset offset2, so as to obtain based on Determine a third offset offset3 and a fourth offset offset4 by using the average value of the k columns of pixels on the left side of the crossing line and the average value of the k columns of pixels on the right side of the crossing line; The correction calculation unit is used to perform the following steps: Determine a dark field image of an input image, and perform a subtraction operation on the input image and the dark field image of the input image; Traversing each pixel point, updating the reference correction matrix by introducing a linear interpolation method, a nonlinear interpolation method or a nearest neighbor principle and judging the area where the pixel is located, dividing the input image into four areas of upper left, upper right, lower left and lower right according to the cross dividing line; Obtaining a parameter param of a reference correction matrix of the area where the pixel is located, and determining pixel correction parameters of the four areas respectively using a first offset offset1, a second offset offset2, a third offset offset3 and a fourth offset offset4; The correction pixel is determined by using the pixel and the pixel correction parameters of the four areas to complete the correction.

5. The device for correcting the non-uniformity of light response on a COMS image sensor according to claim 4, characterized in that: The correction calculation unit is specifically used to perform the following steps: The parameter param of the reference correction matrix of the pixel region is known, and the corresponding four offsets are found by subtracting the pixel intensity of the dark field image from the input image, which are the first offset offset1, the second offset offset2, the third offset offset3, and the fourth offset offset4; The pixel correction parameters for the upper left area are param - offset1 - offset3; The pixel correction parameters for the upper right area are param - offset1 - offset4; The pixel correction parameters for the lower left area are param – offset2 – offset3; The pixel correction parameters for the lower right area are param – offset2 – offset4.

6. The device for correcting the non-uniformity of light response on a COMS image sensor according to claim 4 or 5, characterized in that: The correction calculation unit is specifically used to perform the following steps: A multiplication operation is performed using the pixel and the pixel correction parameters of the four regions to obtain a corrected pixel.

7. A computer device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for correcting photoresponse non-uniformity on a COMS image sensor according to any one of claims 1 to 3.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method for correcting photoresponse non-uniformity on a CMOS image sensor according to any one of claims 1 to 3.

Images