Transverse chromatic aberration correction method

Abstract

The invention provides a transverse chromatic aberration correction method, which is characterized by comprising the following steps of: shooting a checkerboard image to obtain a Raw image, and splitting the Raw image into four small images according to an RGGB color channel; constructing an x-y coordinate system on the small graph, and establishing a first offset preset model of a red channel relative to one green channel and a second offset preset model of a blue channel relative to the other green channel; calculating a first offset of the red channel relative to one green channel, and calculating a second offset of the blue channel relative to the other green channel; calculating coordinates of the center of the lens; calculating a coefficient A1 of the first offset preset model and a coefficient A2 of the second offset preset model, and substituting the coefficients to obtain a first offset model and a second offset model; and performing chromatic aberration correction on the Raw image. The influence of noise and the like on transverse chromatic aberration calibration can be reduced, the center position of the lens is adaptively positioned, the calibration accuracy is improved, and the correction effect is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for correcting lateral chromatic aberration. Background Art

[0002] In optical imaging systems, visible light is refracted when passing through a camera lens. Longer wavelengths result in a greater refractive index, a phenomenon known as dispersion. Because different colors of light have different wavelengths, the different color channels of each pixel cannot be focused on the same point when the image is captured. This causes lateral chromatic aberration (chromatic aberration), a lens-related defect, in the captured image. Lateral chromatic aberration typically occurs at the junction of high and low light levels in an image, and manifests as colored edges, such as "purple fringing" or "green fringing."

[0003] When a digital camera captures light, the lens sensor can only capture light intensity, not color (i.e., wavelength). Currently, color filter arrays (CBFAs) are commonly used, with R, G, and B representing the red, green, and blue filter array elements, respectively. Because human vision is most sensitive to green, the G component in a Bayer CFA (Bayer Pattern Color Filter Array) is typically twice as large as the R and B components, and focusing is typically based on the G channel. When incident light passes through a lens and is focused onto the image plane, the blue, green, and red light rays experience different distance deviations perpendicular to the optical axis. During imaging, lateral chromatic aberration can disrupt image clarity at off-axis points, resulting in color fringing at the edges of black and white patterns. Due to physical properties, lateral chromatic aberration becomes more pronounced as pixels are further from the image center.

[0004] In the prior art, there are two main approaches to color fringing removal. One is to first determine the color fringing location through edge detection and color determination, then reduce its saturation or remove some chromatic aberration through image filtering. However, this approach can lose original image detail and inadvertently damage areas of normal color. The other approach uses calibration followed by correction to correct lateral chromatic aberration. This correction method typically begins with digital image processing. A camera is first used to calibrate an image using a checkerboard pattern. Based on the calibration results, the positional offsets of pixels at different locations in the image are calculated, and the three RGB channels are then aligned. However, in actual calibration, the pixel offsets between different color channels can be very small and not an integer multiple of a pixel. Furthermore, the captured checkerboard image is affected by noise, resulting in unclear edges and difficulty in accurately locating the positions of different color channels. Furthermore, the center of the lens may not necessarily coincide with the center of the image. Existing solutions rarely take these practical factors into account, resulting in inaccurate calibration and suboptimal correction of lateral chromatic aberration. Color retention due to undercorrection or new color fringing due to overcorrection can easily occur. Summary of the Invention

[0005] The present invention aims to provide a method for correcting lateral chromatic aberration that can minimize the impact of external factors such as noise on lateral chromatic aberration calibration, adaptively locate the actual lens center position, improve calibration accuracy, optimize the correction effect, and reduce color retention caused by undercorrection or new color fringing caused by overcorrection.

[0006] In order to achieve the above object, the present invention provides a method for correcting lateral chromatic aberration, which comprises:

[0007] A checkerboard image is captured to obtain a raw image, and the raw image is split into four small images according to RGGB color channels, where the four small images correspond to the red channel, the green channel, the green channel, and the blue channel, respectively;

[0008] Constructing an xy coordinate system on the small image, establishing a first offset preset model of the red channel relative to one of the green channels and a second offset preset model of the blue channel relative to the other green channel, wherein both the first offset preset model and the second offset preset model are related to the coordinates of the lens center;

[0009] Determining edge positions of the checkerboard patterns in the four thumbnails, calculating a first offset of the edge in the thumbnail of the red channel relative to the edge in one thumbnail of the green channel, and calculating a second offset of the edge in the thumbnail of the blue channel relative to the edge in another thumbnail of the green channel;

[0010] Calculating the coordinates of the lens center according to the first offset and the coordinates of the corresponding green channel, and the second offset and the coordinates of the corresponding green channel;

[0011] Calculating a coefficient A1 of the first offset preset model and a coefficient A2 of the second offset preset model according to the coordinates of the lens center, obtaining a first offset model according to the coefficient A1, the coordinates of the lens center, and the first offset preset model, and obtaining a second offset model according to the coefficient A2, the coordinates of the lens center, and the second offset preset model;

[0012] Performing chromatic aberration correction on the Raw image according to the first offset model and the second offset model.

[0013] Optionally, the method for determining the edge positions in the four small images includes: calculating a gradient value for each pixel point in the small image, and finding the edge position according to the gradient value.

[0014] Optionally, the method for calculating the gradient value includes: calculating the gradient value using a Sobel operator, and when the gradient value is lower than a first threshold, setting the gradient value to zero.

[0015] Optionally, the method for finding the edge position includes: for each row / column pixel point of the small image, finding multiple gradient local maximum points in sequence from left to right / from top to bottom, and when the first gradient local maximum point is found, using its x-axis / y-axis coordinate as the x-axis / y-axis coordinate of the current edge, and whenever a new gradient local maximum point is found, if the distance between the newly found gradient local maximum point and the current edge position is less than a third threshold, it is considered that the newly found gradient local maximum point belongs to the current edge, and the newly found gradient local maximum point is added to the set of current edges, and the x-coordinate / y-coordinate mean of each point in the set is calculated and used as the updated edge position, and if the distance between the newly found gradient local maximum point and the current edge position is greater than or equal to the third threshold, it is considered that a new edge is found, and the gradient local maximum point is added to the set of new edges, and each row / column pixel point of the small image is traversed, and the above operation is repeated.

[0016] Optionally, in combination with the gradient value, each edge of the checkerboard pattern is divided into multiple rectangular areas, the coordinates of the gradient maximum value of each row / column of pixels in the rectangular area are found, and the coordinates of the gradient maximum value and the adjacent pixels are weighted averaged by the gradient value to obtain the coordinates of the edge position; the position coordinates of the same edge of each color channel are obtained by the above method, the position coordinates of the edge of a green channel minus the position coordinates of the edge corresponding to the red channel are the first offset, and the position coordinates of the edge of another green channel minus the position coordinates of the edge corresponding to the blue channel are the second offset.

[0017] Optionally, the first offset and the second offset that are greater than the second threshold are discarded, and the mean of the first offset within the same rectangular area and the corresponding coordinate mean on a small image of the green channel are calculated as a red channel offset sample. The mean of the second offset within the same rectangular area and the corresponding coordinate mean on another small image of the green channel are calculated as a blue channel offset sample. The above calculation is performed on all the rectangular areas to obtain multiple red channel offset samples and multiple blue channel offset samples.

[0018] Optionally, abnormal sample screening is performed on all the red channel offset samples and all the blue channel offset samples, and the coordinates of the lens center are calculated based on the screened red channel offset samples and the blue channel offset samples.

[0019] Optionally, the method for screening abnormal samples includes: counting the positive and negative signs of the offset in the red channel offset samples on the same side of the lens center, with the mode being the correct sign, eliminating a small number of abnormal samples whose signs are different from the correct sign, and performing the same operation on the blue channel offset samples; screening all the red channel offset samples and all the blue channel offset samples according to the normal distribution law to remove abnormal samples.

[0020] Optionally, the first offset preset model and the second offset preset model are both quadratic function models. The coordinates of the lens center are calculated by combining the first offset preset model and the second offset preset model using the least squares method, and then the coefficients A1 and A2 are calculated using the least squares method.

[0021] Optionally, the offset of the red channel relative to the green channel and the offset of the blue channel relative to the green channel of each pixel point on the small image are calculated according to the first offset model and the second offset model, and interpolation is performed according to the offset of the red channel relative to the green channel and the offset of the blue channel relative to the green channel, and the interpolation result is the corrected image.

[0022] As configured above, the raw image is split into four small images according to the RGGB color channels, an xy coordinate system is constructed, a first offset preset model and a second offset preset model are established, the edge positions of the checkerboard pattern in the four small images are determined, and the edge offset is calculated. The coordinates of the lens center are then fitted, and the coefficients A1 of the first offset preset model and A2 of the second offset preset model are calculated to complete the calibration of lateral chromatic aberration. Finally, chromatic aberration correction is performed on the raw image based on the first and second offset models. The present invention uses a numerical statistical correlation scheme to minimize the impact of external factors such as noise on lateral chromatic aberration calibration, and adaptively locates the actual lens center position, improving calibration accuracy, thereby optimizing the correction effect and reducing color retention caused by undercorrection or new color fringing caused by overcorrection. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Those skilled in the art will appreciate that the accompanying drawings are provided for a better understanding of the present invention and do not constitute any limitation on the scope of the present invention.

[0024] Figure 1 1 is a flow chart of a method for correcting lateral chromatic aberration according to an embodiment of the present invention;

[0025] Figure 2 FIG. 1 is a schematic diagram of a Bayer array of RGGB according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In this document, unless otherwise specified, the terms "upper", "lower", "left", "right", "inside", "outside", "front", "back", "top", "bottom", etc. are used to indicate directions or positional relationships based on the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction and operation. Therefore, they cannot be understood as limiting the present invention.

[0027] The following is a more detailed description of the specific embodiments of the present invention with reference to schematic diagrams. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the drawings are greatly simplified and not to exact scale, and are only used for the purpose of conveniently and clearly illustrating the embodiments of the present invention.

[0028] Figure 1 This is a flow chart of a method for correcting lateral chromatic aberration according to an embodiment of the present invention. Figure 1 An embodiment of the present invention provides a method for correcting lateral chromatic aberration, which includes at least step S1, step S2, step S3, step S4, step S5 and step S6. Step S1, step S2, step S3, step S4, step S5 and step S6 are described in detail below.

[0029] Step S1: Use the lens module to be calibrated to capture a checkerboard image to obtain a raw image (i.e., a Bayer raw image). Split the raw image into four small images according to the RGGB color channels. The four small images correspond to the red channel, green channel, green channel, and blue channel, respectively. The length and width of each small image are half the length and width of the raw image. It is understood that in the field of image processing technology, small images can also be called downsampled images or subimages. Small images are only used as a name, and the word "small" only means that the size is smaller than that of the raw image.

[0030] Step S2: Construct an xy coordinate system on each thumbnail image to represent the position of each pixel on the thumbnail image using coordinates. Establish a first offset model for the red channel relative to one green channel and a second offset model for the blue channel relative to another green channel. Both the first and second offset models are relative to the coordinates of the lens center.

[0031] Preferably, the first offset preset model and the second offset preset model are both quadratic function models. When the chromatic aberration offset is in the x-direction, the x coordinate of the lens center is one of the coefficients in the first offset preset model / the second offset preset model; when the chromatic aberration offset is in the y-direction, the y coordinate of the lens center is one of the coefficients in the first offset preset model / the second offset preset model.

[0032] Step S3: Determine the edge positions of the checkerboard pattern in the four thumbnails, calculate a first offset of the edge in the red channel thumbnail relative to the edge in one of the green channel thumbnails, and calculate a second offset of the edge in the blue channel thumbnail relative to the edge in another of the green channel thumbnails. It will be appreciated that the checkerboard pattern has multiple horizontal and vertical edges that form the checkerboard pattern.

[0033] Furthermore, the method for determining the edge locations in the four small images includes calculating a gradient value for each pixel in each small image, and finding the approximate location of each edge of the checkerboard pattern based on the gradient value. It is understood that the gradient value is larger at the intersection of black and white (edges) of the checkerboard pattern, while the gradient value is smaller in areas without color change. The method for calculating the gradient value includes calculating the gradient value using a Sobel operator, and setting the gradient value to zero when the gradient value falls below a first threshold.

[0034] Furthermore, the method for finding the edge position includes: for each row / column of pixels in the small image, sequentially finding multiple local gradient maxima from left to right / top to bottom, and when the first local gradient maxima point is found, using its x-axis / y-axis coordinates as the x-axis / y-axis coordinates of the current edge. Whenever a new local gradient maxima point is found, if the distance between the new local gradient maxima point and the current edge position is less than a third threshold, the new local gradient maxima point is considered to belong to the current edge, the new local gradient maxima point is added to the set of current edges, and the x-coordinate / y-coordinate mean of all points in the set is calculated and used as the updated edge position. If the distance between the new local gradient maxima point and the current edge position is greater than or equal to the third threshold, a new edge is considered to have been found, and the new local gradient maxima point is added to the set of new edges. The above operation is repeated for each row / column of pixels in the small image. The edge position found by the above operation is an approximate position.

[0035] Next, a number of red channel offset samples and blue channel offset samples are collected. Combined with the gradient value, each edge of the checkerboard pattern is divided into multiple rectangular areas. For example, the edge at the approximate position found by the above operation is divided into multiple rectangular areas (the rectangular area includes the edge position found above and the area near both sides of the edge). The coordinates of the maximum gradient of each row / column of pixels in the rectangular area are found (if the maximum gradient is less than the first threshold, and it is mentioned above that "when the gradient value is lower than the first threshold, the gradient value is set to zero", so all pixels in the current row / column with a gradient maximum value lower than the first threshold are discarded). The coordinates of the maximum gradient and its adjacent pixels are weighted averaged by the gradient value to obtain the coordinates of the edge position. In this way, for the same edge, each row / column obtains an accurate edge position coordinate. After the position coordinates of the same edge of different color channels are obtained by the above method, the color difference offset can be obtained by subtracting them. Specifically, the position coordinates of the edge of a green channel minus the position coordinates of the corresponding edge of the red channel is the first offset, and the position coordinates of the edge of another green channel minus the position coordinates of the corresponding edge of the blue channel is the second offset. Preferably, the first offset and the second offset that are greater than the second threshold are discarded, and the mean of the first offset within the same rectangular area and the corresponding coordinate mean on a small image of the green channel are calculated as a red channel offset sample. The mean of the second offset within the same rectangular area and the corresponding coordinate mean on another small image of the green channel are calculated as a blue channel offset sample. The above calculation is performed on all rectangular areas to obtain multiple red channel offset samples and multiple blue channel offset samples. It can be understood that when the first offset and the second offset that are greater than the second threshold are discarded, the corresponding coordinates on the small image of the green channel will also be discarded and will not be included in the calculation of the coordinate mean.

[0036] Step S4: Calculate the coordinates of the lens center based on the first offset and its corresponding coordinates on the green channel thumbnail, the second offset and its corresponding coordinates on the green channel thumbnail. Preferably, the coordinates of the lens center can also be calculated based on the red channel offset sample and the blue channel offset sample.

[0037] Preferably, before calculating the coordinates of the lens center, all red channel offset samples and all blue channel offset samples are screened for abnormal samples, and the coordinates of the lens center are calculated based on the screened red channel offset samples and blue channel offset samples. Specifically, the method of screening abnormal samples includes: counting the positive and negative signs of the offsets in the red channel offset samples on the same side of the lens center, with the majority being the correct sign, eliminating a small number of abnormal samples with signs different from the correct sign, and performing the same operation on the blue channel offset samples. It can be understood that, according to the principle of dispersion, the offsets on the same side of the lens center should have the same positive and negative signs; screening all red channel offset samples and all blue channel offset samples according to the normal distribution law to remove abnormal samples. Exemplarily, abnormal samples can be eliminated according to the 2sigma principle in the normal distribution law.

[0038] Step S5: Calculate the coefficient A1 of the first offset preset model and the coefficient A2 of the second offset preset model based on the coordinates of the lens center. Obtain the first offset model based on the coefficient A1, the coordinates of the lens center and the first offset preset model. Obtain the second offset model based on the coefficient A2, the coordinates of the lens center and the second offset preset model.

[0039] Exemplarily, the first offset preset model and the second offset preset model are combined and the coordinates of the lens center are calculated using the least squares method, and then the coefficients A1 and A2 are calculated using the least squares method.

[0040] It is understandable that the signs of the offsets on both sides of the lens center are inconsistent. With the offset as the ordinate and the abscissa still representing the horizontal position coordinate, that is, with the offset as the dependent variable and the horizontal position coordinate as the independent variable, the first offset preset model is a quadratic function with different opening directions but the same shape on both sides of the lens center, and the second offset preset model is a quadratic function with different opening directions but the same shape on both sides of the lens center. The lens center point position has no scattering (that is, the offset is zero). Taking horizontal chromatic aberration as an example, the vertex of the quadratic function is located on the x-axis (the coordinate system here is the aforementioned coordinate system with the offset as the ordinate and the abscissa still representing the horizontal position coordinate). The horizontal coordinate of the lens center is found using the least squares method. After calculating two close / or identical center positions based on the filtered red channel offset samples and blue channel offset samples (the two center positions are one center position of the red channel and one center position of the blue channel), the two center positions are averaged to obtain the final lens center position.

[0041] Step S6: performing chromatic aberration correction on the Raw image according to the first offset model and the second offset model.

[0042] According to the first offset model and the second offset model, the offset of the red channel relative to the green channel and the offset of the blue channel relative to the green channel of each pixel point on the small image are calculated. Interpolation is performed based on the offset of the red channel relative to the green channel and the offset of the blue channel relative to the green channel. The interpolation result is the corrected image.

[0043] As configured above, the raw image is split into four small images according to the RGGB color channels, an xy coordinate system is constructed, a first offset preset model and a second offset preset model are established, the edge positions of the checkerboard pattern in the four small images are determined, and the edge offset is calculated. The coordinates of the lens center are then fitted, and the coefficients A1 of the first offset preset model and A2 of the second offset preset model are calculated to complete the calibration of lateral chromatic aberration. Finally, chromatic aberration correction is performed on the raw image based on the first and second offset models. The present invention uses a numerical statistical correlation scheme to minimize the impact of external factors such as noise on lateral chromatic aberration calibration, and adaptively locates the actual lens center position, improving calibration accuracy, thereby optimizing the correction effect and reducing color retention caused by undercorrection or new color fringing caused by overcorrection.

[0044] A specific embodiment is provided below, which takes the horizontal direction (x-axis direction) chromatic aberration processing as an example:

[0045] Step 1: Establish a chromatic aberration model: Use the lens module to be calibrated to shoot a checkerboard image to obtain a raw image. Split the raw image into four small images according to the RGGB color channels. The length and width of the small images are both half of the original image.

[0046] An xy coordinate system is constructed on each small image to represent the position of each pixel on the small image in the form of coordinates (the x and y coordinates of this coordinate system are both position coordinates). In this embodiment, the origin of the xy coordinate system is set at the upper left corner of the small image to adapt to the scanning direction when the image is processed in the hardware. The scanning direction is from left to right and from top to bottom, the positive direction of the x-axis is rightward, and the positive direction of the y-axis is downward, so as to facilitate calculation. In this embodiment, the horizontal direction (x-axis direction) of the RGGB Bayer pattern (Bayer array) color difference processing is taken as an example. The green channel is set as the first color channel as the reference channel and it is assumed that there is no color difference offset. The red channel is the second color channel and the blue channel is the third color channel. The second and third color channels are respectively based on the green channel in the same column as the reference to establish an offset model relative to the reference channel. You can refer to Figure 2 , Figure 2 Schematic diagram of the RGGB Bayer array, the second color channel (red channel) is based on the green channel in the same column as the second color channel, and the third color channel (blue channel) is based on the green channel in the same column as the third color channel.

[0047] A common model is that the relationship between the absolute value of the offset and the position of the pixel point conforms to a quadratic function, and there is no chromatic aberration at the center of the lens. The first offset preset model / the second offset preset model of this embodiment is:

[0048]

[0049] Where x is the x-coordinate of the pixel, t is the x-axis coordinate of the lens center, and offset represents the x-axis coordinate of the pixel's first color channel minus the x-axis coordinate of the corresponding second or third color channel (in the same column as the first color channel). The signs of the offsets on the left and right sides of the lens center are opposite (for vertical chromatic aberration, the signs of the offsets on the top and bottom sides of the lens center in the small image are opposite). Sign is 1 or -1 depending on the actual color channel (depending on whether the actual color channel is red or blue). The second and third color channels have the same t value but different a values.

[0050] Step 2: Edge positioning and offset calculation can be divided into the following steps:

[0051] Step 201: For each point in the split small image, use the 3x3 Sobel operator to calculate the gradient, and then take the absolute value. Because of the presence of noise, the gradient of some non-edge areas will be larger. Calculate the pixel mean of the current channel (that is, the mean of the pixel values of all pixels in the current channel, the pixel value is easy to obtain) as the first threshold, and set the gradient value below the first threshold to zero. The first threshold is an adaptive threshold. The main body of the checkerboard image is black and white. Using the first threshold can adapt to the ambient brightness when the image is collected and find the edge more accurately. The 3x3 pixels are represented as p1, p2, ..., p9 from left to right and from top to bottom, respectively. The gradient value of the center position p5 is calculated as follows:

[0052] edge(p5)=abs(p1+2×p4+p7-p2-2×p6-p9)

[0053]

[0054] Among them, mean(channel) represents the mean of the pixel values of all pixels in the small image of the current color channel, that is, the first threshold; edge(p5) represents the gradient value of p5; and the abs() function represents the absolute value.

[0055] Step 202: First, roughly determine the edge position and find the approximate rectangular area where each vertical edge is located. The edge positions of different color channels are close. In this embodiment, only the approximate rectangular areas of the two first color channels are calculated. The rectangular area where the edge of the second color channel is located is consistent with the first color channel in the same column as it. The rectangular area where the edge of the third color channel is located is consistent with the first color channel in the same column as it. That is, the left and right dividing lines are consistent with the first color channel in the same column as it.

[0056] Specifically, for each row of the thumbnail in each first color channel, the gradient value is calculated by traversing the pixels from left to right. Mirror padding is used when calculating the gradient at the thumbnail edge. The local maximum gradient point of each row is found, that is, the gradient value is greater than or equal to the two adjacent points on the left and the two adjacent points on the right (in other embodiments, it can also be greater than or equal to the three adjacent points on the left and the three adjacent points on the right, without limitation), and the gradient value is greater than the mean (i.e., the mean of the pixel values of all pixels in the current channel). The x-axis coordinate of the local maximum gradient point in the thumbnail is recorded. For the same vertical edge, the local maximum gradient points obtained by traversing each row will have similar coordinate positions. When the first local maximum gradient point is found, its x-axis coordinate is used as the x-axis coordinate of the current edge. Whenever a new local maximum gradient point is obtained, its x-axis coordinate is first compared with the x-axis coordinate of the current edge. If the distance is less than the set third threshold, it is considered to belong to the same edge. This point is added to the current edge set, and the x-axis coordinate mean of all points in the set is calculated and used as the updated edge position. If the distance is greater than or equal to the set third threshold, it is considered that a new edge has been found, and the local maximum point of the gradient is added to the set of new edges. The above operation is repeated for each row of the small graph.

[0057] Step 203: After obtaining a certain number of approximate edge positions in step 202, each edge set contains multiple local gradient maxima points. The maximum and minimum x-axis coordinates of the local gradient maxima points are the left and right boundaries of the edge rectangular area. The checkerboard grid covers almost the entire image. Ideally, the number of points in each edge should be equal to the image height. Considering the actual image acquisition situation, the effective threshold of the number of points in the edge set is set to half the raw image height (half the raw image width when calculating vertical color difference). If the number of local gradient maxima points contained in the edge set is lower than the effective threshold, the current edge is considered not a regular checkerboard edge and is discarded.

[0058] Step 204: For the set of edges obtained in step 203, further calculate the precise x-axis coordinates of the edges in each small image. Search within the left and right boundaries of each edge (rectangular area), and find a point with a maximum gradient in each row. The coordinates of this point are weighted averaged with the coordinates of the adjacent points (pixels) to the left and right, based on the gradient magnitude, to obtain the final x-axis coordinate. This x-axis coordinate is no longer an integer. Assume that the x-axis coordinate of the pixel corresponding to the maximum gradient is idx, and the gradient value is edge[idx].

[0059] sum edge =edge[idx]+edge[idx+1]+edge[idx-1]

[0060] Where idx+1 represents the x-axis coordinate of the point to the right of the point with the maximum gradient value, idx-1 represents the x-axis coordinate of the point to the left of the point with the maximum gradient value, and idx new is the final x-axis coordinate.

[0061] Combined with the final x-axis coordinates of the points on the edge obtained above, based on the positional difference between the second and third color channels relative to the first color channel on the same edge, the misalignment of the different color channels at the current edge position can be calculated. An offset is calculated for each row, and points with offsets greater than a second threshold are considered abnormal sample points and are removed. For the same vertical edge, it is divided into multiple segments, and each row in each segment has a point corresponding to a final x-axis coordinate. After averaging each segment, an offset sample is obtained (the average value is: the average of all final x-axis coordinates in each segment, and the average of the offsets of all rows in each segment). Each offset sample includes the x-axis coordinate of the edge (after averaging each segment) and the offset of the red channel or blue channel relative to the green channel at that position. In this embodiment, the offset samples are divided into red channel offset samples and blue channel offset samples. The offset of the red channel relative to the green channel is called a red channel offset sample, and the offset of the blue channel relative to the green channel is called a blue channel offset sample.

[0062] Step 3, sample screening: According to the dispersion principle, the offset on the left side of the lens center must have the same sign, and the same sign on the right side, and one is positive and the other is negative. The signs of the offsets in the red channel offset samples and the blue channel offset samples are counted, and the mode is the correct sign. Individual erroneous samples with irregular signs are eliminated. Here, the image center is used as the rough lens center. To avoid accidental damage, statistics and elimination are not performed within a certain area to the left and right of the image center (this range is set based on experience). For each vertical edge, the horizontal distance from the image center is theoretically consistent. All offset samples of each vertical edge of the same color channel theoretically have similar offsets and conform to the normal distribution. First, the x-coordinates of the existing offset samples are scaled down and rounded to the nearest integer. As a result, the positions of the same samples are close. Based on the 2sigma principle, the sample mean is calculated and points with offsets greater than 2 standard deviations from the mean are eliminated, as shown in the following inequality (5).

[0063]

[0064] x idx Indicates the x-coordinate after scaling. Scale indicates the scaling factor, which can be determined based on the actual number of edges. The fewer the edges, the larger the scale. Round indicates that the result is rounded to an integer value. idxEqual offset samples whose number exceeds the fourth threshold are on the same edge. Offset samples that meet the following conditions will be eliminated:

[0065]

[0066]

[0067] Here, mean represents the x-coordinate mean of the remaining offset samples after removing abnormal symbols.

[0068] Step 4: Obtain the lens center position: The offset values on either side of the lens center have different signs. This offset can be fitted to a quadratic function with different opening directions but the same shape. If there is no scattering at the lens center, the offset is 0, and the vertex of the quadratic function must be on the x-axis. Use the least squares method to find the horizontal coordinate T of the lens center and the coefficient a of the quadratic function. After calculating the actual lens center position based on the chromatic aberration of the second and third color channels, take the average of these values to obtain the final horizontal coordinate t of the lens center.

[0069] The least squares method is as follows: let the horizontal coordinate of the current offset sample be x, the offset be s, the left offset of the lens center be positive, and the sample point set be C, or it can be understood as the set C of x coordinates in the offset sample:

[0070]

[0071] Where t0 and t1 are x-axis coordinate thresholds (set empirically) that are symmetrical on either side of the image center. The lens center position searches for the optimal value within this range. st stands for "so that... satisfies..." in mathematics. Equation (6) represents a and t that minimize LOSS. The red channel offset samples and blue channel offset samples (after screening in step 3) are calculated using the above method. The two channels will each obtain two close center values, T1 and T2. The average value is calculated to obtain the final x-coordinate T of the lens center:

[0072]

[0073] Step 5. Solve the model coefficient a: After obtaining the coordinates of the lens center, the quadratic function only has one unknown coefficient a. The least squares method can be used to obtain the a value corresponding to the model.

[0074]

[0075]

[0076] Here, mathematical formula (7) represents a that minimizes LOSS.

[0077] The red channel offset samples and blue channel offset samples (after screening in step 3) are calculated according to the above method respectively. The optimal solution coefficient a=A1 can be obtained for the red channel, and the optimal solution coefficient a=A2 can be obtained for the blue channel. Combined with the x-coordinate T of the lens center calculated in step 4, that is, t=T, the first offset model (corresponding to the red channel) and the second offset model (corresponding to the blue channel) can be obtained by substituting them into formula (1). At this point, the calibration of lateral chromatic aberration is completed.

[0078] Step 6, chromatic aberration correction: After obtaining the first offset model and the second offset model, for each point (pixel) on the small image of the second color channel and the third color channel, you can substitute the first offset model and the second offset model according to its position on the x-axis to obtain the horizontal offset of the second color channel and the third color channel relative to the first color channel, and then perform cubic polynomial interpolation based on the offset. The interpolation result is the corrected image.

[0079] k=ex 3 +bx 2 +cx+d (8)

[0080] x inter =x+offset

[0081] Among them, k represents the pixel value, x inter Indicates the coordinates after the second color channel / third color channel correction. In the same color channel (i.e. the second color channel and the third color channel must each solve the four coefficients e, b, c, d), x inter Substitute the information of the two pixels on the left and right of the position (a total of four pixels) (x1, k1), (x2, k2), (x3, k3), (x4, k4) into formula (8) to obtain its four coefficients e, b, c, d, and then x = x inter Substituting into formula (8) can calculate the pixel value, that is, the second color channel and the third color channel are calculated according to this method, and the corrected pixel value of the red channel and the pixel value of the blue channel are obtained respectively, completing the chromatic aberration correction in the x-axis direction.

[0082] The above is an example of processing horizontal (x-axis) chromatic aberration. The same principle applies to vertical (y-axis) chromatic aberration. For the raw image corrected for x-axis chromatic aberration, the chromatic aberration is calculated and corrected based on the y-axis coordinate. Repeating steps 1-6 will yield the final corrected image, which involves splitting the raw image into four smaller images again.

[0083] It should be noted that references in the specification to "one embodiment," "an embodiment," "a specific embodiment," "some embodiments," etc., merely indicate that the described embodiment may include a particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is within the knowledge of persons skilled in the relevant art to implement such feature, structure, or characteristic in conjunction with other embodiments, regardless of whether such feature, structure, or characteristic is explicitly described.

[0084] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. The systems disclosed in the embodiments are described briefly because they correspond to the methods disclosed in the embodiments. For relevant details, refer to the method description.

[0085] It should also be noted that although the present invention has been disclosed above with reference to preferred embodiments, the above embodiments are not intended to limit the present invention. For any person skilled in the art, without departing from the scope of the technical solution of the present invention, the technical content disclosed above can be used to make many possible changes and modifications to the technical solution of the present invention, or to modify it into an equivalent embodiment with equivalent changes. Therefore, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of protection of the technical solution of the present invention.

[0086] It should also be understood that, unless otherwise specified or indicated, the terms "first", "second", "third", etc. in the specification are only used to distinguish the various components, elements, steps, etc. in the specification, and are not used to indicate the logical relationship or sequential relationship between the various components, elements, steps, etc.

[0087] It should also be understood that the terms described herein are intended to describe particular embodiments only and are not intended to limit the scope of the invention. It should be noted that the singular forms "a" and "an" as used herein and in the appended claims include plural references unless the context clearly indicates otherwise. For example, a reference to "a step" or "a device" means a reference to one or more steps or devices, and may include secondary steps as well as secondary devices. All conjunctions used should be understood in their broadest sense. Also, the word "or" should be understood to have the definition of a logical "or" rather than a logical "exclusive or" unless the context clearly indicates otherwise. Furthermore, implementation of the methods and / or apparatus in embodiments of the present invention may include performing selected tasks manually, automatically, or in combination.

Claims

1. A method for correcting lateral chromatic aberration, characterized in that: include: A checkerboard image is captured to obtain a raw image, and the raw image is split into four small images according to RGGB color channels, where the four small images correspond to the red channel, the green channel, the green channel, and the blue channel, respectively; Constructing an xy coordinate system on the small image, establishing a first offset preset model of the red channel relative to one of the green channels and a second offset preset model of the blue channel relative to the other green channel, wherein both the first offset preset model and the second offset preset model are related to the coordinates of the lens center; Determining edge positions of the checkerboard patterns in the four thumbnails, calculating a first offset of the edge in the thumbnail of the red channel relative to the edge in one thumbnail of the green channel, and calculating a second offset of the edge in the thumbnail of the blue channel relative to the edge in another thumbnail of the green channel; Calculating the coordinates of the lens center according to the first offset and the coordinates of the corresponding green channel, and the second offset and the coordinates of the corresponding green channel; Calculating a coefficient A1 of the first offset preset model and a coefficient A2 of the second offset preset model according to the coordinates of the lens center, obtaining a first offset model according to the coefficient A1, the coordinates of the lens center, and the first offset preset model, and obtaining a second offset model according to the coefficient A2, the coordinates of the lens center, and the second offset preset model; Performing chromatic aberration correction on the Raw image according to the first offset model and the second offset model.

2. The method for correcting lateral chromatic aberration according to claim 1, wherein: The method for determining edge positions in the four small images includes: calculating a gradient value for each pixel point in the small image, and finding the edge position according to the gradient value.

3. The method for correcting lateral chromatic aberration according to claim 2, wherein: The method for calculating the gradient value includes: calculating the gradient value by using a Sobel operator, and when the gradient value is lower than a first threshold, setting the gradient value to zero.

4. The method for correcting lateral chromatic aberration according to claim 2, wherein: The method for finding the edge position includes: for each row / column pixel point of the small image, finding multiple gradient local maximum points from left to right / from top to bottom in sequence, when the first gradient local maximum point is found, its x-axis / y-axis coordinate is used as the x-axis / y-axis coordinate of the current edge, each time a new gradient local maximum point is found, if the distance between the newly found gradient local maximum point and the current edge position is less than a third threshold, then it is considered that the newly found gradient local maximum point belongs to the current edge, the newly found gradient local maximum point is added to the set of current edges, the x-coordinate / y-coordinate mean of each point in the set is calculated and used as the updated edge position, if the distance between the newly found gradient local maximum point and the current edge position is greater than or equal to the third threshold, then it is considered that a new edge is found, the gradient local maximum point is added to the set of new edges, each row / column pixel point of the small image is traversed, and the above operation is repeated.

5. The method for correcting lateral chromatic aberration according to claim 2, wherein: In combination with the gradient value, each edge of the checkerboard pattern is divided into multiple rectangular areas, the coordinates of the gradient maximum value of each row / column of pixels in the rectangular area are found, and the coordinates of the gradient maximum value and the adjacent pixels are weighted averaged using the gradient value to obtain the coordinates of the edge position; the position coordinates of the same edge of each color channel are obtained using the above method, and the position coordinates of the edge of a green channel minus the position coordinates of the corresponding edge of the red channel are the first offset, and the position coordinates of the edge of another green channel minus the position coordinates of the corresponding edge of the blue channel are the second offset.

6. The method for correcting lateral chromatic aberration according to claim 5, wherein: The first offset and the second offset that are greater than the second threshold are discarded, and the mean of the first offset within the same rectangular area and the corresponding coordinate mean on a small image of the green channel are calculated as a red channel offset sample. The mean of the second offset within the same rectangular area and the corresponding coordinate mean on another small image of the green channel are calculated as a blue channel offset sample. The above calculation is performed on all the rectangular areas to obtain multiple red channel offset samples and multiple blue channel offset samples.

7. The method for correcting lateral chromatic aberration according to claim 6, wherein: Abnormal sample screening is performed on all the red channel offset samples and all the blue channel offset samples, and the coordinates of the lens center are calculated according to the screened red channel offset samples and the blue channel offset samples.

8. The method for correcting lateral chromatic aberration according to claim 7, wherein: The method for screening abnormal samples includes: counting the positive and negative signs of the offsets in the red channel offset samples on the same side of the lens center, with the mode being the correct sign, eliminating a small number of abnormal samples whose signs are different from the correct sign, and performing the same operation on the blue channel offset samples; and screening all the red channel offset samples and all the blue channel offset samples according to the normal distribution law to remove abnormal samples.

9. The method for correcting lateral chromatic aberration according to claim 1, wherein: The first offset preset model and the second offset preset model are both quadratic function models. The coordinates of the lens center are calculated by combining the first offset preset model and the second offset preset model using the least squares method, and then the coefficients A1 and A2 are calculated using the least squares method.

10. The method for correcting lateral chromatic aberration according to claim 1, wherein: According to the first offset model and the second offset model, the offset of the red channel relative to the green channel and the offset of the blue channel relative to the green channel of each pixel point on the small image are calculated, and interpolation is performed according to the offset of the red channel relative to the green channel and the offset of the blue channel relative to the green channel. The interpolation result is the corrected image.

Images