Lens shading correction method, system, electronic device and storage medium

Abstract

The present application discloses a lens shading correction method, system, electronic device and storage medium, and relates to the field of machine vision. The method comprises: obtaining an image to be corrected; based on a preset coordinate conversion model, converting the pixel coordinates of each pixel point in the image to be corrected to obtain a first coordinate; based on a correction coefficient model, calculating the function value of the first coordinate as the correction coefficient of the corresponding pixel point in the image to be corrected; correcting the pixel value in the image to be corrected based on the correction coefficient to obtain a corrected image. Based on the above processing, the shading correction scheme provided by the present application does not need to be recalibrated under a uniform light source. Through coordinate conversion and using the correction coefficient model of a calibrated imaging module, lens shading correction can be achieved, which expands the scope of application of the shading correction scheme.

Description

Technical Field

[0001] The present application relates to the field of machine vision, and in particular to a lens shading correction method, system, electronic device and storage medium. Background Art

[0002] For an imaging module formed by a camera and a lens, if the light passes through the lens and shines vertically on the central area of ​​the image sensor in the camera, the light shines on other areas of the image sensor at a certain angle. Due to the process of the image sensor, the image sensor has different responsiveness to light at different incident angles. Specifically, the greater the incident angle of the light, the weaker the responsiveness, resulting in a bright middle and dark surrounding in the generated image, i.e., a shadow problem.

[0003] Existing shadow correction methods usually require a uniform light source and bind the camera and lens together for correction, so that the generated set of correction coefficients can only be used for a specific camera and lens combination.

[0004] In the field of industrial machine vision, users usually purchase cameras and lenses separately, so that the factory-calibrated imaging module will face the problem of shadow correction again after the user replaces the lens. However, since most users' working scenes do not have a standard environment with uniform light source, they are unable to correct the shadow problem autonomously.

[0005] Therefore, a lens shading correction method is urgently needed to solve the above problems. Summary of the invention

[0006] The present application proposes a lens shading correction method, system, electronic device and storage medium, which are used to realize shading correction under non-uniform light source for an imaging module after lens replacement.

[0007] To achieve the above objectives, this application proposes the following technical solutions:

[0008] In a first aspect of the present application, a lens shading correction method is provided, comprising:

[0009] Acquire an image to be corrected; wherein the image to be corrected represents an image generated by an imaging module to be corrected; the imaging module includes a camera and a lens;

[0010] Based on a preset coordinate transformation model, transform the pixel coordinates of each pixel point in the image to be corrected to obtain a first coordinate;

[0011] Based on the correction coefficient model, the function value of the first coordinate is calculated as the correction coefficient of the corresponding pixel point in the image to be corrected; wherein the correction coefficient model is generated by a calibrated imaging module; the calibrated imaging module is the same as the camera in the imaging module to be corrected;

[0012] Based on the correction coefficients, the pixel values ​​in the image to be corrected are corrected to obtain a corrected image.

[0013] Optionally, the coordinate transformation model includes: a linear function model of the first coordinate and the pixel coordinate in the horizontal direction and the vertical direction respectively; wherein, the parameters in the linear function model are determined based on the offset of the optical center point and the scaling factor; the optical center point represents the pixel point corresponding to the optical axis of the lens in the imaging module; the scaling factor is the ratio of the image distance of the calibrated imaging module to the image distance of the imaging module to be calibrated.

[0014] Optionally, the coordinate transformation model includes:

[0015] x'=k u (x+x u );

[0016] y'=k u (y+y u );

[0017] Where x represents the coordinate of the pixel point in the horizontal direction; y represents the coordinate of the pixel point in the vertical direction; x' represents the converted coordinate in the horizontal direction; y' represents the converted coordinate in the vertical direction; x u Indicates the displacement of the optical center point between the calibrated imaging module and the imaging module to be calibrated in the horizontal direction; y u Indicates the displacement of the optical center point between the corrected imaging module and the imaging module to be corrected in the vertical direction; k u Indicates the zoom ratio.

[0018] Optionally, the coordinate transformation model corresponds to the color channel one by one, and the coordinate transformation models corresponding to different color channels have different scaling factors to be compatible with the correction of incident light of different wavelengths.

[0019] Optionally, the correction coefficient model also includes:

[0020] f(x,y)=f h (xx c )×f v (yy c );

[0021] Where f(x,y) represents the correction coefficient corresponding to the pixel point (x,y); c Indicates the horizontal coordinate of the optical center; y c Indicates the coordinate of the optical center in the vertical direction; f h (xx c ) is expressed as a polynomial fitting model based on the horizontal coordinates, (xx c ) is the input independent variable; f v (yy c) is represented as a polynomial fitting model based on the vertical coordinates, (yy c ) is the input independent variable.

[0022] Optionally, only function terms of even degree are retained in the polynomial fitting model.

[0023] Optionally, the calibration process of the correction coefficient model includes:

[0024] Obtain a standard image generated under a uniform light source;

[0025] The standard image is divided into multiple regions, and the ratio of the maximum pixel mean in all regions to the pixel mean in each region is calculated as the pixel ratio;

[0026] A quadratic function is fitted to the pixel ratio and horizontal coordinate in each area, and the horizontal coordinate corresponding to the extreme value in the fitted function is used as the horizontal coordinate x of the optical center. c ;

[0027] A quadratic function is fitted to the pixel ratio and vertical coordinate in each area, and the coordinate corresponding to the extreme value in the fitted function is used as the vertical coordinate y of the optical center. c ;

[0028] Based on the pixel ratio and coordinates in each area, the remaining parameters in the correction coefficient model are fitted; wherein, in the process of fitting the remaining parameters in the correction coefficient model, the loss function is a nonlinear optimization algorithm.

[0029] Optionally, when the camera is a linear array camera with a single row of pixels, in the parameter fitting process of the correction coefficient model, each pixel point is regarded as a single area.

[0030] In a second aspect of the present application, a lens shading correction system is provided, comprising:

[0031] An image acquisition module is used to acquire an image to be corrected; wherein the image to be corrected represents an image generated by an imaging module to be corrected; the imaging module includes a camera and a lens;

[0032] A coordinate conversion module, used for converting the pixel coordinates of each pixel point in the image to be corrected based on a preset coordinate conversion model to obtain a first coordinate;

[0033] A coefficient calculation module, used to calculate the function value of the first coordinate based on the correction coefficient model as the correction coefficient of the corresponding pixel point in the image to be corrected; wherein the correction coefficient model is generated by the calibrated imaging module; the calibrated imaging module is the same as the camera in the imaging module to be corrected;

[0034] The pixel correction module is used to correct the pixel values ​​in the image to be corrected based on the correction coefficient to obtain the corrected image.

[0035] In a third aspect of the present application, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0036] Memory, used to store computer programs;

[0037] The processor is used to implement the lens shading correction method described in any one of the first aspects when executing the program stored in the memory.

[0038] In a fourth aspect of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the lens shading correction method described in any one of the first aspects is implemented.

[0039] The beneficial effects of this application are as follows:

[0040] The present application provides a lens shading correction method, comprising: acquiring an image to be corrected; wherein the image to be corrected represents an image generated by an imaging module to be corrected; the imaging module comprises a camera and a lens; based on a preset coordinate conversion model, converting the pixel coordinates of each pixel point in the image to be corrected to obtain a first coordinate; based on a correction coefficient model, calculating a function value of the first coordinate as a correction coefficient for a corresponding pixel point in the image to be corrected; wherein the correction coefficient model is generated by a calibrated imaging module; the calibrated imaging module is the same as the camera in the imaging module to be corrected; based on the correction coefficient, correcting the pixel values ​​in the image to be corrected to obtain a corrected image.

[0041] Based on the above processing, the present application uses a coordinate transformation model to transform the coordinates of pixels in the image to be corrected, which is used to correct the adverse effects caused by the optical center offset and image distance under different lenses, and then calculates the correction coefficient corresponding to the first coordinate through the correction coefficient model generated by the calibrated imaging module to achieve shadow correction in the image to be corrected.

[0042] Therefore, for the imaging module after the lens is replaced, the shadow correction scheme provided in the present application does not need to be recalibrated under a uniform light source. Lens shadow correction can be achieved through coordinate transformation and the use of the correction coefficient model of the calibrated imaging module, thereby expanding the scope of application of the shadow correction scheme. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0044] Figure 1 is a flow chart of a lens shading correction method provided by the present application;

[0045] Figure 2 It is a schematic diagram of the principle of coordinate transformation provided by this application;

[0046] Figure 3 It is a schematic diagram of the principle of an imaging formula provided by this application;

[0047] Figure 4 is a structural diagram of a lens shading correction system provided by the present application;

[0048] Figure 5 It is a structural diagram of an electronic device provided by this application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0050] Lens Shading Correction (LSC) is an important part of image signal processing (ISP), which is usually used to correct the uneven brightness of the image caused by the optical characteristics of the lens. Lens shading is mainly divided into luma shading and color shading. The former is brighter in the center of the image and darker around it, while the latter is inconsistent in color between the center and the surroundings.

[0051] During the lens shading correction process, the camera and lens need to be bound together and the corresponding correction coefficient model needs to be calibrated for the same set of imaging modules. However, the calibrated correction coefficient model can only be used for the above imaging modules.

[0052] In the application scenarios of industrial machine vision, the camera and lens in the imaging module are usually delivered independently, and different lenses are required for different scenarios. Specifically, during the factory calibration process, the lens used by the camera manufacturer is different from the lens actually used by the user, so the manufacturer cannot solve the lens shadow problem encountered by the user in advance. At the same time, the user's work scene usually does not have the calibration conditions of uniform light source, which makes it impossible to recalibrate the corresponding correction coefficient model for the imaging module after the lens is replaced.

[0053] For the imaging module after lens replacement, this application proposes a lens shading correction method to achieve shading correction under non-uniform light source. Figure 1 As shown, the method comprises the following steps:

[0054] S1. Obtain an image to be corrected.

[0055] The image to be corrected refers to an image generated by an imaging module to be corrected. The imaging module includes a camera and a lens.

[0056] S2. Based on a preset coordinate transformation model, transform the pixel coordinates of each pixel point in the image to be corrected to obtain a first coordinate.

[0057] S3. Based on the correction coefficient model, calculate the function value of the first coordinate as the correction coefficient of the corresponding pixel point in the image to be corrected.

[0058] The correction coefficient model is obtained by pre-calibrating a calibrated imaging module; the calibrated imaging module is identical to the camera in the imaging module to be calibrated.

[0059] S4. Correct the pixel values ​​in the image to be corrected based on the correction coefficients to obtain a corrected image.

[0060] Based on the above processing, the present application uses a coordinate transformation model to transform the coordinates of pixels in the image to be corrected, which is used to correct the influencing factors of optical center point offset and image distance under different lenses, and calculates the correction coefficient under the first coordinate by the correction coefficient model corresponding to the calibrated imaging module to achieve shadow correction in the image to be corrected.

[0061] Therefore, for the imaging module after the lens is replaced, the shadow correction scheme provided in the present application does not need to be recalibrated under a uniform light source. Lens shadow correction can be achieved through coordinate transformation and the use of the correction coefficient model of the calibrated imaging module, thereby expanding the application scope of the shadow correction scheme.

[0062] For step S1, the imaging module represents an imaging system including a camera and a lens, which is divided into a calibrated imaging module and an imaging module to be calibrated. Among them, for an imaging system composed of a camera and a lens, if the shadow correction has been completed (that is, the correction coefficient model has been calibrated), it is called a calibrated imaging module. In the technical solution provided in the present application, when the lens in the calibrated imaging module is replaced, the replaced imaging module is used as the imaging module to be calibrated.

[0063] In addition, for a calibrated imaging module, if the internal lens optical axis or image distance changes, the changed imaging module can also be used as a module to be calibrated, and can also be calibrated using the lens shading correction solution provided in the present application.

[0064] For step S2, the working principle of the coordinate transformation process is described as follows:

[0065] If the calibrated imaging module includes camera 1 and lens 1, the imaging module to be calibrated includes camera 1 and lens 2. Simplify the lens into a pinhole imaging model, such as Figure 2 As shown, assuming that the optical center of lens 1 is located at A and the optical center of lens 2 is located at A ’ , under the same light source, for the incident light passing through the optical center and with the same incident angle, the pixel points B and B ’ , their light angles are consistent and their shadow correction coefficients are the same, such as Figure 2 As shown, line segment AB is parallel to line segment A. ’ B ’ Therefore, the shading correction coefficients of lens 1 and lens 2 are scaled proportionally with respect to the pixel coordinates. The optical center represents the position of the optical axis in the lens.

[0066] At the same time, the position of the optical center under different lenses will also shift, so the offset of the optical center point also needs to be corrected in the coordinate transformation model.

[0067] In some embodiments, the coordinate transformation model is represented as a combination of linear function models of the first coordinate and the pixel coordinate in the horizontal direction and the vertical direction, respectively.

[0068] Specifically, the combination of linear function models can be: x'=ax+b1; y'=ay+b2;

[0069] Among them, x represents the coordinate of the pixel point in the horizontal direction; y represents the coordinate of the pixel point in the vertical direction; x' represents the converted coordinate in the horizontal direction; y' represents the converted coordinate in the vertical direction; a represents the magnification parameter; b1 represents the constant parameter of the linear function model in the horizontal direction; b2 represents the constant parameter of the linear function model in the vertical direction.

[0070] In one implementation, the parameters in the linear function model (including parameters a, b1 and b2) are determined based on the offset of the optical center point and the zoom factor, wherein the optical center point represents the pixel point corresponding to the optical axis of the lens in the imaging module.

[0071] In one implementation, the ratio of the image distance between the calibrated imaging module and the imaging module to be calibrated is calculated as the zoom factor.

[0072] In some embodiments, the coordinate transformation model may be:

[0073] x'=k u (x+x u );

[0074] y'=k u (y+y u );

[0075] Where x represents the coordinate of the pixel point in the horizontal direction; y represents the coordinate of the pixel point in the vertical direction; x' represents the converted coordinate in the horizontal direction; y' represents the converted coordinate in the vertical direction; x u Indicates the displacement of the optical center point between the calibrated imaging module and the imaging module to be calibrated in the horizontal direction; y u Indicates the displacement of the optical center point between the corrected imaging module and the imaging module to be corrected in the vertical direction; k u Indicates the zoom ratio.

[0076] The pinhole imaging model is as follows Figure 3 As shown, combined with the aforementioned working principle of the coordinate transformation model, it can be known that under different lenses, for pixel points with the same shading correction coefficient, their pixel coordinates are scaled proportionally, and the scaling factor (i.e., the zoom factor) is the ratio between the image distances under different lenses.

[0077] It can be understood that the parameter a of different coordinate transformation models is equivalent to the parameter k u , parameters b1 and b2 are equivalent to k u × u and k u ×y u .

[0078] The lens shading correction scheme provided in the present application can also be applied to the chromaticity correction process in color scenes. In one implementation, the coordinate transformation model corresponds to the color channel one by one, and the coordinate transformation models corresponding to different color channels have different scaling factors to be compatible with the correction of incident light of different wavelengths. When the camera is a color camera, coordinate transformation models with different scaling factors are set for different color channels. Among them, Bayer CFA is currently more commonly used in color cameras, including four color channels of G, R, B, and G. For this situation, four coordinate transformation models with different scaling factors need to be constructed.

[0079] For chromaticity correction, due to the different refractive indices of light of different wavelengths, some poor quality lenses may have dispersion problems, so different k values ​​can be set for different color channels. u parameters to further optimize lens compatibility.

[0080] In some embodiments, the lens shading correction scheme provided by the present application can also be applied to a linear array camera or a TDI (Time Delay Integration) camera with a single row of pixels. In this case, it is only necessary to process the lens shading correction in a single direction. Specifically, the linear array camera only needs to consider the shading correction in the direction parallel to the pixel row, or the TDI camera does not need to process the shading correction in the TDI superposition direction.

[0081] For the parameter k in the coordinate transformation model u、 x cu and cu For solving the parameters such as , this application also provides different solutions.

[0082] For zoom factor k u , the image distance v can be solved using the imaging formula: 1 / u+1 / v=1 / f. Among them, u represents the object distance, which can be obtained by measuring the distance between the lens and the object being measured, and f represents the focal length, which can be obtained by directly reading the imaging parameters in the imaging module.

[0083] Then, the ratio between the image distance v1 of the calibrated imaging module and the image distance v2 of the imaging module to be calibrated is calculated as the scaling factor k. u That is, k u =v1 / v2.

[0084] Regarding the offset of the optical center point, in one implementation, it can be calculated from the optical center point of the imaging module to be corrected. For example, under the same shooting conditions, the imaging module to be corrected and the corrected imaging module are allowed to shoot the same area of ​​the object to be measured, and generate two images with the same resolution. Compare the pixel values ​​in the two images, select the area with the same pixel value as the image generated by the corrected imaging module in the image generated by the imaging module to be corrected, and select the central pixel point of the area as the optical center point of the imaging module to be corrected. Then, calculate the pixel offset from the optical center point in the corrected imaging module to obtain x u and u .

[0085] In actual work, the method of multiple cyclic adjustments is usually adopted to fit the parameters of the above coordinate transformation model. The fitting method includes the following steps:

[0086] Step 1: Use the imaging module to be calibrated to acquire a reference image.

[0087] Step 2: Determine whether the reference image is bilaterally symmetrical. If so, do not adjust x. u If not, adjust the parameter x based on the preset adjustment step size. u The initial value of is set, and the process returns to step 1 until the reference image is bilaterally symmetrical.

[0088] And determine whether the reference image is symmetrical up and down. If so, do not adjust y u If not, adjust the parameter y based on the preset adjustment step size. u The initial value of is set, and the process returns to step 1 until the reference image is symmetrical from top to bottom.

[0089] Step 3: Determine whether the quality score of the reference image is greater than a preset threshold. If so, the parameter adjustment is completed and the adjusted parameter value is used as the fitting parameter of the coordinate transformation model. If not, adjust the parameter k based on the preset adjustment step size. u And return to step 1 until the quality score of the reference image is greater than the preset threshold.

[0090] Among them, k u The initial value of x is 1, u ,y u The initial value of is 0.

[0091] For step 2, the methods for determining whether the image is symmetrical include:

[0092] Judgment method 1: In step 1, take a photo of the same object under test (e.g., calibration plate) in the four corners of the image to generate a reference image. Then, measure whether the pixel mean values ​​of the four corners of the reference image are the same to determine whether it is vertically symmetrical or horizontally symmetrical.

[0093] Specifically, if the grayscale values ​​of the four corners are lower on the left and higher on the right in the horizontal direction, then based on the preset step size, reduce x u Similarly, if the grayscale values ​​of the four corners are low at the top and high at the bottom in the vertical direction, based on the preset step size, reduce y u The initial value of . Among them, in the technical solution provided by the present application, the pixel coordinate system takes the vertex of the image as the coordinate origin, the number of pixel columns is the coordinate in the X-axis direction (i.e., the horizontal direction), and the number of pixel rows is the coordinate in the Y-axis direction (i.e., the vertical direction). In addition, the adjustment step sizes of different parameters may be different.

[0094] Judgment method 2: Rotate the imaging module to be calibrated 180 degrees and then shoot to generate a new reference image. Compare the local area means of the four corners with the reference image before rotation to determine whether it is symmetrical and adjust the initial value of the parameter.

[0095] Specifically, if compared with the reference image before rotation, in the horizontal direction, the grayscale values ​​of the two left corners decrease, and the grayscale values ​​of the two right corners increase, then based on the preset step size, x is reduced. u If the grayscale values ​​of the upper two corners decrease and the grayscale values ​​of the lower two corners increase in the vertical direction, then based on the preset step size, reduce y u The initial value of .

[0096] For step 3, the quality score can be obtained by calculating the average of the image scores provided by multiple users. In step 3, the shading correction effect of the lens in the imaging module to be corrected is optimized by adjusting the parameter ku.

[0097] Based on the above processing, it can be seen that the solution provided in this application can realize lens shading correction in the user's usage scenario without the user having a uniform light source. Different lenses can be adapted by adjusting the parameters in the preset coordinate transformation model in the camera.

[0098] For step S3, the correction coefficient model represents a model for calculating the correction coefficient based on the coordinates of the pixel points. In some embodiments, the correction coefficient model includes: a high-order fitting model based on the distance between the pixel point and the optical center point, and a lookup table containing a one-to-one correspondence between the pixel point coordinates and the correction coefficients.

[0099] The construction of the high-order fitting model is as follows: the lens shadow becomes more and more serious from the optical center of the image to the surrounding area. Based on the principle of central symmetry, the distance between the pixel point and the optical center point is taken as the radius R and as the sampling point. Then these sampling points are fitted into a high-order curve by high-order fitting, that is, the high-order fitting model is obtained. Among them, the Lagrange interpolation method can be used to interpolate to obtain the high-order fitting model, and then the distance R between each pixel point and the center of the image is substituted into the fitted high-order fitting model, and the corresponding correction coefficient can be calculated for shadow correction.

[0100] The construction process of the lookup table is as follows: Take an image under uniform light conditions as the reference image for correction. Compare the actual image taken with the ideal flat lighting image and calculate a correction table. The correction table can be represented as a one-dimensional or two-dimensional lookup table, where each element in the table represents the brightness correction coefficient for the corresponding pixel position. Afterwards, the correction table is converted into a lookup table (LUT) for fast lookup and application during real-time image processing.

[0101] In actual work, the pixels in the image sensor may have different characteristics in the horizontal and vertical directions in response to the angle of light, which makes the output of the correction coefficient model based on the radius R inaccurate and the correction effect needs to be improved. The correction coefficient model based on the lookup table provides a one-to-one correspondence between the correction coefficients and the pixels. Although it can ensure the accuracy of the correction coefficients, it requires a large data storage space and cannot meet the purpose of realizing lens shading correction in hardware.

[0102] In order to solve the problems faced by the above correction coefficient model, the present application also provides a new correction coefficient model, including: f(x, y) = f h (xx c )×f v (yy c );

[0103] Where f(x,y) represents the correction coefficient corresponding to the pixel point (x,y); c Indicates the horizontal coordinate of the optical center; y cIndicates the coordinate of the optical center in the vertical direction; f h (xx c ) is expressed as a polynomial fitting model based on the horizontal coordinates, (xx c ) is the input independent variable; f v (yy c ) is represented as a polynomial fitting model based on the vertical coordinates, (yy c ) is the input independent variable.

[0104] Based on the above processing, the present application provides a correction coefficient model that is calibrated for the horizontal and vertical directions respectively. The correction coefficients provided by the model are more in line with the characteristics of the image sensor's response to light angles, effectively improving the effect of lens shading correction.

[0105] In one implementation, the polynomial fitting model f provided in this application h ( ) and the polynomial fitting model f v In ( ), only function terms with even degrees are retained.

[0106] Based on the above processing, the number of parameters to be fitted in the polynomial fitting model can be reduced, which not only reduces the amount of calculation in the model fitting process and shortens the time of the parameter fitting process, but also reduces the risk of falling into the local optimal value in the parameter fitting process, ensuring the accuracy of the final fitting parameters. In addition, for the horizontal or vertical direction, the response of the image sensor pixel to the light angle is symmetrical relative to the sensor normal direction (i.e., the lens optical axis). It can be seen that retaining only even-numbered terms in the function terms does not affect the accuracy of the output results of the polynomial fitting model.

[0107] The correction coefficient model is explained below using a quadratic polynomial as an example.

[0108] First, the correction coefficient model of the 4th-order polynomial is:

[0109]

[0110] Among them, k h1 , k h2 and k h3 represents the parameters to be fitted in the horizontal direction, k v1 , k v2 and k v3 Indicates the parameters to be fitted in the vertical direction.

[0111] Secondly, the parameter fitting process of the correction coefficient model includes the following steps:

[0112] Step a: obtain a standard image generated under a uniform light source.

[0113] Step b: divide the standard image into multiple regions, and calculate the ratio of the maximum pixel mean in all regions to the pixel mean in each region as the pixel ratio.

[0114] Step c: Fit a quadratic function to the pixel ratio and horizontal coordinate in each area, and use the horizontal coordinate corresponding to the extreme value in the fitted function as the horizontal coordinate x of the optical center. c .

[0115] Step d: Fit a quadratic function to the pixel ratio and vertical coordinate in each area, and use the coordinates corresponding to the extreme values ​​in the fitted function as the vertical coordinates y of the optical center. c .

[0116] For step a, after the camera is matched with the lens to form an imaging module, a uniform light source is photographed to obtain a reference image. Preferably, by selecting a suitable lens, the grayscale value in the center of the reference image is 1.3 to 3 times the grayscale value of the four corners, and the light source brightness and exposure time are adjusted so that the grayscale values ​​of the darkest and brightest areas in the reference image are within the grayscale value range of 20% to 80%.

[0117] Based on the above range setting, it is possible to avoid the grayscale value of the reference image being too uniform, prevent the inability to fit an accurate angle response curve (ie, a polynomial fitting model), and avoid the four corners being too dark, so as to ensure the accuracy of the calculation results after the area division in step b.

[0118] For step b, the reference image is divided into N×N regions, where N can be 16, 32, 64, or 128, for ease of hardware implementation. When calculating the pixel mean in each region, the pixels in each region are added up and then directly shifted to obtain the pixel mean without actually performing division.

[0119] The coordinates of the physical center of each region are marked as (x i , y i ), the pixel ratio m is calculated according to the following formula i .

[0120]

[0121] Among them, x i Indicates the horizontal coordinate of the area, y i Represents the vertical coordinate of the region, i\in \left [ {1,{N}^{2}} \right ] .

[0122] For step c and step d, since the optical center point in the lens shadow is the brightest, the extreme point can be found by quadratic function fitting to determine the optical center point xc and c .

[0123] Specifically, take all the areas where the horizontal center line passes through in the reference image and transform them into i With m i Perform quadratic function fitting and find the x corresponding to the extreme value of the quadratic function i value, which is x c .

[0124] And, take all the areas where the vertical center line of the reference image passes through, and divide them into i With m i Perform quadratic function fitting and find the y corresponding to the extreme value of the quadratic function i Value, that is, y c .

[0125] In some embodiments, the parameter fitting process further comprises the following steps:

[0126] Step e: fitting the coefficients of the polynomial fitting model based on the pixel ratio and coordinates in each region, wherein in the fitting process, a nonlinear optimization algorithm is used to obtain the function value of the loss function.

[0127] For step e, x in all regions i , y i , m i As input data, the 6 parameters k in the f(x,y) function are fitted by nonlinear optimization algorithms (such as Newton method, LM algorithm, etc.) h1 , k h2 ,k h3 , k v1 , k v2 and k v3 .

[0128] When the solution provided in the present application performs color shading correction, it is necessary to perform parameter fitting on different color channels in the color image based on the correction coefficient model to obtain different fitting parameters to adapt to different color channels.

[0129] When the camera in the present application is a linear array camera with a single row of pixels, in the parameter fitting process of the correction coefficient model, each pixel point is used as a single area to complete the parameter fitting. Based on the above processing, the number of areas can be increased to improve the accuracy of parameter fitting.

[0130] In addition, the correction coefficient model corresponding to a line array camera with a single row of pixels only needs to include a model in a single direction (horizontal or vertical).

[0131] For step S4, the product of the correction coefficient and the pixel value in the image to be corrected is calculated as the corrected pixel value, thereby obtaining the corrected image. Specifically, the corrected image data is calculated by the following formula.

[0132] I'(x,y)=I(x,y)×f(x',y')

[0133] Wherein, I'(x,y) represents the pixel value of the pixel in the corrected image, I(x,y) represents the pixel value of the pixel in the image to be corrected, and f(x',y') represents the correction coefficient corresponding to the pixel (x',y').

[0134] In some embodiments, the present application also provides a lens shading correction system, such as Figure 4 As shown, the correction system includes:

[0135] The image acquisition module 401 is used to acquire the image to be calibrated; wherein the image to be calibrated represents an image generated by the imaging module to be calibrated; the imaging module includes a camera and a lens.

[0136] The coordinate conversion module 402 is used to convert the pixel coordinates of each pixel point in the image to be corrected based on a preset coordinate conversion model to obtain a first coordinate.

[0137] The coefficient calculation module 403 is used to calculate the function value of the first coordinate based on the correction coefficient model as the correction coefficient of the corresponding pixel point in the image to be corrected; wherein the correction coefficient model is generated by the calibrated imaging module during the shadow correction process; the calibrated imaging module is the same as the camera in the imaging module to be corrected.

[0138] The pixel correction module 404 is used to correct the pixel values ​​in the image to be corrected based on the correction coefficient to obtain a corrected image.

[0139] The present application also provides an electronic device, such as Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504.

[0140] Memory 503, used for storing computer programs;

[0141] The processor 501 is used to implement any of the above-mentioned lens shading correction methods when executing the program stored in the memory 503.

[0142] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0143] The communication interface is used for communication between the above electronic device and other devices.

[0144] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0145] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0146] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned lens shading correction method steps is implemented.

[0147] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the steps of the lens shading correction method in the above embodiments.

[0148] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A lens shading correction method, characterized in that: include: Acquire an image to be corrected; wherein the image to be corrected represents an image generated by an imaging module to be corrected; the imaging module includes a camera and a lens; Based on a preset coordinate transformation model, transform the pixel coordinates of each pixel point in the image to be corrected to obtain a first coordinate; Based on the correction coefficient model, the function value of the first coordinate is calculated as the correction coefficient of the corresponding pixel point in the image to be corrected; wherein the correction coefficient model is generated by a calibrated imaging module; the calibrated imaging module is the same as the camera in the imaging module to be corrected; Based on the correction coefficient, correct the pixel values ​​in the image to be corrected to obtain a corrected image; The coordinate transformation model includes linear function models of the first coordinate and the pixel coordinate in the horizontal direction and the vertical direction respectively; wherein the parameters in the linear function model are determined based on the offset of the optical center point and the zoom factor; the optical center point represents the pixel point corresponding to the optical axis of the lens in the imaging module; the zoom factor is the ratio of the image distance of the calibrated imaging module to the image distance of the imaging module to be calibrated.

2. The method according to claim 1, characterized in that The coordinate transformation model includes: x'=k u (x+x u ); y'=k u (y+y u ); Where x represents the coordinate of the pixel point in the horizontal direction; y represents the coordinate of the pixel point in the vertical direction; x' represents the converted coordinate in the horizontal direction; y' represents the converted coordinate in the vertical direction; x u Indicates the displacement of the optical center point between the calibrated imaging module and the imaging module to be calibrated in the horizontal direction; y u Indicates the displacement of the optical center point between the corrected imaging module and the imaging module to be corrected in the vertical direction; k u Indicates the zoom ratio.

3. The method according to claim 2, characterized in that The coordinate transformation model corresponds to the color channel one by one, and the scaling factors in the coordinate transformation models corresponding to different color channels are different to accommodate and correct incident light of different wavelengths.

4. The method according to claim 1, characterized in that: The correction coefficient model includes: f(x,y)=f h (x-x c )×f v (y-y c ); Where f(x,y) represents the correction coefficient corresponding to the pixel point (x,y); c Indicates the horizontal coordinate of the optical center; y c Indicates the coordinate of the optical center in the vertical direction; f h (xx c ) is expressed as a polynomial fitting model based on the horizontal coordinates, (xx c ) is the input independent variable; f v (yy c ) is represented as a polynomial fitting model based on the vertical coordinates, (yy c ) is the input independent variable.

5. The method according to claim 4, characterized in that Only function terms with even degrees are retained in the polynomial fitting model.

6. The method according to claim 1, characterized in that The calibration process of the correction coefficient model includes: Obtain a standard image generated under a uniform light source; Divide the standard image into multiple regions, and calculate the ratio of the maximum pixel mean in all regions to the pixel mean in each region as the pixel ratio; A quadratic function is fitted to the pixel ratio and horizontal coordinate in each area, and the horizontal coordinate corresponding to the extreme value in the fitted function is used as the horizontal coordinate x of the optical center. c ; A quadratic function is fitted to the pixel ratio and vertical coordinate in each area, and the coordinate corresponding to the extreme value in the fitted function is used as the vertical coordinate y of the optical center. c ; Based on the pixel ratio and coordinates in each area, the remaining parameters in the correction coefficient model are fitted; wherein, in the process of fitting the remaining parameters in the correction coefficient model, the loss function is a nonlinear optimization algorithm.

7. The method according to claim 6, characterized in that When the camera is a linear array camera with a single row of pixels, each pixel point is regarded as a single area during the calibration process of the correction coefficient model.

8. A lens shading correction system, characterized in that: include: An image acquisition module, used to acquire an image to be calibrated; wherein the image to be calibrated represents an image generated by an imaging module to be calibrated; the imaging module includes a camera and a lens; A coordinate conversion module, used for converting the pixel coordinates of each pixel point in the image to be corrected based on a preset coordinate conversion model to obtain a first coordinate; The coordinate transformation model includes a linear function model of the first coordinate and the pixel coordinate in the horizontal direction and the vertical direction respectively; wherein the parameters in the linear function model are determined based on the offset of the optical center point and the zoom factor; the optical center point represents the pixel point corresponding to the optical axis of the lens in the imaging module; the zoom factor is the ratio of the image distance of the calibrated imaging module to the image distance of the imaging module to be calibrated; A coefficient calculation module, used to calculate the function value of the first coordinate based on the correction coefficient model as the correction coefficient of the corresponding pixel point in the image to be corrected; wherein the correction coefficient model is generated by the calibrated imaging module; the calibrated imaging module is the same as the camera in the imaging module to be corrected; The pixel correction module is used to correct the pixel values ​​in the image to be corrected based on the correction coefficient to obtain a corrected image.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the lens shading correction method described in any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium, characterized in that: A computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the lens shading correction method according to any one of claims 1 to 7 is implemented.

Images