Shadow correction method and system, electronic equipment and storage medium
By calculating the correction coefficient based on the polynomial model of the horizontal and vertical coordinate differences of pixels, the problem of difficult and poor results of lens shadow calibration in the prior art is solved, and a more accurate and efficient shadow correction effect is achieved.
Patent Information
- Application Number
- CN202411478820.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-22
- Publication Date
- 2025-01-03
AI Technical Summary
The existing lens shadow correction method is difficult to implement on the hardware side, and the output results of the high-order fit model based on radius are inaccurate. The model data storage space based on the lookup table is large, which cannot effectively improve the shadow correction effect.
The correction coefficient is calculated using a polynomial model based on the difference between the horizontal and vertical directions of the pixel coordinates, and the image correction coefficient model is used to perform the pre-calibrated correction coefficient model, and the memory space is reduced by using the correction coefficient model composed of the polynomial model, which is suitable for the hardware side.
It improves the accuracy and effect of lens shadow correction, reduces the calculation amount and parameter fitting time during the polynomial fitting process, and is suitable for hardware-side implementation.
Smart Images
Figure CN119251112A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing, and in particular relates to a shadow correction method, system, electronic device and storage medium. Background Art
[0002] Lens Shading Correction (LSC) is an important part of image signal processing (ISP), which is usually used to correct the uneven brightness of the image caused by the optical characteristics of the lens. Lens shading is mainly divided into luma shading and color shading. The former is brighter in the center of the image and darker around it, while the latter is inconsistent in color between the center and the surroundings.
[0003] In existing lens shading correction schemes, lens shading correction is usually performed using a lookup table with a one-to-one correspondence between pixel coordinates and correction coefficients or a high-order fitting model based on a radius R between a pixel and an optical center point.
[0004] However, due to the different characteristics of the pixel response to light angle in the horizontal and vertical directions in the image sensor, the output result of the high-order fitting model based on radius R is not accurate, resulting in poor quality of shading correction. The correction coefficient model based on the lookup table provides a one-to-one correspondence between the correction coefficients and the pixels. Although it can ensure the accuracy of the correction coefficients, it requires a large data storage space and cannot implement lens shading correction on the hardware side.
[0005] Therefore, a shadow correction method is urgently needed to solve the above problem. Summary of the invention
[0006] The present application proposes a shadow correction method, system, electronic device and storage medium, which are used to further improve the effect of shadow correction on the basis of implementing lens shadow correction on the hardware side.
[0007] To achieve the above objectives, this application proposes the following technical solutions: In a first aspect of the present application, a shadow correction method is provided, comprising: Acquire an image to be corrected; Based on a pre-calibrated correction coefficient model, a correction coefficient of each pixel in the image to be corrected is calculated; the correction coefficient model is obtained by multiplying a first polynomial model based on the horizontal coordinates of the pixel and a second polynomial model based on the vertical coordinates of the pixel; wherein the independent variable of the first polynomial model is the coordinate difference between the pixel and the optical center point in the horizontal direction; the independent variable of the second polynomial model is the coordinate difference between the pixel and the optical center point in the vertical direction; the optical center point represents the pixel corresponding to the optical axis of the lens; Based on the correction coefficient, the pixel value of the pixel is corrected to obtain a corrected image.
[0008] Optionally, in the first polynomial model and the second polynomial model, only function terms with even degrees are set.
[0009] Optionally, the correction coefficient model is: f(x,y)=f h (xx c )×f v (yy c ); Where f(x,y) represents the correction coefficient corresponding to the pixel (x,y); x represents the horizontal coordinate of the pixel; y represents the vertical coordinate of the pixel; x c Indicates the horizontal coordinate of the optical center; y c Indicates the coordinate of the optical center in the vertical direction; (xx c ) represents the coordinate difference between the pixel (x, y) and the optical center in the horizontal direction; (yy c ) represents the coordinate difference between the pixel (x, y) and the optical center in the vertical direction; f h (xx c ) is represented as the first polynomial model; f v (yy c ) is expressed as the second polynomial model; (xx c ) is the input independent variable; (yy c ) is the input independent variable.
[0010] Optionally, the calibration process of the correction coefficient model includes: Get a standard image under a uniform light source; The standard image is divided into multiple regions, and the ratio of the maximum pixel mean in all regions to the pixel mean in each region is calculated as the pixel ratio; A quadratic function is fitted to the pixel ratio and horizontal coordinate in each area, and the horizontal coordinate corresponding to the extreme value in the fitted function is used as the horizontal coordinate x of the optical center. c ; A quadratic function is fitted to the pixel ratio and vertical coordinate in each area, and the coordinate corresponding to the extreme value in the fitted function is used as the vertical coordinate y of the optical center. c ; Based on the pixel ratios and coordinates in each region, the remaining parameters in the correction coefficient model are fitted.
[0011] Optionally, in the process of fitting the remaining parameters in the correction coefficient model, the loss function is a nonlinear optimization algorithm.
[0012] Optionally, if the optical axis and / or image distance of the lens in the imaging module of the calibrated correction coefficient model changes, the shadow correction method further includes: Acquire an image to be processed; wherein the image to be processed represents an image generated by the changed imaging module; Based on a preset coordinate transformation model, the coordinates of each pixel in the image to be processed are transformed to obtain a first coordinate; Based on the calibrated correction coefficient model, the function value of the first coordinate is calculated as the correction coefficient of the corresponding pixel in the image to be processed; Based on the correction coefficient, correct the pixel values in the image to be processed to obtain a corrected image; Among them, the coordinate transformation model is represented as a combination of linear function models of the first coordinate and the pixel coordinate in the horizontal direction and the vertical direction respectively; the parameters in the linear function model are determined based on the offset of the optical center point and the scaling factor; the scaling factor represents the ratio of the image distance between the corrected imaging module and the imaging module to be corrected.
[0013] Optionally, the coordinate transformation model includes: x'=k u (x+x u ); y'=k u (y+y u ); Where x represents the coordinate of the pixel point in the horizontal direction; y represents the coordinate of the pixel point in the vertical direction; x' represents the converted coordinate in the horizontal direction; y' represents the converted coordinate in the vertical direction; x u Indicates the displacement of the optical center point between the calibrated imaging module and the imaging module to be calibrated in the horizontal direction; y u Indicates the displacement of the optical center point between the corrected imaging module and the imaging module to be corrected in the vertical direction; k u Indicates the zoom ratio.
[0014] In a second aspect of the present application, a shadow correction system is provided, comprising: An image acquisition module, used for acquiring an image to be corrected; A coefficient calculation module is used to calculate the correction coefficient of each pixel in the image to be corrected based on a pre-calibrated correction coefficient model; the correction coefficient model is obtained by multiplying a first polynomial model based on the coordinates of the pixel in the horizontal direction and a second polynomial model based on the coordinates of the pixel in the vertical direction; wherein the input of the first polynomial model is the coordinate difference between the pixel and the optical center point in the horizontal direction; the input of the second polynomial model is the coordinate difference between the pixel and the optical center point in the vertical direction; the optical center point represents the pixel point corresponding to the optical axis of the lens; The image correction module is used to correct the pixel value of the pixel based on the correction coefficient to obtain a corrected image.
[0015] In a third aspect of the present application, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the shadow correction method described in any one of the first aspects when executing the program stored in the memory.
[0016] In a fourth aspect of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the shadow correction method described in any one of the first aspects is implemented.
[0017] The beneficial effects of this application are as follows: The present application provides a shadow correction method, including: acquiring an image to be corrected; calculating the correction coefficient of each pixel in the image to be corrected based on a pre-calibrated correction coefficient model; the correction coefficient model is obtained by multiplying a first polynomial model based on the coordinates of the pixel in the horizontal direction and a second polynomial model based on the coordinates of the pixel in the vertical direction; wherein the input of the first polynomial model is the coordinate difference between the pixel and the optical center point in the horizontal direction; the input of the second polynomial model is the coordinate difference between the pixel and the optical center point in the vertical direction; the optical center point represents the pixel point corresponding to the optical axis of the lens; based on the correction coefficient, the pixel value of the pixel is corrected to obtain a corrected image.
[0018] Based on the above processing, the present application provides a correction coefficient model that is calibrated for both the horizontal and vertical directions. The correction coefficient obtained by using the correction coefficient model is more in line with the characteristics of the image sensor's response to the light angle, effectively improving the effect of lens shading correction. At the same time, the correction coefficient model constructed using the polynomial model requires a small storage space, which can make the shading correction solution provided by the present application applicable to the hardware end. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of a shadow correction method provided by the present application; Figure 2 It is a schematic diagram of the principle of coordinate transformation provided by this application; Figure 3 It is a schematic diagram of the principle of an imaging formula provided by this application; Figure 4 is a structural diagram of a shadow correction system provided by the present application; Figure 5 It is a structural diagram of an electronic device provided by this application. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.
[0021] In order to solve the problem that the existing shadow correction method cannot be implemented on the hardware side or the shadow correction effect is poor, the present application provides a shadow correction method, such as Figure 1 As shown, the method comprises the following steps: S1. Obtain an image to be corrected.
[0022] S2. Calculate the correction coefficient of each pixel in the image to be corrected based on the pre-calibrated correction coefficient model.
[0023] Among them, the correction coefficient model is obtained by multiplying a first polynomial model based on the coordinates of the pixel in the horizontal direction by a second polynomial model based on the coordinates of the pixel in the vertical direction; wherein the input of the first polynomial model is the coordinate difference between the pixel and the optical center point in the horizontal direction; the input of the second polynomial model is the coordinate difference between the pixel and the optical center point in the vertical direction; the optical center point represents the pixel point corresponding to the optical axis of the lens.
[0024] S3. Correct the pixel value of the pixel based on the correction coefficient to obtain a corrected image.
[0025] Based on the above processing, the present application provides a correction coefficient model that is calibrated for both the horizontal and vertical directions. The correction coefficient obtained by using the correction coefficient model is more in line with the characteristics of the image sensor's response to the light angle, effectively improving the effect of lens shading correction. At the same time, the correction coefficient model constructed using the polynomial model requires low storage space and can be applied to the hardware end.
[0026] In some embodiments, the correction coefficient model is: f(x,y)=f h (xx c )×f v (yy c ); Where f(x,y) represents the correction coefficient corresponding to the pixel (x,y); x represents the horizontal coordinate of the pixel; y represents the vertical coordinate of the pixel; x c Indicates the horizontal coordinate of the optical center; y c Indicates the coordinate of the optical center in the vertical direction; (xx c) represents the coordinate difference between the pixel (x, y) and the optical center in the horizontal direction; (yy c ) represents the coordinate difference between the pixel (x, y) and the optical center in the vertical direction; f h (xx c ) is represented as the first polynomial model; f v (yy c ) is expressed as the second polynomial model; (xx c ) is the input independent variable; (yy c ) is the input independent variable.
[0027] In one implementation, only function terms with even degrees are set in the first polynomial model and the second polynomial model.
[0028] Based on the above processing, the number of parameters to be fitted in the polynomial fitting model can be reduced, which can not only reduce the amount of calculation in the model fitting process and shorten the time of the parameter fitting process, but also reduce the risk of falling into the local optimal value in the parameter fitting process, thereby ensuring the accuracy of the final fitting parameters. In addition, for the horizontal or vertical direction, the response of the image sensor pixel to the light angle is symmetrical relative to the sensor normal direction (i.e., the lens optical axis). It can be seen that in the two polynomial models, retaining only the even-numbered terms does not affect the accuracy of the output results.
[0029] The correction coefficient model is explained below using a quadratic polynomial as an example.
[0030] First, the correction coefficient model of the 4th-order polynomial is:
[0031] Among them, k h1 , k h2 and k h3 represents the parameters to be fitted in the horizontal direction, k v1 , k v2 and k v3 Indicates the parameters to be fitted in the vertical direction.
[0032] In some embodiments, the calibration process of the correction coefficient model includes the following steps: Step a: Acquire a standard image under a uniform light source.
[0033] Step b: divide the standard image into multiple regions, and calculate the ratio of the maximum pixel mean in all regions to the pixel mean in each region as the pixel ratio.
[0034] Step c: Fit a quadratic function to the pixel ratio and horizontal coordinate in each area, and use the horizontal coordinate corresponding to the extreme value in the fitted function as the horizontal coordinate x of the optical center.c .
[0035] Step d: Fit a quadratic function to the pixel ratio and vertical coordinate in each area, and use the coordinates corresponding to the extreme values in the fitted function as the vertical coordinates y of the optical center. c .
[0036] Step e: Based on the pixel ratios and coordinates in each region, fitting is performed to obtain the remaining parameters in the correction coefficient model.
[0037] For step a, after the camera is matched with the lens to form an imaging module, a uniform light source is photographed to obtain a reference image. Preferably, by selecting a suitable lens, the grayscale value in the center of the reference image is 1.3 to 3 times the grayscale value of the four corners, and the light source brightness and exposure time are adjusted so that the grayscale values of the darkest and brightest areas in the reference image are within the grayscale value range of 20% to 80%.
[0038] Based on the above range setting, it is possible to avoid the grayscale value of the reference image being too uniform, prevent the inability to fit an accurate angle response curve (ie, a polynomial fitting model), and avoid the four corners being too dark, so as to ensure the accuracy of the calculation results after the area division in step b.
[0039] For step b, the reference image is divided into N×N regions, where N can be 16, 32, 64, or 128, for ease of hardware implementation. When calculating the pixel mean in each region, the pixels in each region are added up and then directly shifted to obtain the pixel mean without actually performing division.
[0040] The coordinates of the physical center of each region are marked as (x i , y i ), the pixel ratio m is calculated according to the following formula i .
[0041]
[0042] Among them, x i Indicates the horizontal coordinate of the area, y i Represents the vertical coordinate of the region, i\in \left [ {1,{N}^{2}} \right ] .
[0043] For step c and step d, since the optical center point in the lens shadow is the brightest, the extreme point can be found by quadratic function fitting to determine the optical center point x c and c .
[0044] Specifically, take all the areas where the horizontal center line passes through in the reference image and transform them intoi With m i Perform quadratic function fitting and find the x corresponding to the extreme value of the quadratic function i value, which is x c .
[0045] And, take all the areas where the vertical center line of the reference image passes through, and divide them into i With m i Perform quadratic function fitting and find the y corresponding to the extreme value of the quadratic function i Value, that is, y c .
[0046] Based on the above processing, the coordinates of the optical center are directly solved using the extreme values in the quadratic function, which reduces the number of parameters to be fitted in the correction coefficient. This not only shortens the time for parameter fitting, but also effectively reduces the risk of falling into the local optimal solution in the subsequent fitting process, thereby improving the accuracy of the fitting parameters.
[0047] For step e, based on the pixel ratio and coordinates in each region, the coefficients in the polynomial fitting model are fitted. In the fitting process, a nonlinear optimization algorithm is used to obtain the function value of the loss function, that is, the loss function is a nonlinear optimization algorithm.
[0048] Specifically, all the x in the region i , y i , m i As input data, the 6 parameters k in the f(x,y) function are fitted by nonlinear optimization algorithms (such as Newton method, LM algorithm, etc.) h1 , k h2 , k h3 , k v1 , k v2 and k v3 .
[0049] When the solution provided in the present application performs color shading correction, it is necessary to perform parameter fitting on different color channels in the color image based on the correction coefficient model to obtain different fitting parameters to adapt to different color channels.
[0050] When the camera in the present application is a linear array camera with a single row of pixels, in the parameter fitting process of the correction coefficient model, each pixel point is used as a single area to complete the parameter fitting. Based on the above processing, the number of areas can be increased to improve the accuracy of parameter fitting.
[0051] In addition, the correction coefficient model corresponding to a line array camera with a single row of pixels only needs to include a model in a single direction (horizontal or vertical).
[0052] For step S3, the product of the correction coefficient and the pixel value in the image to be corrected is calculated as the corrected pixel value, thereby obtaining the corrected image. Specifically, the corrected image data is calculated by the following formula: I'(x,y)=I(x,y)×f(x',y') Wherein, I'(x,y) represents the pixel value of the pixel in the corrected image, I(x,y) represents the pixel value of the pixel in the image to be corrected, and f(x',y') represents the correction coefficient corresponding to the pixel (x',y').
[0053] The calibration process of the correction coefficient model is to bind the camera and the lens together, that is, to calibrate the same imaging module, so that a set of calibrated correction coefficient models can only be used for an imaging module formed by a specific camera and lens combination. Among them, the imaging module refers to an imaging system including a camera and a lens.
[0054] For industrial machine vision application scenarios, the camera and lens in the imaging module are usually delivered independently, and different lenses are required for different scenarios. Specifically, during the factory calibration process, the lens used by the camera manufacturer is different from the lens actually used by the user, so the manufacturer cannot solve the lens shadow problem encountered by the user in advance. At the same time, the user's work scene usually does not have the calibration conditions of uniform light source, which makes it impossible to recalibrate the corresponding correction coefficient model for the imaging module after the lens is replaced.
[0055] After the lens of the imaging module is replaced, the internal lens optical axis and / or image distance thereof are changed. In order to realize the shadow correction thereof without uniform light source, the shadow correction method proposed in the present application further includes the following steps: S4. Obtain the image to be processed.
[0056] The image to be processed represents the image generated by the changed imaging module.
[0057] S5. Based on a preset coordinate transformation model, transform the pixel coordinates of each pixel in the image to be processed to obtain a first coordinate.
[0058] S6. Based on the calibrated correction coefficient model, calculate the function value of the first coordinate as the correction coefficient of the corresponding pixel in the image to be processed.
[0059] S7. Correct the pixel values in the image to be processed based on the correction coefficient to obtain a corrected image.
[0060] The coordinate transformation model is represented as a combination of linear function models of the first coordinate and the pixel coordinate in the horizontal direction and the vertical direction respectively; the parameters in the linear function model are determined based on the offset and the zoom factor of the optical center point; the zoom factor represents the ratio of the image distance between the calibrated imaging module and the imaging module to be calibrated; the calibrated imaging module represents the imaging module with the calibration coefficient model before the change; the imaging module to be calibrated represents the imaging module after the change. The imaging module represents a system composed of a camera and a lens.
[0061] Based on the above processing, the present application uses a coordinate transformation model to transform the coordinates of pixels in the image to be processed, which is used to correct the adverse effects caused by the optical center offset and image distance under different lenses, and then obtains the correction coefficient based on the converted coordinates through the correction coefficient model of the calibrated imaging module to achieve shadow correction in the image to be processed.
[0062] Therefore, for the changed imaging module, the shadow correction scheme provided in the present application does not need to be recalibrated under a uniform light source. Lens shadow correction can be achieved through coordinate transformation and the use of the correction coefficient model of the calibrated imaging module, thereby expanding the scope of application of the shadow correction scheme.
[0063] For step S5, the working principle of the coordinate conversion process is described as follows: If the calibrated imaging module includes camera 1 and lens 1, the imaging module to be calibrated includes camera 1 and lens 2. Simplify the lens into a pinhole imaging model, such as Figure 2 As shown, assuming that the optical center of lens 1 is located at A and the optical center of lens 2 is located at A', under the same light source, for the pixel B and pixel B' formed by the incident light passing through the optical center and with the same incident angle, the light angles are consistent and the shadow correction coefficients are also the same, as shown in Figure 2 As shown, line segment AB is parallel to line segment A'B'. Therefore, the shading correction coefficients of lens 1 and lens 2 are scaled proportionally relative to the pixel coordinates. Among them, the optical center represents the position of the optical axis in the lens.
[0064] At the same time, the position of the optical center under different lenses will also shift, so the offset of the optical center point also needs to be corrected in the coordinate transformation model.
[0065] In some embodiments, the coordinate transformation model is represented as a combination of linear function models of the first coordinate and the pixel coordinate in the horizontal direction and the vertical direction, respectively.
[0066] Specifically, the combination of linear function models can be: x'=ax+b1; y'=ay+b2; Among them, x represents the coordinate of the pixel point in the horizontal direction; y represents the coordinate of the pixel point in the vertical direction; x' represents the converted coordinate in the horizontal direction; y' represents the converted coordinate in the vertical direction; a represents the magnification parameter; b1 represents the constant parameter of the linear function model in the horizontal direction; b2 represents the constant parameter of the linear function model in the vertical direction.
[0067] In one implementation, the parameters in the linear function model (including parameters a, b1 and b2) are determined based on the offset of the optical center point and the zoom factor, wherein the optical center point represents the pixel point corresponding to the optical axis of the lens in the imaging module.
[0068] In one implementation, the ratio of the image distance between the calibrated imaging module and the imaging module to be calibrated is calculated as the zoom factor.
[0069] In some embodiments, the coordinate transformation model may be: x'=k u (x+x u ); y'=k u (y+y u ); Where x represents the coordinate of the pixel point in the horizontal direction; y represents the coordinate of the pixel point in the vertical direction; x' represents the converted coordinate in the horizontal direction; y' represents the converted coordinate in the vertical direction; x u Indicates the displacement of the optical center point between the calibrated imaging module and the imaging module to be calibrated in the horizontal direction; y u Indicates the displacement of the optical center point between the corrected imaging module and the imaging module to be corrected in the vertical direction; k u Indicates the zoom ratio.
[0070] The pinhole imaging model is as follows Figure 3 As shown, combined with the aforementioned working principle of the coordinate transformation model, it can be known that under different lenses, for pixel points with the same shading correction coefficient, their pixel coordinates are scaled proportionally, and the scaling factor (i.e., the zoom factor) is the ratio between the image distances under different lenses.
[0071] It can be understood that the parameter a of different coordinate transformation models is equivalent to the parameter k u , parameters b1 and b2 are equivalent to k u × u and k u ×y u .
[0072] The shadow correction scheme provided in the present application can also be applied to the chromaticity correction process in color scenes. In one implementation, the coordinate transformation model corresponds to the color channel one by one, and the coordinate transformation models corresponding to different color channels have different scaling factors to be compatible with the correction of incident light of different wavelengths. When the camera is a color camera, coordinate transformation models with different scaling factors are set for different color channels. Among them, Bayer CFA is currently more commonly used in color cameras, including four color channels of G, R, B, and G. For this situation, four coordinate transformation models with different scaling factors need to be constructed.
[0073] For chromaticity correction, due to the different refractive indices of light of different wavelengths, some poor quality lenses may have dispersion problems, so different k values can be set for different color channels. u parameters to further optimize lens compatibility.
[0074] In some embodiments, the shadow correction scheme provided by the present application can also be applied to a linear array camera or a TDI (Time Delay Integration) camera with a single row of pixels. In this case, it is only necessary to process the lens shadow correction in a single direction. Specifically, the linear array camera only needs to consider the shadow correction in the direction parallel to the pixel row, or the TDI camera does not need to process the shadow correction in the TDI superposition direction.
[0075] For the parameter k in the coordinate transformation model u , x cu and cu For solving the parameters such as , this application also provides different solutions.
[0076] For zoom factor k u , the image distance v can be solved using the imaging formula: 1 / u+1 / v=1 / f. Among them, u represents the object distance, which can be obtained by measuring the distance between the lens and the object being measured, and f represents the focal length, which can be obtained by directly reading the imaging parameters in the imaging module.
[0077] Then, the ratio between the image distance v1 of the calibrated imaging module and the image distance v2 of the imaging module to be calibrated is calculated as the scaling factor k. u That is, k u =v1 / v2.
[0078] Regarding the offset of the optical center point, in one implementation, it can be calculated from the optical center point of the imaging module to be corrected. For example, under the same shooting conditions, the imaging module to be corrected and the corrected imaging module are allowed to shoot the same area of the object to be measured, and generate two images with the same resolution. Compare the pixel values in the two images, select the area with the same pixel value as the image generated by the corrected imaging module in the image generated by the imaging module to be corrected, and select the central pixel point of the area as the optical center point of the imaging module to be corrected. Then, calculate the pixel offset from the optical center point in the corrected imaging module to obtain x u and u .
[0079] In actual work, the method of multiple cyclic adjustments is usually adopted to fit the parameters of the above coordinate transformation model. The fitting method includes the following steps: Step 1: Use the imaging module to be calibrated to acquire a reference image.
[0080] Step 2: Determine whether the reference image is bilaterally symmetrical. If so, do not adjust x. u If not, adjust the parameter x based on the preset adjustment step size. u The initial value of is set, and the process returns to step 1 until the reference image is bilaterally symmetrical.
[0081] And determine whether the reference image is symmetrical up and down. If so, do not adjust y u If not, adjust the parameter y based on the preset adjustment step size. u The initial value of is set, and the process returns to step 1 until the reference image is symmetrical from top to bottom.
[0082] Step 3: Determine whether the quality score of the reference image is greater than a preset threshold. If so, the parameter adjustment is completed and the adjusted parameter value is used as the fitting parameter of the coordinate transformation model. If not, adjust the parameter k based on the preset adjustment step size. u And return to step 1 until the quality score of the reference image is greater than the preset threshold.
[0083] Among them, k u The initial value of x is 1, u ,y u The initial value of is 0.
[0084] Regarding step 2, there are two ways to determine whether the image is symmetrical.
[0085] Judgment method 1: In step 1, take a photo of the same object under test (e.g., calibration plate) in the four corners of the image to generate a reference image. Then, measure whether the pixel mean values of the four corners of the reference image are the same to determine whether it is vertically symmetrical or horizontally symmetrical.
[0086] Specifically, if the grayscale values of the four corners are lower on the left and higher on the right in the horizontal direction, then based on the preset step size, reduce x u Similarly, if the grayscale values of the four corners are low at the top and high at the bottom in the vertical direction, based on the preset step size, reduce y u The initial value of . Among them, in the technical solution provided by the present application, the pixel coordinate system takes the vertex of the image as the coordinate origin, the number of pixel columns is the coordinate in the X-axis direction (i.e., the horizontal direction), and the number of pixel rows is the coordinate in the Y-axis direction (i.e., the vertical direction). In addition, the adjustment step sizes of different parameters may be different.
[0087] Judgment method 2: Rotate the imaging module to be calibrated 180 degrees and then shoot to generate a new reference image. Compare the local area means of the four corners with the reference image before rotation to determine whether it is symmetrical and adjust the initial value of the parameter.
[0088] Specifically, if compared with the reference image before rotation, in the horizontal direction, the grayscale values of the two left corners decrease, and the grayscale values of the two right corners increase, then based on the preset step size, x is reduced. u If the grayscale values of the upper two corners decrease and the grayscale values of the lower two corners increase in the vertical direction, then based on the preset step size, reduce y u The initial value of .
[0089] For step 3, the quality score can be obtained by calculating the average of the image scores provided by multiple users. In step 3, the shading correction effect of the lens in the imaging module to be corrected is optimized by adjusting the parameter ku.
[0090] Based on the above processing, it can be seen that the solution provided in this application can realize lens shading correction in the user's usage scenario without the user having a uniform light source. Different lenses can be adapted by adjusting the parameters in the preset coordinate transformation model in the camera.
[0091] In some embodiments, the present application also provides a shadow correction system, such as Figure 4 As shown, the correction system includes: The image acquisition module 401 is used to acquire the image to be corrected.
[0092] The coefficient calculation module 402 is used to calculate the correction coefficient of each pixel in the image to be corrected based on a pre-calibrated correction coefficient model; the correction coefficient model is obtained by multiplying a first polynomial model based on the horizontal coordinates of the pixel and a second polynomial model based on the vertical coordinates of the pixel; wherein the input of the first polynomial model is the coordinate difference between the pixel and the optical center point in the horizontal direction; the input of the second polynomial model is the coordinate difference between the pixel and the optical center point in the vertical direction; the optical center point represents the pixel point corresponding to the optical axis of the lens.
[0093] The image correction module 403 is used to correct the pixel value of the pixel based on the correction coefficient to obtain a corrected image.
[0094] The present application also provides an electronic device, such as Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504. Memory 503, used for storing computer programs; The processor 501 is used to implement any of the above-mentioned shadow correction methods when executing the program stored in the memory 503.
[0095] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0096] The communication interface is used for communication between the above electronic device and other devices.
[0097] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0098] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0099] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned shadow correction method steps is implemented.
[0100] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any step of the shadow correction method in the above embodiments.
[0101] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A shadow correction method, characterized in that: include: Acquire an image to be corrected; Based on a pre-calibrated correction coefficient model, a correction coefficient of each pixel in the image to be corrected is calculated; the correction coefficient model is obtained by multiplying a first polynomial model based on the horizontal coordinates of the pixel and a second polynomial model based on the vertical coordinates of the pixel; wherein the independent variable of the first polynomial model is the coordinate difference between the pixel and the optical center point in the horizontal direction; the independent variable of the second polynomial model is the coordinate difference between the pixel and the optical center point in the vertical direction; the optical center point represents the pixel corresponding to the optical axis of the lens; Based on the correction coefficient, the pixel value of the pixel is corrected to obtain a corrected image.
2. The shadow correction method according to claim 1, characterized in that: In the first polynomial model and the second polynomial model, only function terms with even degrees are set.
3. The shadow correction method according to claim 1, characterized in that: The correction coefficient model is: f(x,y)=f h (x-x c )×f v (y-y c ); Where f(x,y) represents the correction coefficient corresponding to the pixel (x,y); x represents the horizontal coordinate of the pixel; y represents the vertical coordinate of the pixel; x c Indicates the horizontal coordinate of the optical center; y c Indicates the coordinate of the optical center in the vertical direction; (xx c ) represents the coordinate difference between the pixel (x, y) and the optical center in the horizontal direction; (yy c ) represents the coordinate difference between the pixel (x, y) and the optical center in the vertical direction; f h (xx c ) is represented as the first polynomial model; f v (yy c ) is expressed as the second polynomial model; (xx c ) is the input independent variable; (yy c ) is the input independent variable.
4. The shadow correction method according to claim 1, characterized in that: The calibration process of the correction coefficient model includes: Get a standard image under a uniform light source; The standard image is divided into multiple regions, and the ratio of the maximum pixel mean in all regions to the pixel mean in each region is calculated as the pixel ratio; A quadratic function is fitted to the pixel ratio and horizontal coordinate in each area, and the horizontal coordinate corresponding to the extreme value in the fitted function is used as the horizontal coordinate x of the optical center. c ; A quadratic function is fitted to the pixel ratio and vertical coordinate in each area, and the coordinate corresponding to the extreme value in the fitted function is used as the vertical coordinate y of the optical center. c ; Based on the pixel ratios and coordinates in each region, the remaining parameters in the correction coefficient model are fitted.
5. The shadow correction method according to claim 1, characterized in that: In the process of fitting the remaining parameters in the correction coefficient model, the loss function is a nonlinear optimization algorithm.
6. The shadow correction method according to claim 1, characterized in that: If the optical axis and / or image distance of the lens in the imaging module of the calibrated correction coefficient model changes, the shadow correction method further includes: Acquire an image to be processed; wherein the image to be processed represents an image generated by the changed imaging module; Based on a preset coordinate transformation model, the coordinates of each pixel in the image to be processed are transformed to obtain a first coordinate; Based on the calibrated correction coefficient model, the function value of the first coordinate is calculated as the correction coefficient of the corresponding pixel in the image to be processed; Based on the correction coefficient, correct the pixel values in the image to be processed to obtain a corrected image; Among them, the coordinate transformation model is represented as a combination of linear function models of the first coordinate and the pixel coordinate in the horizontal direction and the vertical direction respectively; the parameters in the linear function model are determined based on the offset of the optical center point and the scaling factor; the scaling factor represents the ratio of the image distance of the corrected imaging module to the image distance of the imaging module to be corrected.
7. The shadow correction method according to claim 6, characterized in that: The coordinate transformation model includes: x'=k u (x+x u ); y'=k u (y+y u ); Where x represents the coordinate of the pixel point in the horizontal direction; y represents the coordinate of the pixel point in the vertical direction; x' represents the converted coordinate in the horizontal direction; y' represents the converted coordinate in the vertical direction; x u Indicates the displacement of the optical center point between the calibrated imaging module and the imaging module to be calibrated in the horizontal direction; y u Indicates the displacement of the optical center point between the corrected imaging module and the imaging module to be corrected in the vertical direction; k u Indicates the zoom ratio.
8. A shadow correction system, characterized in that: include: An image acquisition module, used for acquiring an image to be corrected; A coefficient calculation module is used to calculate the correction coefficient of each pixel in the image to be corrected based on a pre-calibrated correction coefficient model; the correction coefficient model is obtained by multiplying a first polynomial model based on the coordinates of the pixel in the horizontal direction and a second polynomial model based on the coordinates of the pixel in the vertical direction; wherein the input of the first polynomial model is the coordinate difference between the pixel and the optical center point in the horizontal direction; the input of the second polynomial model is the coordinate difference between the pixel and the optical center point in the vertical direction; the optical center point represents the pixel point corresponding to the optical axis of the lens; The image correction module is used to correct the pixel value of the pixel based on the correction coefficient to obtain a corrected image.
9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the shadow correction method described in any one of claims 1 to 7 when executing the program stored in the memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the shadow correction method according to any one of claims 1 to 7 is implemented.