Image distortion correction method
By calculating the phase distribution and second-order derivative of the raster image, image distortion correction is directly performed, which solves the problem of large errors in the distortion model in the prior art, and realizes high-precision image distortion correction to meet the needs of high-precision images.
Patent Information
- Application Number
- CN202410642896.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-23
- Publication Date
- 2024-12-27
AI Technical Summary
The prior art has large errors in image distortion correction due to the simplification of the distortion model, which cannot meet the requirements of high-precision images, especially when image edge correction is large, and traditional methods are difficult to meet the accuracy requirements of three-dimensional reconstruction and photogrammetry.
By taking raster images, calculate their horizontal and vertical phase distributions, use the second-order derivative of the phase distribution to determine the main point position, and use this to correct image distortion to avoid errors caused by distortion model calculations, and achieve high-precision image distortion correction .
High-precision image distortion correction is achieved when the distortion coefficient is unknown, avoiding errors caused by distortion model calculation, and high-precision correction of a single image can be performed to meet the needs of high-precision images.
Smart Images
Figure CN119205587A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an image distortion correction method. Background Art
[0002] Image distortion is caused by manufacturing or installation errors of the lens group, which leads to a deviation between the theoretical imaging position and the actual imaging position of the photographed object on the image plane. Correcting image distortion is an important step in three-dimensional reconstruction and photogrammetry using sequential images.
[0003] At present, the method adopted for camera distortion correction is to first checkerboard the image, solve the distortion constraint model by using the geometric constraint relationship between the grid points and the imaging points in the checkerboard, and finally correct the image distortion based on the distortion model. However, the distortion model is a simplified theoretical model constructed according to the cause of distortion formation, and there is a large error with the actual distortion pattern. Therefore, the method of using the distortion model for distortion correction cannot completely correct the image distortion, especially when the distortion correction error at the edge of the image is large. As the requirements for measurement accuracy in application scenarios such as three-dimensional reconstruction and photogrammetry become higher and higher, traditional distortion correction methods can no longer meet the needs of various scenarios for high-precision images, and image distortion correction methods with higher correction accuracy must be developed. Summary of the invention
[0004] The purpose of the present invention is to overcome the deficiencies of the above prior art and provide a high-precision image distortion correction method.
[0005] The purpose of the present invention is achieved through the following technical solutions:
[0006] An image distortion correction method comprises: S1, photographing a grating image, wherein the grating image contains distortion; S2, calculating the phase distribution of the grating image to obtain a horizontal phase distribution Q x and vertical phase distribution Q y ; S3, based on the horizontal phase distribution Q x and vertical phase distribution Q y Calculate the distortion of the grating image; S4, correct the grating image based on the distortion.
[0007] Preferably, step S2 comprises:
[0008] S21, performing horizontal filtering and vertical filtering on the grating image respectively to obtain a first one-dimensional grating image and a second one-dimensional grating image which are perpendicular to each other, and determining a sampling interval P according to the first one-dimensional grating image and the second one-dimensional grating image k ;
[0009] S22, calculating the horizontal phase distribution Q of the first one-dimensional grating image x, calculate the vertical phase distribution Q of the second one-dimensional grating image y ;
[0010] S23, respectively calculating the horizontal phase distribution Q x and vertical phase distribution Q y The second-order derivative of; the calculation formula is:
[0011]
[0012] f is the horizontal phase distribution Q x and vertical phase distribution Q y The i-th and j-th elements of .
[0013] Preferably, the horizontal phase distribution Q of the first one-dimensional grating image is calculated x Steps and calculation of the vertical phase distribution Q of the second one-dimensional grating image y The steps of calculating the horizontal phase distribution Q of the first one-dimensional grating image are the same as x include:
[0014] The first pixel value of the first row of the first one-dimensional raster image is taken out and placed at the first row and first column of the blank reference image matrix, and then the first pixel value of the first row of the first one-dimensional raster image is sequentially k The pixel value at +1 is taken and placed in the first row mP of the reference image matrix k +1, where m = 1, 2, 3, ...;
[0015] Using the same method as above, the k+1th pixel value of the first row of the first one-dimensional raster image is taken out and placed in the k+1th row and k+1th column of the reference image matrix, and then the pixel values mP in the first row of the first one-dimensional raster image are sequentially replaced. k The pixel value at position +k+2 is taken and placed in the k+1th row mP of the reference image matrix. k +k+2, where k=0,1,2,……P k -1;
[0016] Interpolate the pixel values of the blank positions in the reference image matrix and obtain P k Phase shifted moiré fringes, the phase difference between each moiré fringes is 2π / P k ;
[0017] Perform discrete Fourier transform on each row of the first one-dimensional grating image to obtain the phase distribution Q of the first one-dimensional grating image. x .
[0018] Preferably, the calculation formula for the grayscale value of the phase-shifted moiré fringe is:
[0019]
[0020] Among them, P k is the sampling interval; k represents the moiré fringe number;
[0021] The phase distribution of the phase-shifted moiré fringes is:
[0022]
[0023] Preferably, step S3 comprises: determining the principal point position (u0, v0) of the grating image, the formula being:
[0024] |Q″ x |+|Q″ y |=min|Q″ x |+|Q″ y |
[0025] Among them, Q x is the horizontal phase distribution Q x The second derivative and Q" y is the vertical phase distribution Q y The second derivative of
[0026] Based on the horizontal phase distribution Q x The second-order derivative Q" x Calculate the distortion δ of the first one-dimensional raster image x (i, j), based on the vertical phase distribution Q y The second-order derivative Q" y Calculate the distortion δ of the second one-dimensional raster image y (i, j), the calculation formulas are:
[0027]
[0028]
[0029] Among them, p t is the grating period.
[0030] Preferably, step S4 comprises: according to the distortion δ x (i, j) and distortion δ y (i, j) The grating image coordinates are corrected, and the correction formula is:
[0031] Preferably, step S1 comprises: grating pattern production and grating image shooting; wherein, the grating image shooting comprises: using a distortion camera to shoot the grating pattern, the angle between the photographic axis of the distortion camera and the grating pattern plane is between 85° and 95°, and the grating pattern exceeds the camera frame.
[0032] Preferably, the grating pattern is produced by projection, printing or washing, the grating pattern adopts a rectangular array, and the array spacing of the grating pattern is 5 to 20 pixels.
[0033] Preferably, the filtering method adopts low-pass filtering or mean filtering.
[0034] Preferably, the method for interpolating the pixel values of the blank positions in the reference image matrix adopts one of linear interpolation, quadratic interpolation, cubic interpolation and spline interpolation.
[0035] The present invention has the following advantages over the prior art:
[0036] This application uses the photographed grating pattern to obtain the phase change of the grating image in different directions, and uses the phase change to correct the image distortion, avoiding the problem of large errors caused by calculating the distortion through the distortion model, and this solution can correct the distortion when the distortion coefficient is unknown. In addition, this application uses a photographed grating pattern to calculate the phase of the distorted image, and the phase before distortion is obtained through the phase distribution characteristics of the distorted image, realizing high-precision distortion correction of a single distorted image. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0038] Figure 1 Schematic diagram of the process of the image distortion correction method of Example 1.
[0039] Figure 2 (a) is the grating image before distortion of Example 1.
[0040] Figure 2 (b) is the distorted grating image of Example 1.
[0041] Figure 3 (a) is the two-dimensional grating pattern of Example 1.
[0042] Figure 3 (b) is the two-dimensional grating image taken in Example 1.
[0043] Figure 3 (c) is the same as that of Example 1 Figure 3 (b) One-dimensional grating image obtained by filtering.
[0044] Figure 3 (d) is the insertion of Example 1 Figure 3 (c) Matrix of pixel values of the reference image.
[0045] Figure 3 (e) is the reference image matrix after difference in Example 1.
[0046] Figure 3 (f) is the first row phase distribution diagram of Example 1.
[0047] FIG4( a ) is a distorted image of Example 2. FIG.
[0048] FIG. 4( b ) is a one-dimensional grating image after horizontal filtering of Example 2. FIG.
[0049] FIG. 4( c ) is a one-dimensional grating image after vertical filtering in Example 2. FIG.
[0050] FIG4(d) shows the horizontal phase distribution of Example 2. ...
[0051] FIG4(e) shows the vertical phase distribution of Example 2.
[0052] FIG4( f ) is the corrected image of Example 2. DETAILED DESCRIPTION
[0053] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0054] Example 1
[0055] Figure 1 FIG. 1 is a flow chart of the image distortion correction method of the present invention. Figure 1 As shown, an image distortion correction method includes:
[0056] S1, photographing a grating image, wherein the grating image contains distortion; specifically, step S1 includes: grating pattern production and grating image photography; wherein the grating pattern production adopts one of projection, printing or washing, the grating pattern adopts a rectangular array, and the array spacing of the grating pattern is 5 to 20 pixels. The grating image photography includes: photographing the grating pattern with a distortion camera, the angle between the photographic axis of the distortion camera and the grating pattern plane is between 85° and 95°, and the grating pattern exceeds the camera frame. Figure 2 (a) is the grating image before distortion of Example 1. Figure 2 (b) is the distorted grating image of Example 1.
[0057] When the photographic axis is perpendicular to the grating plane, the grating plane is parallel to the image plane, otherwise the grating image will be perspectively deformed; in order to ensure that the photographed two-dimensional grating can be filtered to obtain a one-dimensional grating, when the grating direction and the image coordinate direction are less than 5°, it can be ensured that the two-dimensional grating image is filtered to obtain a one-dimensional grating image, and the value of the grating period has an effect on the displacement correction result of less than 1%; in the grating image photographed in this embodiment, the grating pattern should cover the entire screen, especially the four corners of the image, because the distortion of the four corners of the image is the largest, and only after displacement correction can the accuracy of subsequent three-dimensional reconstruction or photogrammetry be guaranteed.
[0058] S2, calculate the phase distribution of the grating image to obtain the horizontal phase distribution Q x and vertical phase distribution Q y ;
[0059] In this embodiment, the calculation of the phase distribution of the grating image is the key to distortion correction. Step S2 includes:
[0060] S21, performing horizontal filtering and vertical filtering on the grating image respectively to obtain a first one-dimensional grating image and a second one-dimensional grating image which are perpendicular to each other, and determining a sampling interval P according to the first one-dimensional grating image and the second one-dimensional grating image k ; The sampling interval is the integer closest to the one-dimensional grating period; the filtering method adopts low-pass filtering or mean filtering.
[0061] S22, calculating the horizontal phase distribution Q of the first one-dimensional grating image x , calculate the vertical phase distribution Q of the second one-dimensional grating image y ;
[0062] In this embodiment, the horizontal phase distribution Q of the first one-dimensional grating image is calculated. x Steps and calculation of the vertical phase distribution Q of the second one-dimensional grating image y The steps are the same as in Figure 3 (a)- Figure 3 (f), the calculation of the horizontal phase distribution Q of the first one-dimensional grating image x include:
[0063] The first one-dimensional grating image ( Figure 3 (c)) The first pixel value of the first row is taken out and placed in the blank reference image matrix ( Figure 3 (d)) and then sequentially place the first row of the first one-dimensional grating image in mP k The pixel value at +1 is taken and placed in the first row mP of the reference image matrix k +1, where m = 1, 2, 3, ...;
[0064] Using the same method as above, the k+1th pixel value of the first row of the first one-dimensional raster image is taken out and placed in the k+1th row and k+1th column of the reference image matrix, and then the pixel values mP in the first row of the first one-dimensional raster image are sequentially replaced. k The pixel value at position +k+2 is taken and placed in the k+1th row mP of the reference image matrix. k +k+2, where k=0,1,2,……P k -1;
[0065] Interpolate the pixel values of the blank positions in the reference image matrix and obtain P k Phase shifted moiré fringes, the phase difference between each moiré fringes is 2π / P k ; The method for interpolating the pixel values of the blank positions in the reference image matrix adopts one of linear interpolation, quadratic interpolation, cubic interpolation and spline interpolation.
[0066] Perform discrete Fourier transform on each row of the first one-dimensional grating image to obtain the phase distribution Q of the first one-dimensional grating image. x .
[0067] In this embodiment, the sampled pixel values need to be interpolated to obtain phase-shifted moiré fringes. Figure 3 (c) The pixel values at the blank positions in the same row are calculated from the existing pixel values. The interpolation methods used can be linear interpolation, quadratic interpolation, cubic interpolation, and spline interpolation. Different interpolation methods will lead to slightly different measurement results. In addition, due to the different computational complexities of different interpolation methods, the calculation time of different interpolation methods will also be different. After the above steps, P can be obtained. k Phase shifted moiré fringes, the phase difference between each moiré fringes is 2π / P k The calculation formula of the gray value of the phase-shifted moiré fringe is:
[0068]
[0069] Among them, P k is the sampling interval; k represents the moiré fringe number;
[0070] The phase distribution of the moiré fringes is:
[0071]
[0072] The above steps are performed on the second one-dimensional grating image to obtain the vertical phase distribution Q y .
[0073] In this embodiment, for an undistorted grating image, its phase distribution is an inclined plane, and for a distorted grating pattern, its phase distribution is a curved surface. The distorted image has no distortion at the principal point. The farther from the principal point, the greater the distortion. The principal point position is generally close to the center of the image. Since the second-order derivative of the inclined plane is zero, the principal point position can be determined using the second-order derivative of the phase distribution. In order to obtain the principal point position, the second-order derivative of the phase distribution Q" is calculated. x and Q" y ,
[0074] S23, respectively calculating the horizontal phase distribution Q x and vertical phase distribution Q y The second-order derivative of; the calculation formula is:
[0075]
[0076] f is the horizontal phase distribution Q x and vertical phase distribution Q y The i-th and j-th elements of .
[0077] S3, based on the horizontal phase distribution Q x and vertical phase distribution Q y calculating a distortion of the raster image;
[0078] It should be noted that distortion displacement is not displacement in the traditional sense, but a kind of imaging deviation, which is academically called distortion.
[0079] Step S3 includes: determining the principal point position (u0, v0) of the grating image, the formula is:
[0080] |Q″ x |+|Q″ y |=min(|Q″ x |+|Q″ y |)
[0081] Among them, Q x is the horizontal phase distribution Q x The second derivative and Q" y is the vertical phase distribution Q y The second-order derivative of the one-dimensional grating phase at the principal point is not absolutely zero due to image noise and other reasons, so it can traverse the entire image area so that |Q″ x |+|Q″ y |=min|Q″ x |+|Q″ y The point where the result is established is the main point position (u0,v0):
[0082] Based on the horizontal phase distribution Q x The second-order derivative Q"x Calculate the distortion δ of the first one-dimensional raster image x (i, j), based on the vertical phase distribution Q y The second-order derivative Q" y Calculate the distortion δ of the second one-dimensional raster image y (i, j), the calculation formulas are:
[0083]
[0084]
[0085] Among them, p t is the grating period.
[0086] S4, correcting the grating image based on the distortion.
[0087] It should be noted that in step S4, not only grating images can be corrected, but any image can also be corrected.
[0088] In this embodiment, the image distortion correction is to correct the image coordinates according to the calculated distortion, and the method adopted is to superimpose the distortion into the coordinate index of the distorted image, and then use the interpolation algorithm to re-interpolate the pixel value at the integer pixel position. Therefore, step S4 includes: according to the distortion δ x (i, j) and distortion δ y (i, j) The grating image coordinates are corrected, and the correction formula is:
[0089] This application uses the photographed grating pattern to obtain the phase change of the grating image in different directions, and uses the phase change to correct the image distortion, avoiding the problem of large errors caused by calculating the distortion through the distortion model, and this solution can correct the distortion when the distortion coefficient is unknown. In addition, this application uses a photographed grating pattern to calculate the phase of the distorted image, and the phase before distortion is obtained through the phase distribution characteristics of the distorted image, realizing high-precision distortion correction of a single distorted image.
[0090] Example 2
[0091] The difference between the present embodiment 1 and the embodiment 2 is that: firstly, a grating pattern with a period of 5 mm is printed, and a camera is aimed at the grating pattern plane to take a picture. The size of the image is 5340 pixels × 3890 pixels. The grating distortion image is shown in FIG4(a). Obvious image distortion can be seen by observing the surroundings of the image. The grating period is 8.3 pixels, and the grating sampling interval is 8 pixels. FIG4(b) shows the horizontal and vertical gratings obtained by filtering the two-dimensional grating with a low-pass filter. FIG4(c) shows the calculated phase distribution of the horizontal and vertical gratings. The phase distribution of the horizontal and vertical gratings is obtained by using |Q″ x|+|Q″ y |=min|Q″ x |+|Q″ y The calculated principal point position is (2518, 2131). After calculating the distortion, the distorted image is corrected. The corrected image is shown in Figure 4(d). It can be seen that there is no obvious distortion around the image after the distortion correction.
[0092] The above specific implementation modes are preferred embodiments of the present invention and cannot be used to limit the present invention. Any other changes or other equivalent replacement methods that do not deviate from the technical solution of the present invention are included in the protection scope of the present invention.
Claims
1. A method for correcting image distortion, characterized in that: include: S1, photographing a grating image, wherein the grating image contains distortion; S2, calculate the phase distribution of the grating image to obtain the horizontal phase distribution Q x and vertical phase distribution Q y ; S3, based on the horizontal phase distribution Q x and vertical phase distribution Q y calculating a distortion of the raster image; S4, correcting the grating image based on the distortion.
2. The image distortion correction method according to claim 1, characterized in that: Step S2 includes: S21, performing horizontal filtering and vertical filtering on the grating image respectively to obtain a first one-dimensional grating image and a second one-dimensional grating image which are perpendicular to each other, and determining a sampling interval P according to the first one-dimensional grating image and the second one-dimensional grating image k ; S22, calculating the horizontal phase distribution Q of the first one-dimensional grating image x , calculate the vertical phase distribution Q of the second one-dimensional grating image y ; S23, respectively calculating the horizontal phase distribution Q x and vertical phase distribution Q y The second-order derivative of; the calculation formula is: f is the horizontal phase distribution Q x and vertical phase distribution Q y The i-th and j-th elements of .
3. The image distortion correction method according to claim 2, characterized in that: Calculate the horizontal phase distribution Q of the first one-dimensional grating image x Steps and calculation of the vertical phase distribution Q of the second one-dimensional grating image y The steps of calculating the horizontal phase distribution Q of the first one-dimensional grating image are the same as x include: The first pixel value of the first row of the first one-dimensional raster image is taken out and placed at the first row and first column of the blank reference image matrix, and then the first pixel value of the first row of the first one-dimensional raster image is sequentially k The pixel value at +1 is taken and placed in the first row mP of the reference image matrix k +1, where m = 1, 2, 3, ...; Using the same method as above, the k+1th pixel value of the first row of the first one-dimensional raster image is taken out and placed in the k+1th row and k+1th column of the reference image matrix, and then the pixel values mP in the first row of the first one-dimensional raster image are sequentially replaced. k The pixel value at position +k+2 is taken and placed in the k+1th row mP of the reference image matrix. k +k+2, where k=0,1,2,……P k -1; Interpolate the pixel values of the blank positions in the reference image matrix and obtain P k Phase shifted moiré fringes, the phase difference between each moiré fringes is 2π / P k ; Perform discrete Fourier transform on each row of the first one-dimensional grating image to obtain the phase distribution Q of the first one-dimensional grating image. x .
4. The image distortion correction method according to claim 3, characterized in that: The calculation formula for the grayscale value of the phase-shifted moiré fringe is: Among them, P k is the sampling interval; k represents the moiré fringe number; The phase distribution of the phase-shifted moiré fringes is:
5. The image distortion correction method according to claim 3, characterized in that: Step S3 includes: determining the principal point position (u0, v0) of the grating image, the formula is: |Q″ x |+|Q″ y |=min(|Q″ x |+|Q″ y |) Among them, Q x is the horizontal phase distribution Q x The second derivative and Q" y is the vertical phase distribution Q y The second derivative of Based on the horizontal phase distribution Q x The second-order derivative Q" x Calculate the distortion δ of the first one-dimensional raster image x (i, j), based on the vertical phase distribution Q y The second-order derivative Q" y Calculate the distortion δ of the second one-dimensional raster image y (i, j), the calculation formulas are: Among them, p t is the grating period.
6. The image distortion correction method according to claim 1, characterized in that: Step S4 includes: according to the distortion δ x (i, j) and distortion δ y (i, j) The grating image coordinates are corrected, and the correction formula is:
7. The image distortion correction method according to claim 1, characterized in that: Step S1 includes: grating pattern production and grating image shooting; wherein, the grating image shooting includes: using a distortion camera to shoot the grating pattern, the angle between the photographic axis of the distortion camera and the grating pattern plane is between 85° and 95°, and the grating pattern exceeds the camera frame.
8. The image distortion correction method according to claim 7, characterized in that: The grating pattern is produced by projection, printing or washing, the grating pattern adopts a rectangular array, and the array spacing of the grating pattern is 5 to 20 pixels.
9. The image distortion correction method according to claim 1, characterized in that: The filtering method adopts low-pass filtering or mean filtering.
10. The image distortion correction method according to claim 1, characterized in that: The method for interpolating the pixel values of the blank positions in the reference image matrix adopts one of linear interpolation, quadratic interpolation, cubic interpolation and spline interpolation.