Free-form surface distortion correction method based on moving least square surface fitting

Abstract

The invention discloses a free-form surface distortion correction method based on moving least square surface fitting, and the method is characterized in that the method comprises the steps: S1, calibrating the internal and external parameters of a camera and the external parameters of a calibration plate through employing a Zhang Zhengyou calibration method; s2, according to the points on the calibration plate, calculating a function relationship between image coordinates and ideal image coordinates by using moving least square curved surface fitting; and S3, according to the function relationship, calculating positions of all pixel points in the ideal image corresponding to the original image to obtain a distortion correction matrix. According to the method, image distortion caused by the free-form surface lens can be effectively corrected, non-radial distortion caused by the free-form surface lens is further reduced on the basis of existing calibration, and the method is simple, easy to operate and effective through verification.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a free-form surface distortion correction method based on moving least squares surface fitting. Background Art

[0002] Freeform surface lens refers to a type of aspherical lens, whose surface is free-form and can be any continuous curved shape. This type of lens has unique advantages in optical design. The following are several main advantages of freeform surface lenses summarized from a professional optical design perspective:

[0003] Improved optical performance: Freeform lenses can effectively correct aberrations, including spherical aberration, coma, astigmatism, and field curvature, because they provide more degrees of freedom to control the path of light.

[0004] System simplification: Using freeform surfaces can reduce the number of lenses required in the system, as one freeform surface can potentially perform the functions of multiple traditional surfaces. This can reduce weight, size, and the number of components, thus simplifying system design.

[0005] Enhanced design flexibility: Free-form surfaces provide greater design space and flexibility, allowing designers to design more complex and precise optical systems to meet specific application requirements.

[0006] Improved imaging quality: Freeform lenses can provide better imaging performance and reduce optical distortion, especially in wide-angle lenses and high-performance imaging systems.

[0007] Improved system integration: Free-form lenses can improve the integration of optical systems by reducing the number of lens elements, thereby reducing space requirements. They are particularly suitable for aerospace, military and other occasions that require compact optical systems.

[0008] Improve system anti-interference ability: The asymmetry of free-form surfaces can provide better resistance to vibration and mechanical stress, which is an important feature for optical systems in harsh environments

[0009] Customized design: Free-form lenses can be customized according to the needs of specific applications to meet personalized optical performance requirements.

[0010] New application areas: The use of free-form surfaces opens up new application areas. For example, in head-mounted displays (such as VR and AR devices), free-form surfaces can provide better user experience and higher imaging quality.

[0011] Although free-form lenses offer many advantages, they are also relatively difficult and expensive to design and manufacture. Therefore, they are usually used in applications that require extremely high optical performance. As manufacturing technology advances and costs decrease, free-form lenses are expected to be widely used in more fields.

[0012] In order to meet the above advantages, the free-form surface lens is not designed to be radially symmetrical. The existing camera lens distortion models are basically radially symmetrical, so it does not meet the traditional camera lens distortion model: radial distortion and tangential distortion. This image distortion correction introduces new challenges.

[0013] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0014] To this end, the method proposed in the present invention can effectively correct the image distortion caused by the free-form surface lens, and further reduce the non-radial distortion caused by the free-form surface lens on the basis of the existing calibration. It is simple and easy to operate and has been proven to be effective.

[0015] The present invention provides a free-form surface distortion correction method based on moving least squares surface fitting, which is characterized by comprising:

[0016] Step S1: calibrate the camera's internal parameters, external parameters, and external parameters of the calibration board using Zhang Zhengyou's calibration method;

[0017] Step S2: according to the points on the calibration plate, a functional relationship between the image coordinates and the ideal image coordinates is calculated using a moving least squares surface fitting method;

[0018] Step S3: According to the functional relationship, the positions of all pixels in the ideal image corresponding to the original image are calculated to obtain a distortion correction matrix.

[0019] Optionally, the free-form surface distortion correction method based on moving least squares surface fitting is characterized in that step S1 comprises:

[0020] Step S11: placing the calibration plate at a first position and using a camera to capture a first image;

[0021] Step S12: placing the calibration plate at a second position, and using the camera to capture a second image; the first position and the second position are at different distances from the camera;

[0022] Step S13: Calibrate the intrinsic parameters and extrinsic parameters of the camera using the Zhang Zhengyou calibration method according to the first image and the second image, and extract the landmark points of the calibration plate on the first image and the second image respectively to obtain image point pairs.

[0023] Optionally, the free-form surface distortion correction method based on moving least squares surface fitting is characterized in that step S2 comprises:

[0024] Step S21: Calculating the ideal point of the corner point according to the intrinsic parameters of the camera, the extrinsic parameters of the camera and the extrinsic parameters of the calibration plate;

[0025] Step S22: constructing a fitting function using a moving least squares surface fitting method;

[0026] Step S23: Determine the parameters of the fitting function according to the ideal point and coordinates of the corner point.

[0027] Optionally, the free-form surface distortion correction method based on moving least squares surface fitting is characterized in that, in step S22, the basis function is a quadratic basis function p(x)=(1,x,y,x 2 ,xy,y 2 ) T .

[0028] Optionally, the free-form surface distortion correction method based on moving least squares surface fitting is characterized in that, in step S22, the weight function is a cubic spline function.

[0029] Optionally, the free-form surface distortion correction method based on moving least squares surface fitting is characterized in that, in step S22, the size of the support domain is determined by the surface design of the lens.

[0030] Optionally, the free-form surface distortion correction method based on moving least squares surface fitting is characterized in that the larger the slope of the surface, the smaller the support domain.

[0031] Optionally, the free-form surface distortion correction method based on moving least squares surface fitting is characterized in that the area of ​​each support region on the surface is equal.

[0032] Optionally, the free surface distortion correction method based on moving least squares surface fitting is characterized in that, in step S22, the surface direction of the lens is determined according to the ideal point of the corner point, and then a fitting function is constructed according to the moving least squares surface fitting method.

[0033] Optionally, the free-form surface distortion correction method based on moving least squares surface fitting is characterized in that step S3 comprises:

[0034] Step S31: Calculate the pixel points according to the functional relationship to establish a pixel-by-pixel correspondence between the ideal image and the original image;

[0035] Step S32: According to the corresponding relationship, a bilinear interpolation method is used to establish a corresponding function between the ideal image and the original image;

[0036] Step S33: generating a distortion correction matrix according to the corresponding function.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention firstly performs preliminary calibration on the camera using the Zhang Zhengyou calibration method, and then calculates the distortion correction matrix according to the points on the calibration plate using the moving least square surface fitting function relationship, thereby reducing the non-radial distortion brought to the free-form surface lens in the prior art, and the operation is simple and the correction effect is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0040] Figure 1 It is a flowchart of the steps of a free-form surface distortion correction method based on moving least squares surface fitting in an embodiment of the present invention;

[0041] Figure 2 Schematic diagram of a non-rotationally symmetric lateral distortion in an embodiment of the present invention;

[0042] Figure 3 Schematic diagram of a non-rotationally symmetric longitudinal distortion in an embodiment of the present invention;

[0043] Figure 4 A flowchart of the steps of calibration using Zhang Zhengyou calibration method in an embodiment of the present invention;

[0044] Figure 5 A flowchart of a step of calculating a functional relationship in an embodiment of the present invention;

[0045] Figure 6A schematic diagram of the relationship between a support domain and a curved surface in an embodiment of the present invention;

[0046] Figure 7 The figure is a flowchart of the steps of calculating a distortion correction matrix in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0048] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] The embodiment of the present invention provides a free-form surface distortion correction method based on moving least squares surface fitting, aiming to solve the problems existing in the prior art.

[0050] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0051] The present invention effectively corrects the image distortion caused by the free-form surface lens, further reduces the non-radial distortion caused by the free-form surface lens on the basis of existing calibration, is simple and easy to operate, and has been proven to be effective.

[0052] Figure 1 FIG. 1 is a flow chart of the steps of a free-form surface distortion correction method based on moving least squares surface fitting in an embodiment of the present invention. Figure 1 As shown, in an embodiment of the present invention, a method for correcting distortion of a free-form surface based on moving least squares surface fitting includes the following steps:

[0053] Step S1: Use Zhang Zhengyou calibration method to calibrate the camera's intrinsic parameters, extrinsic parameters, and extrinsic parameters of the calibration board.

[0054] In this step, calibration is performed by taking multiple pictures of a calibration plate (usually a chessboard) at different angles and positions. Multiple sets of calibration plate pictures fill the camera's acquisition window, so that each pixel can obtain a calibrated image. The Zhang Zhengyou calibration method is used to calculate the camera's internal parameters (focal length, principal point coordinates, etc.) and external parameters (rotation matrix and translation vector). These parameters describe the relative position relationship between the camera and the calibration plate. Determine the parameters of the calibration plate itself, such as grid size, corner point coordinates, etc. These parameters are used for subsequent conversion between image points and ideal image coordinates. Figure 2 and Figure 3 As shown, when the mirror surface is non-rotationally symmetric, the lateral and longitudinal distortion laws are quite different, which will cause a large difference between the calibration result of this step and the lens itself. Figure 2 and Figure 3 Distortion refers to the degree of distortion, and Percent refers to the proportion.

[0055] Step S2: According to the points on the calibration plate, a functional relationship between the image coordinates and the ideal image coordinates is calculated using a moving least squares surface fitting method.

[0056] In this step, several feature points are selected on the calibration plate, usually corner points or marking points are selected as feature points. Determine the coordinates of the feature points in the distorted image and their coordinates in the ideal (distortion-free) image. The ideal coordinates can be calculated by the design size of the calibration plate and the intrinsic and extrinsic parameters of the camera. Using the moving least squares surface fitting technology, a surface or function is fitted based on these corresponding points. The function can describe the mapping relationship from the distorted image coordinates to the ideal image coordinates. Moving least squares surface fitting is a localized fitting method that can find the best fitting surface in each local area, thereby more accurately reflecting the distortion.

[0057] Step S3: According to the functional relationship, the positions of all pixels in the ideal image corresponding to the original image are calculated to obtain a distortion correction matrix.

[0058] In this step, the position of each pixel in the ideal image corresponding to the original distorted image is calculated according to the functional relationship obtained by fitting, so as to construct a distortion correction matrix. For each pixel in the ideal image, the corresponding position in the original distorted image is reversely calculated using the functional relationship obtained in step S2. Due to the finiteness of the image, all pixels can be calculated in units of pixels to obtain the distortion correction matrix of all pixels on the image.

[0059] This embodiment obtains the internal and external parameters of the camera and the external parameters of the calibration plate through camera calibration, and then uses the moving least squares surface fitting technology to calculate the functional relationship between the image coordinates and the ideal image coordinates. Finally, the distortion correction matrix (or the mapping relationship for realizing distortion correction) is calculated according to this functional relationship. It can more accurately correct situations with complex distortions such as free-form surfaces, and is suitable for scenes requiring high-precision image processing.

[0060] Figure 4 FIG. 1 is a flow chart of the steps of calibration using Zhang Zhengyou calibration method in an embodiment of the present invention. Figure 4 As shown, in an embodiment of the present invention, a calibration step using the Zhang Zhengyou calibration method includes:

[0061] Step S11: placing the calibration plate at a first position, and using a camera to capture a first image.

[0062] In this step, prepare a calibration plate of known size and pattern (usually a checkerboard pattern) and ensure that its pattern is clear and the edges are distinct. Place the calibration plate steadily in the camera's field of view and ensure that it is fully visible and unobstructed. Use the camera to shoot the calibration plate from a fixed position to ensure that the image is clear and in focus.

[0063] Step S12: placing the calibration plate at a second position, and using the camera to capture a second image.

[0064] In this step, the first position and the second position are at different distances from the camera. By changing the position of the calibration plate (or the position of the camera), more calibration images at different angles and postures are obtained to increase the diversity and accuracy of the calibration data. Move the calibration plate to a position different from the first position to ensure that its distance from the camera changes. Keep the position and settings of the camera unchanged (or make minor adjustments as needed) and take another photo of the calibration plate. Repeat this process to take multiple images of the calibration plate at different positions and postures to increase the redundancy and robustness of the calibration data.

[0065] Step S13: Calibrate the intrinsic parameters and extrinsic parameters of the camera using the Zhang Zhengyou calibration method according to the first image and the second image, and extract the landmark points of the calibration plate on the first image and the second image respectively to obtain image point pairs.

[0066] In this step, by analyzing the correspondence between the corner point positions of the calibration plate in the calibration image and the known pattern, the camera's intrinsic parameters (such as focal length, optical center, etc.) and extrinsic parameters (such as the position and posture of the camera in the world coordinate system) are solved. During processing, each calibration image is preprocessed, including grayscale, filtering and denoising, etc., to improve the accuracy of corner point detection. Corner points in the calibration image are detected using a corner point detection algorithm (such as Harris corner point detection, Shi-Tomasi corner point detection, etc.). Based on the known calibration plate pattern and the detected corner point positions, the correspondence between the corner points in the world coordinate system and the image coordinate system is calculated. Using the mathematical model and algorithm of Zhang Zhengyou's calibration method, the camera's intrinsic and extrinsic parameters are solved through an optimization algorithm (such as the least squares method).

[0067] Select the marker points on the calibration plate image and construct the image point pairs through the positions on the image at different distances. The image point pairs are used in the subsequent processing. The marker points can be the corner points of the calibration plate or the points set in the calibration plate.

[0068] Figure 5 FIG. 1 is a flow chart of the steps of calculating a functional relationship in an embodiment of the present invention. Figure 5 As shown, a step of calculating a functional relationship in an embodiment of the present invention includes:

[0069] Step S21: Calculate the ideal point of the corner point according to the intrinsic parameters of the camera, the extrinsic parameters of the camera and the extrinsic parameters of the calibration plate.

[0070] In this step, the calibration results of the camera and the design parameters of the calibration plate are used to calculate the positions (i.e., ideal points) of the corner points on the calibration plate in the image without distortion. The coordinates of each corner point on the calibration plate in the world coordinate system are known (this is usually determined by the design size and layout of the calibration plate). Using the camera's intrinsic and extrinsic parameters, these world coordinates can be transformed into the camera's image coordinate system to obtain the ideal image coordinates of each corner point. This conversion process usually involves the conversion from the world coordinate system to the camera coordinate system (using extrinsic parameters), and the conversion from the camera coordinate system to the image coordinate system (using intrinsic parameters).

[0071] Step S22: constructing a fitting function using a moving least squares surface fitting method.

[0072] In this step, a mathematical function (or surface) is constructed that can describe the mapping relationship from the distorted image coordinates to the ideal image coordinates. The moving least squares surface fitting method is a localized fitting method that can find the best fitting surface in each local area. Select appropriate basis functions (such as polynomials, radial basis functions, etc.) and weight functions to define the fitting surface. Determine the parameterization method of the fitting surface, for example, use image coordinates (x, y) as input and output the corresponding distortion correction amount (dx, dy).

[0073] Step S23: Determine the parameters of the fitting function according to the ideal point and coordinates of the corner point.

[0074] In this step, the parameters of the fitting function are solved using the ideal position of the corner point calculated in step S21 and the coordinate position in the actual image. The adjustment of the parameters can be determined according to the number and data of the calibration points. The more calibration points and the more images collected, the more accurate the fitting function parameters can be, and the more accurate the parameters are.

[0075] There are several difficulties and key points in the implementation of this embodiment:

[0076] 1. Complexity of Moving Least Squares Surface Fitting

[0077] Selection of basis functions and weight functions: The selection of basis functions and weight functions in the moving least squares method is crucial to the fitting effect, but there are currently no specific theoretical rules and they need to be determined through experience or experiments.

[0078] Computational complexity: As the amount of data increases, the computational complexity of the moving least squares method increases significantly, especially when processing high-resolution images, the computational complexity may become very large.

[0079] Invertibility of matrix A: When constructing the fitting function, it is necessary to solve the inverse of matrix A. When the number of basis functions m is large or the nodes are improperly arranged, matrix A may be ill-conditioned or singular, resulting in difficulty in solving or increased errors.

[0080] 2. Handling of nonlinear distortion: The distortion of free-form surfaces is often nonlinear, which increases the difficulty of correction. Although the moving least squares method can handle local nonlinear distortion well, further optimization may still be needed on a global scale.

[0081] 3. Calculation and algorithm complexity: When using moving least squares surface fitting to calculate the functional relationship between image coordinates and ideal image coordinates, a large number of mathematical operations and optimization problems are involved. Compared with the traditional least squares method, the moving least squares method has advantages in processing large amounts of discrete data and complex shapes, but it also brings greater calculation and algorithm complexity. In particular, the selection of support domain and weight function requires careful design to balance fitting accuracy and computational efficiency.

[0082] 4. Establishment of fitting function: The establishment of fitting function is not based on traditional polynomials or other functions, but is composed of a coefficient vector and basis functions, where the coefficient vector is a function of the coordinates. Compared with traditional methods, this method provides higher flexibility and adaptability, but also requires researchers to have higher mathematical modeling capabilities to ensure that the fitting function can accurately reflect the relationship between the image coordinates and the ideal image coordinates.

[0083] 5. Determination of the support domain: How to reasonably divide the support domain and define the weight function on this basis to ensure the accuracy and stability of the fitting results is a technical challenge.

[0084] In order to at least partially solve the above technical problems, the present invention further provides the following improvement scheme for this embodiment:

[0085] In some embodiments, in step S22, the basis function is a quadratic basis function p(x)=(1, x, y, x 2 ,xy,y 2 ) T In view of the characteristics of the image, the present embodiment sets the basis function to be a quadratic basis function, which can control the amount of calculation while ensuring the calculation accuracy.

[0086] In some embodiments, in step S22, the weight function is a cubic spline function ω(s)= Where r is the maximum distance from the center of the support domain. The weight function should be non-negative and monotonically decreasing as the support domain expands from the center outward. The weight function should also have compact support, that is, it is not equal to 0 in the support domain (the influence area of ​​x) and is all 0 outside the support domain.

[0087] In some embodiments, in step S22, the size of the support region is determined by the curved surface design of the lens. Figure 6 As shown, the support domain refers to the size of the image. Figure 6The plane where support domain 1 and support domain 2 are located is the imaging module of the sensor. The sizes of different support domains can be different, and the specific size is determined by the curved surface corresponding to the light path irradiated thereon. Since the lens surface is irregular, the size of the support domain is determined by the area of ​​the lens surface in this embodiment, and the light path of the calibration plate is comprehensively considered, so that the size of the support domain can more reasonably represent the influence of the light path.

[0088] In some embodiments, the greater the slope of the curved surface, the smaller the support domain. The greater the slope of the curved surface, the more severe the change of the light path at that position on the curved surface. In this case, if the support domain is small, more feature points will be collected, so that the range of each feature point and the support domain is smaller, so that the fitting effect is better and the accuracy is improved.

[0089] In some embodiments, the area of ​​each support region on the curved surface is equal. Figure 6 As shown, the corresponding optical path areas of support domain 1 and support domain 2 on curved surface 3 are the same, but the areas of support domain 1 and support domain 2 are different. The corresponding areas of each support domain on the curved surface are equal, so that the optical path range affected by each support domain is equal, thereby better balancing the impact of the optical path.

[0090] In some embodiments, in step S22, the curved surface direction of the lens is determined according to the ideal point of the corner point, and then a fitting function is constructed according to the moving least squares surface fitting method. Since the curved surface of the lens is asymmetric, it is different in each direction. In this embodiment, the curved surface direction of the lens is determined according to the ideal point of the corner point in combination with the design of the lens, so that the construction of the fitting function is more accurate.

[0091] Figure 7 FIG. 1 is a flowchart of a step of calculating a distortion correction matrix in an embodiment of the present invention. Figure 7 As shown, a step of calculating a distortion correction matrix in an embodiment of the present invention includes:

[0092] Step S31: Calculate the pixel points according to the functional relationship to establish a pixel-by-pixel correspondence between the ideal image and the original image.

[0093] In this step, the distortion correction model (fitting function) obtained in step S2 is used to calculate the corresponding position of each pixel in the ideal image in the original distorted image. For each pixel (x', y') in the ideal image, the distortion correction model (which may be a complex function or mapping table) is used to calculate its corresponding position (x, y) in the original image. This process is usually reverse, that is, the mapping from the ideal image coordinates to the original image coordinates. It should be noted that the calculated original image coordinates (x, y) may be non-integer values, because distortion usually causes the position of the pixel to shift.

[0094] Step S32: According to the corresponding relationship, a bilinear interpolation method is used to establish a corresponding function between the ideal image and the original image.

[0095] In this step, since the original image coordinates calculated in step S31 may be non-integer values, we need to obtain the pixel values ​​at these non-integer coordinates by interpolation. Bilinear interpolation is a commonly used interpolation method, which can interpolate the pixel values ​​of non-integer coordinates between the four nearest integer coordinate points. For each non-integer coordinate (x, y) calculated in step S31, find the four nearest integer coordinate points around it. Using the pixel values ​​at these four integer coordinate points, the pixel values ​​at the non-integer coordinates (x, y) are calculated by the bilinear interpolation formula. In this way, we have established a corresponding function from the ideal image to the original image (actually the pixel values ​​obtained by interpolation).

[0096] Step S33: generating a distortion correction matrix according to the corresponding function.

[0097] In this step, by generating a distortion correction matrix, the camera does not need to perform complex calculations during use, and can directly use the distortion correction matrix to obtain a corrected image through simple calculations. The distortion correction matrix is ​​a sparse matrix with the same size as the image, in which each element represents a mapping from the ideal image coordinates to the original image (or after interpolation) coordinates. Traverse each pixel point in the ideal image, and use the method in steps S31 and S32 to calculate the corresponding pixel value of the original image (or after interpolation). Arrange these calculated pixel values ​​in the coordinate order of the ideal image to generate a corrected image.

[0098] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0099] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A free-form surface distortion correction method based on moving least squares surface fitting, characterized in that: include: Step S1: calibrate the camera's internal parameters, external parameters, and external parameters of the calibration board using Zhang Zhengyou's calibration method; Step S2: according to the points on the calibration plate, a functional relationship between the image coordinates and the ideal image coordinates is calculated using a moving least squares surface fitting method; Step S3: According to the functional relationship, the positions of all pixels in the ideal image corresponding to the original image are calculated to obtain a distortion correction matrix.

2. The free-form surface distortion correction method based on moving least squares surface fitting according to claim 1, characterized in that: Step S1 includes: Step S11: placing the calibration plate at a first position and using a camera to capture a first image; Step S12: placing the calibration plate at a second position, and using the camera to capture a second image; the first position and the second position are at different distances from the camera; Step S13: Calibrate the intrinsic parameters and extrinsic parameters of the camera using the Zhang Zhengyou calibration method according to the first image and the second image, and extract the landmark points of the calibration plate on the first image and the second image respectively to obtain image point pairs.

3. The free-form surface distortion correction method based on moving least squares surface fitting according to claim 1, characterized in that: Step S2 includes: Step S21: Calculating the ideal point of the corner point according to the intrinsic parameters of the camera, the extrinsic parameters of the camera and the extrinsic parameters of the calibration plate; Step S22: constructing a fitting function using a moving least squares surface fitting method; Step S23: Determine the parameters of the fitting function according to the ideal point and coordinates of the corner point.

4. The free-form surface distortion correction method based on moving least squares surface fitting according to claim 3, characterized in that: In step S22, the basis function is a quadratic basis function p(x)=(1,x,y,x 2 ,xy,y 2 ) T .

5. The free-form surface distortion correction method based on moving least squares surface fitting according to claim 3, characterized in that: In step S22, the weight function is a cubic spline function 6. The free-form surface distortion correction method based on moving least squares surface fitting according to claim 3, characterized in that: In step S22, the size of the support region is determined by the curved surface design of the lens.

7. The free-form surface distortion correction method based on moving least squares surface fitting according to claim 6, characterized in that: The greater the slope of the curved surface, the smaller the support region.

8. The free-form surface distortion correction method based on moving least squares surface fitting according to claim 6, characterized in that: The area of ​​each of the support regions on the curved surface is equal.

9. The free-form surface distortion correction method based on moving least squares surface fitting according to claim 3, characterized in that: In step S22, the curved surface direction of the lens is determined according to the ideal point of the corner point, and then a fitting function is constructed according to a moving least squares curved surface fitting method.

10. The free-form surface distortion correction method based on moving least squares surface fitting according to claim 1, characterized in that: Step S3 includes: Step S31: Calculate the pixel points according to the functional relationship to establish a pixel-by-pixel correspondence between the ideal image and the original image; Step S32: According to the corresponding relationship, a bilinear interpolation method is used to establish a corresponding function between the ideal image and the original image; Step S33: generating a distortion correction matrix according to the corresponding function.

Images