Lens distortion correction method, system and device and storage medium

Abstract

The invention discloses a lens distortion correction method, system and device and a storage medium, and the method comprises the steps: S1, calibrating the internal and external parameters of a camera according to a calibration plate image, and obtaining the external parameters of a calibration plate; s2, calculating a function relationship between the image coordinates and ideal image coordinates by using moving least square curved surface fitting; and S3, according to the function relationship, calculating positions of all pixel points in the ideal image corresponding to the original image to obtain a distortion correction matrix. According to the method, image distortion caused by the free-form surface lens can be effectively corrected, non-radial distortion caused by the free-form surface lens is further reduced on the basis of existing calibration, and the method is simple, easy to operate and effective through verification.

Description

Background Art

[0002] Freeform surface lens refers to a type of aspherical lens, whose surface is free-form and can be any continuous curved shape. This type of lens has unique advantages in optical design. The following are several main advantages of freeform surface lenses summarized from a professional optical design perspective:

[0003] Improved optical performance: Freeform lenses can effectively correct aberrations, including spherical aberration, coma, astigmatism, and field curvature, because they provide more degrees of freedom to control the path of light.

[0004] System simplification: Using freeform surfaces can reduce the number of lenses required in the system, as one freeform surface can potentially perform the functions of multiple traditional surfaces. This can reduce weight, size, and the number of components, thus simplifying system design.

[0005] Enhanced design flexibility: Free-form surfaces provide greater design space and flexibility, allowing designers to design more complex and precise optical systems to meet specific application requirements.

[0006] Improved imaging quality: Freeform lenses can provide better imaging performance and reduce optical distortion, especially in wide-angle lenses and high-performance imaging systems.

[0007] Improved system integration: Free-form lenses can improve the integration of optical systems by reducing the number of lens elements, thereby reducing space requirements. They are particularly suitable for aerospace, military and other occasions that require compact optical systems.

[0008] Improve system anti-interference ability: The asymmetry of free-form surfaces can provide better resistance to vibration and mechanical stress, which is an important feature for optical systems in harsh environments

[0009] Customized design: Free-form lenses can be customized according to the needs of specific applications to meet personalized optical performance requirements.

[0010] New application areas: The use of free-form surfaces opens up new application areas. For example, in head-mounted displays (such as VR and AR devices), free-form surfaces can provide better user experience and higher imaging quality.

[0011] Although free-form lenses offer many advantages, they are also relatively difficult and expensive to design and manufacture. Therefore, they are usually used in applications that require extremely high optical performance. As manufacturing technology advances and costs decrease, free-form lenses are expected to be widely used in more fields.

[0012] In order to meet the above advantages, the free-form surface lens is not designed to be radially symmetrical. The existing camera lens distortion models are basically radially symmetrical, so it does not meet the traditional camera lens distortion model: radial distortion and tangential distortion. This image distortion correction introduces new challenges.

[0013] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0014] To this end, the method proposed in the present invention can effectively correct the image distortion caused by the free-form surface lens, and further reduce the non-radial distortion caused by the free-form surface lens on the basis of the existing calibration. It is simple and easy to operate and has been proven to be effective.

[0015] In a first aspect, the present invention provides a lens distortion correction method, characterized by comprising:

[0016] Step S1: Calibrate the intrinsic parameters and extrinsic parameters of the camera according to the calibration plate image, and obtain the extrinsic parameters of the calibration plate;

[0017] Step S2: using moving least squares surface fitting to calculate the functional relationship between the image coordinates and the ideal image coordinates;

[0018] Step S3: According to the functional relationship, the positions of all pixels in the ideal image corresponding to the original image are calculated to obtain a distortion correction matrix.

[0019] Optionally, the lens distortion correction method is characterized in that step S2 comprises:

[0020] Step S21: Calculating the ideal point of the corner point according to the intrinsic parameters of the camera, the extrinsic parameters of the camera and the extrinsic parameters of the calibration plate;

[0021] Step S22: constructing a fitting function using a moving least squares surface fitting method;

[0022] Step S23: Determine the parameters of the fitting function according to the ideal point and coordinates of the corner point.

[0023] Optionally, the lens distortion correction method is characterized in that, in step S22, the basis function is a quadratic basis function p(x)=(1,x,y,x 2 ,xy,y 2 ) T ;

[0024] The weight function is a cubic spline function

[0025] Optionally, the lens distortion correction method is characterized in that, in step S22, the size of the support domain is determined by the curved surface design of the lens.

[0026] Optionally, the lens distortion correction method is characterized in that the greater the slope of the curved surface, the smaller the support domain.

[0027] Optionally, the lens distortion correction method is characterized in that the area of ​​each support region on the curved surface is equal.

[0028] Optionally, the lens distortion correction method is characterized in that step S3 comprises:

[0029] Step S31: Calculate the pixel points according to the functional relationship to establish a pixel-by-pixel correspondence between the ideal image and the original image;

[0030] Step S32: According to the corresponding relationship, a bilinear interpolation method is used to establish a corresponding function between the ideal image and the original image;

[0031] Step S33: generating a distortion correction matrix according to the corresponding function.

[0032] In a second aspect, the present invention provides a lens distortion correction system, used to implement any of the above-mentioned lens distortion correction methods, characterized in that it includes:

[0033] An initial calibration module, used to calibrate the intrinsic parameters and extrinsic parameters of the camera according to the calibration plate image, and obtain the extrinsic parameters of the calibration plate;

[0034] A fitting module, used to calculate the functional relationship between the image coordinates and the ideal image coordinates using a moving least squares surface fitting according to the points on the calibration plate;

[0035] The distortion module is used to calculate the positions of all pixels in the ideal image corresponding to the original image according to the functional relationship to obtain a distortion correction matrix.

[0036] In a third aspect, the present invention provides a lens distortion correction device, characterized in that it includes:

[0037] processor;

[0038] a memory storing executable instructions of the processor;

[0039] The processor is configured to execute the steps of any of the aforementioned lens distortion correction methods by executing the executable instructions.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium for storing a program, wherein the program, when executed, implements the steps of any of the aforementioned lens distortion correction methods.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention firstly performs preliminary calibration on the camera using the Zhang Zhengyou calibration method, and then calculates the distortion correction matrix according to the points on the calibration plate using the moving least square surface fitting function relationship, thereby reducing the non-radial distortion brought to the free-form surface lens in the prior art, and the operation is simple and the correction effect is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0044] Figure 1 is a flowchart of the steps of a lens distortion correction method in an embodiment of the present invention;

[0045] Figure 2 Schematic diagram of a non-rotationally symmetric lateral distortion in an embodiment of the present invention;

[0046] Figure 3 Schematic diagram of a non-rotationally symmetric longitudinal distortion in an embodiment of the present invention;

[0047] Figure 4 A flowchart of a calibration step according to a calibration plate image in an embodiment of the present invention;

[0048] Figure 5 A flowchart of a step of calculating a functional relationship in an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of the relationship between a support domain and a curved surface in an embodiment of the present invention;

[0050] Figure 7 is a flow chart of steps for calculating a distortion correction matrix in an embodiment of the present invention;

[0051] Figure 8 Schematic diagram of the structure of a lens distortion correction system in an embodiment of the present invention;

[0052] Fig. 9is a structural schematic diagram of a lens distortion correction device in an embodiment of the present invention; and

[0053] Fig.10 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0055] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0056] A lens distortion correction method provided by an embodiment of the present invention is intended to solve the problems existing in the prior art.

[0057] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0058] The present invention effectively corrects the image distortion caused by the free-form surface lens, further reduces the non-radial distortion caused by the free-form surface lens on the basis of existing calibration, is simple and easy to operate, and has been proven to be effective.

[0059] Figure 1 FIG. 1 is a flowchart of a lens distortion correction method according to an embodiment of the present invention. Figure 1 As shown, the steps of a lens distortion correction method in an embodiment of the present invention include:

[0060] Step S1: Calibrate the intrinsic parameters and extrinsic parameters of the camera according to the calibration plate image, and obtain the extrinsic parameters of the calibration plate.

[0061] In this step, calibration is performed by taking multiple pictures of a calibration plate (usually a chessboard) at different angles and positions. Multiple groups of calibration plate pictures fill the camera's acquisition window, so that a calibrated image can be obtained for each pixel. The calibration plate image is used to calculate the camera's internal parameters (focal length, principal point coordinates, etc.) and external parameters (rotation matrix and translation vector). These parameters describe the relative position relationship between the camera and the calibration plate. Using calibration plate images to calibrate a camera is a commonly used technical means in the field. Various methods of calibrating a camera using calibration plate images in the prior art can be applied to this step, and will not be repeated in this embodiment. Determine the parameters of the calibration plate itself, such as grid size, corner point coordinates, etc. These parameters are used for subsequent conversion between image points and ideal image coordinates. Figure 2 and Figure 3 As shown, when the mirror surface is non-rotationally symmetric, the lateral and longitudinal distortion laws are quite different, which will cause a large difference between the calibration result of this step and the lens itself. Figure 2 and Figure 3 Distortion refers to the degree of distortion, and Percent refers to the proportion.

[0062] Step S2: Calculate the functional relationship between the image coordinates and the ideal image coordinates using moving least squares surface fitting.

[0063] In this step, several feature points are selected on the calibration plate, usually corner points or marking points are selected as feature points. Determine the coordinates of the feature points in the distorted image and their coordinates in the ideal (distortion-free) image. The ideal coordinates can be calculated by the design size of the calibration plate and the intrinsic and extrinsic parameters of the camera. Using the moving least squares surface fitting technology, a surface or function is fitted based on these corresponding points. The function can describe the mapping relationship from the distorted image coordinates to the ideal image coordinates. Moving least squares surface fitting is a localized fitting method that can find the best fitting surface in each local area, thereby more accurately reflecting the distortion.

[0064] Step S3: According to the functional relationship, the positions of all pixels in the ideal image corresponding to the original image are calculated to obtain a distortion correction matrix.

[0065] In this step, the position of each pixel in the ideal image corresponding to the original distorted image is calculated according to the functional relationship obtained by fitting, so as to construct a distortion correction matrix. For each pixel in the ideal image, the corresponding position in the original distorted image is reversely calculated using the functional relationship obtained in step S2. Due to the finiteness of the image, all pixels can be calculated in units of pixels to obtain the distortion correction matrix of all pixels on the image.

[0066] This embodiment obtains the internal and external parameters of the camera and the external parameters of the calibration plate through camera calibration, and then uses the moving least squares surface fitting technology to calculate the functional relationship between the image coordinates and the ideal image coordinates. Finally, the distortion correction matrix (or the mapping relationship for realizing distortion correction) is calculated according to this functional relationship. It can more accurately correct situations with complex distortions such as free-form surfaces, and is suitable for scenes requiring high-precision image processing.

[0067] Figure 4 FIG. 1 is a flow chart of the steps of calibration according to the calibration plate image in an embodiment of the present invention. Figure 4 As shown, in an embodiment of the present invention, a step of calibrating according to a calibration plate image includes:

[0068] Step S11: placing the calibration plate at a first position, and using a camera to capture a first image.

[0069] In this step, prepare a calibration plate of known size and pattern (usually a checkerboard pattern) and ensure that its pattern is clear and the edges are distinct. Place the calibration plate steadily in the camera's field of view and ensure that it is fully visible and unobstructed. Use the camera to shoot the calibration plate from a fixed position to ensure that the image is clear and in focus.

[0070] Step S12: placing the calibration plate at a second position, and using the camera to capture a second image.

[0071] In this step, the first position and the second position are at different distances from the camera. By changing the position of the calibration plate (or the position of the camera), more calibration images at different angles and postures are obtained to increase the diversity and accuracy of the calibration data. Move the calibration plate to a position different from the first position to ensure that its distance from the camera changes. Keep the position and settings of the camera unchanged (or make minor adjustments as needed) and take another photo of the calibration plate. Repeat this process to take multiple images of the calibration plate at different positions and postures to increase the redundancy and robustness of the calibration data.

[0072] Step S13: Calibrate the intrinsic parameters and extrinsic parameters of the camera using the Zhang Zhengyou calibration method according to the first image and the second image, and extract the landmark points of the calibration plate on the first image and the second image respectively to obtain image point pairs.

[0073] In this step, by analyzing the correspondence between the corner point positions of the calibration plate in the calibration image and the known pattern, the camera's intrinsic parameters (such as focal length, optical center, etc.) and extrinsic parameters (such as the position and posture of the camera in the world coordinate system) are solved. During processing, each calibration image is preprocessed, including grayscale, filtering and denoising, etc., to improve the accuracy of corner point detection. Corner points in the calibration image are detected using a corner point detection algorithm (such as Harris corner point detection, Shi-Tomasi corner point detection, etc.). Based on the known calibration plate pattern and the detected corner point positions, the correspondence between the corner points in the world coordinate system and the image coordinate system is calculated. Using the mathematical model and algorithm of Zhang Zhengyou's calibration method, the camera's intrinsic and extrinsic parameters are solved through an optimization algorithm (such as the least squares method).

[0074] Select the marker points on the calibration plate image and construct the image point pairs through the positions on the image at different distances. The image point pairs are used in the subsequent processing. The marker points can be the corner points of the calibration plate or the points set in the calibration plate.

[0075] Figure 5 FIG. 1 is a flow chart of the steps of calculating a functional relationship in an embodiment of the present invention. Figure 5 As shown, a step of calculating a functional relationship in an embodiment of the present invention includes:

[0076] Step S21: Calculate the ideal point of the corner point according to the intrinsic parameters of the camera, the extrinsic parameters of the camera and the extrinsic parameters of the calibration plate.

[0077] In this step, the calibration results of the camera and the design parameters of the calibration plate are used to calculate the positions (i.e., ideal points) of the corner points on the calibration plate in the image without distortion. The coordinates of each corner point on the calibration plate in the world coordinate system are known (this is usually determined by the design size and layout of the calibration plate). Using the camera's intrinsic and extrinsic parameters, these world coordinates can be transformed into the camera's image coordinate system to obtain the ideal image coordinates of each corner point. This conversion process usually involves the conversion from the world coordinate system to the camera coordinate system (using extrinsic parameters), and the conversion from the camera coordinate system to the image coordinate system (using intrinsic parameters).

[0078] Step S22: constructing a fitting function using a moving least squares surface fitting method.

[0079] In this step, a mathematical function (or surface) is constructed that can describe the mapping relationship from the distorted image coordinates to the ideal image coordinates. The moving least squares surface fitting method is a localized fitting method that can find the best fitting surface in each local area. Select appropriate basis functions (such as polynomials, radial basis functions, etc.) and weight functions to define the fitting surface. Determine the parameterization method of the fitting surface, for example, use image coordinates (x, y) as input and output the corresponding distortion correction amount (dx, dy).

[0080] Step S23: Determine the parameters of the fitting function according to the ideal point and coordinates of the corner point.

[0081] In this step, the parameters of the fitting function are solved using the ideal position of the corner point calculated in step S21 and the coordinate position in the actual image. The adjustment of the parameters can be determined according to the number and data of the calibration points. The more calibration points and the more images collected, the more accurate the fitting function parameters can be, and the more accurate the parameters are.

[0082] There are several difficulties and key points in the implementation of this embodiment:

[0083] 1. Complexity of Moving Least Squares Surface Fitting

[0084] Selection of basis functions and weight functions: The selection of basis functions and weight functions in the moving least squares method is crucial to the fitting effect, but there are currently no specific theoretical rules and they need to be determined through experience or experiments.

[0085] Computational complexity: As the amount of data increases, the computational complexity of the moving least squares method increases significantly, especially when processing high-resolution images, the amount of calculation may become very large.

[0086] Invertibility of matrix A: When constructing the fitting function, it is necessary to solve the inverse of matrix A. When the number of basis functions m is large or the nodes are improperly arranged, matrix A may be ill-conditioned or singular, resulting in difficulty in solving or increased errors.

[0087] 2. Handling of nonlinear distortion: The distortion of free-form surfaces is often nonlinear, which increases the difficulty of correction. Although the moving least squares method can handle local nonlinear distortion well, further optimization may still be needed on a global scale.

[0088] 3. Calculation and algorithm complexity: When using moving least squares surface fitting to calculate the functional relationship between image coordinates and ideal image coordinates, a large number of mathematical operations and optimization problems are involved. Compared with the traditional least squares method, the moving least squares method has advantages in processing large amounts of discrete data and complex shapes, but it also brings greater calculation and algorithm complexity. In particular, the selection of support domain and weight function requires careful design to balance fitting accuracy and computational efficiency.

[0089] 4. Establishment of fitting function: The establishment of fitting function is not based on traditional polynomials or other functions, but is composed of a coefficient vector and basis functions, where the coefficient vector is a function of the coordinates. Compared with traditional methods, this method provides higher flexibility and adaptability, but also requires researchers to have higher mathematical modeling capabilities to ensure that the fitting function can accurately reflect the relationship between the image coordinates and the ideal image coordinates.

[0090] 5. Determination of the support domain: How to reasonably divide the support domain and define the weight function on this basis to ensure the accuracy and stability of the fitting results is a technical challenge.

[0091] In order to at least partially solve the above technical problems, the present invention further provides the following improvement scheme for this embodiment:

[0092] In some embodiments, in step S22, the basis function is a quadratic basis function p(x)=(1, x, y, x 2 ,xy,y 2 ) T ; The weight function is a cubic spline function s'=xx i , Where r is the farthest distance of the support domain from the center point. In view of the characteristics of the image, this embodiment sets the basis function as a quadratic basis function, which can control the amount of calculation while ensuring the calculation accuracy. The weight function should be non-negative and monotonically decrease as the support domain expands from the center to the outside. The weight function should also have compact support, that is, it is not equal to 0 in the support domain (the influence area of ​​x) and is all 0 outside the support domain.

[0093] In some embodiments, in step S22, the size of the support region is determined by the curved surface design of the lens. Figure 6 As shown, the support domain refers to the size of the image. Figure 6 The plane where support domain 1 and support domain 2 are located is the imaging module of the sensor. The sizes of different support domains can be different, and the specific size is determined by the curved surface corresponding to the light path irradiated thereon. Since the lens surface is irregular, the size of the support domain is determined by the area of ​​the lens surface in this embodiment, and the light path of the calibration plate is comprehensively considered, so that the size of the support domain can more reasonably represent the influence of the light path.

[0094] In some embodiments, the greater the slope of the curved surface, the smaller the support domain. The greater the slope of the curved surface, the more severe the change of the light path at that position on the curved surface. In this case, if the support domain is small, more feature points will be collected, so that the range of each feature point and the support domain is smaller, so that the fitting effect is better and the accuracy is improved.

[0095] In some embodiments, the area of ​​each support region on the curved surface is equal. Figure 6 As shown, the corresponding optical path areas of support domain 1 and support domain 2 on curved surface 3 are the same, but the areas of support domain 1 and support domain 2 are different. The corresponding areas of each support domain on the curved surface are equal, so that the optical path range affected by each support domain is equal, thereby better balancing the impact of the optical path.

[0096] In some embodiments, in step S22, the curved surface direction of the lens is determined according to the ideal point of the corner point, and then a fitting function is constructed according to the moving least squares surface fitting method. Since the curved surface of the lens is asymmetric, it is different in each direction. In this embodiment, the curved surface direction of the lens is determined according to the ideal point of the corner point in combination with the design of the lens, so that the construction of the fitting function is more accurate.

[0097] Figure 7 FIG. 1 is a flowchart of a step of calculating a distortion correction matrix in an embodiment of the present invention. Figure 7 As shown, a step of calculating a distortion correction matrix in an embodiment of the present invention includes:

[0098] Step S31: Calculate the pixel points according to the functional relationship to establish a pixel-by-pixel correspondence between the ideal image and the original image.

[0099] In this step, the distortion correction model (fitting function) obtained in step S2 is used to calculate the corresponding position of each pixel in the ideal image in the original distorted image. For each pixel (x', y') in the ideal image, the distortion correction model (which may be a complex function or mapping table) is used to calculate its corresponding position (x, y) in the original image. This process is usually reverse, that is, the mapping from the ideal image coordinates to the original image coordinates. It should be noted that the calculated original image coordinates (x, y) may be non-integer values, because distortion usually causes the position of the pixel to shift.

[0100] Step S32: According to the corresponding relationship, a bilinear interpolation method is used to establish a corresponding function between the ideal image and the original image.

[0101] In this step, since the original image coordinates calculated in step S31 may be non-integer values, we need to obtain the pixel values ​​at these non-integer coordinates by interpolation. Bilinear interpolation is a commonly used interpolation method, which can interpolate the pixel values ​​of non-integer coordinates between the four nearest integer coordinate points. For each non-integer coordinate (x, y) calculated in step S31, find the four nearest integer coordinate points around it. Using the pixel values ​​at these four integer coordinate points, the pixel values ​​at the non-integer coordinates (x, y) are calculated by the bilinear interpolation formula. In this way, we have established a corresponding function from the ideal image to the original image (actually the pixel values ​​obtained by interpolation).

[0102] Step S33: generating a distortion correction matrix according to the corresponding function.

[0103] In this step, by generating a distortion correction matrix, the camera does not need to perform complex calculations during use, and can directly use the distortion correction matrix to obtain a corrected image through simple calculations. The distortion correction matrix is ​​a sparse matrix with the same size as the image, in which each element represents a mapping from the ideal image coordinates to the original image (or after interpolation) coordinates. Traverse each pixel point in the ideal image, and use the method in steps S31 and S32 to calculate the corresponding pixel value of the original image (or after interpolation). Arrange these calculated pixel values ​​in the coordinate order of the ideal image to generate a corrected image.

[0104] Figure 8 FIG. 1 is a schematic diagram of the structure of a lens distortion correction system according to an embodiment of the present invention. Figure 8 As shown, a lens distortion correction system in an embodiment of the present invention includes:

[0105] An initial calibration module, used to calibrate the intrinsic parameters and extrinsic parameters of the camera according to the calibration plate image, and obtain the extrinsic parameters of the calibration plate;

[0106] A fitting module, used to calculate the functional relationship between the image coordinates and the ideal image coordinates using a moving least squares surface fitting according to the points on the calibration plate;

[0107] The distortion module is used to calculate the positions of all pixels in the ideal image corresponding to the original image according to the functional relationship to obtain a distortion correction matrix.

[0108] This embodiment can effectively correct the image distortion caused by the free-form surface lens, and further reduce the non-radial distortion caused by the free-form surface lens on the basis of existing calibration. It is simple and easy to operate and has been proven to be effective.

[0109] The embodiment of the present invention further provides a lens distortion correction device, comprising a processor and a memory, wherein executable instructions of the processor are stored, wherein the processor is configured to execute the steps of a lens distortion correction method by executing the executable instructions.

[0110] As described above, this embodiment can effectively correct the image distortion caused by the free-form surface lens, and further reduce the non-radial distortion caused by the free-form surface lens on the basis of the existing calibration. It is simple and easy to operate and has been proven to be effective.

[0111] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0112] Fig. 9 Schematic diagram of the structure of a lens distortion correction device in an embodiment of the present invention. Fig. 9 The electronic device 600 according to this embodiment of the present invention is described. Fig. 9 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0113] like Fig. 9 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0114] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the above-mentioned lens distortion correction method of this specification. For example, the processing unit 610 can execute the following steps: Figure 1 Follow the steps shown in .

[0115] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202 , and may further include a read-only storage unit (ROM) 6203 .

[0116] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a grid environment.

[0117] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0118] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more grids (e.g., a local area network (LAN), a wide area network (WAN), and / or a public grid, such as the Internet) through a grid adapter 660. The grid adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Fig. 9 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0119] In an embodiment of the present invention, a computer-readable storage medium is further provided for storing a program, and when the program is executed, the steps of a lens distortion correction method are implemented. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary implementations of the present invention described in the above lens distortion correction method section of this specification.

[0120] As shown above, this embodiment can effectively correct the image distortion caused by the free-form surface lens, and further reduce the non-radial distortion caused by the free-form surface lens on the basis of existing calibration. It is simple and easy to operate and has been proven to be effective.

[0121] Fig.10 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Fig.10As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0122] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0123] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0124] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of grid, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0125] This embodiment can effectively correct the image distortion caused by the free-form surface lens, and further reduce the non-radial distortion caused by the free-form surface lens on the basis of existing calibration. It is simple and easy to operate and has been proven to be effective.

[0126] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0127] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A lens distortion correction method, characterized in that: include: Step S1: Calibrate the intrinsic parameters and extrinsic parameters of the camera according to the calibration plate image, and obtain the extrinsic parameters of the calibration plate; Step S2: using moving least squares surface fitting to calculate the functional relationship between the image coordinates and the ideal image coordinates; Step S3: According to the functional relationship, the positions of all pixels in the ideal image corresponding to the original image are calculated to obtain a distortion correction matrix.

2. The lens distortion correction method according to claim 1, characterized in that: Step S2 includes: Step S21: Calculating the ideal point of the corner point according to the intrinsic parameters of the camera, the extrinsic parameters of the camera and the extrinsic parameters of the calibration plate; Step S22: constructing a fitting function using a moving least squares surface fitting method; Step S23: Determine the parameters of the fitting function according to the ideal point and coordinates of the corner point.

3. The lens distortion correction method according to claim 2, characterized in that: In step S22, the basis function is a quadratic basis function p(x)=(1,x,y,x 2 ,xy,y 2 ) T ; The weight function is a cubic spline function 4. The lens distortion correction method according to claim 2, characterized in that: In step S22, the size of the support region is determined by the curved surface design of the lens.

5. The lens distortion correction method according to claim 4, characterized in that: The greater the slope of the curved surface, the smaller the support region.

6. The lens distortion correction method according to claim 4, characterized in that: The area of ​​each of the support regions on the curved surface is equal.

7. The lens distortion correction method according to claim 1, characterized in that: Step S3 includes: Step S31: Calculate the pixel points according to the functional relationship to establish a pixel-by-pixel correspondence between the ideal image and the original image; Step S32: According to the corresponding relationship, a bilinear interpolation method is used to establish a corresponding function between the ideal image and the original image; Step S33: generating a distortion correction matrix according to the corresponding function.

8. A lens distortion correction system, used to implement the lens distortion correction method according to any one of claims 1 to 7, characterized in that: include: An initial calibration module, used to calibrate the intrinsic parameters and extrinsic parameters of the camera according to the calibration plate image, and obtain the extrinsic parameters of the calibration plate; A fitting module, used to calculate the functional relationship between the image coordinates and the ideal image coordinates using a moving least squares surface fitting according to the points on the calibration plate; The distortion module is used to calculate the positions of all pixels in the ideal image corresponding to the original image according to the functional relationship to obtain a distortion correction matrix.

9. A lens distortion correction device, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to perform the steps of the lens distortion correction method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed, the steps of the lens distortion correction method described in any one of claims 1 to 7 are implemented.

Images