A method for correcting radial and tangential distortion in infrared images

Abstract

This invention provides a method for correcting radial and tangential distortion in infrared images, belonging to the field of image distortion correction technology. The method includes the following steps: selecting key feature points on the infrared image and clarifying their relative positional relationships; selecting a distortion model; optimizing and calibrating the distortion coefficients based on a system of equations and an inequality-constrained objective optimization strategy; mapping from all pixels in the distorted image to the distortion-free image based on the optimized distortion coefficient values; filling empty points in the neighborhood of empty points in the distortion-free image with non-empty pixels; and obtaining the final corrected image. By calibrating the unknown coefficients in the distortion model based on key feature points in a single infrared image, the method eliminates the need for a calibration board or moving imaging equipment, expanding its applicability and enabling high-precision correction of large-distortion, low-resolution infrared images. Furthermore, the novel empty-point filling approach avoids complex coordinate mapping, resulting in more efficient and higher-quality restoration of empty pixel values.

Description

Technical Field

[0001] This invention relates to the field of image distortion correction technology, specifically to a method for correcting radial and tangential distortion in infrared images. Background Technology

[0002] Infrared images are widely used in target detection tasks such as identifying abnormal working conditions or detecting product defects. However, deviations in the manufacturing and installation of optical lenses can lead to radial and tangential distortion in infrared images. These distortions severely interfere with preprocessing steps such as image registration and region cropping, significantly limiting the accuracy of image tag creation and severely impacting target detection accuracy. Therefore, radial and tangential distortion correction of infrared images is of great importance for various target detection tasks.

[0003] Currently, the main methods for image distortion correction include Zhang Zhengyou calibration, direct nonlinear transformation, and self-learning methods. However, existing methods are mostly based on visible light images, which presents several problems when applied to infrared images: Since ordinary calibration boards cannot effectively image infrared images, specially made calibration boards are required for infrared image distortion correction. However, for infrared images with excessively large or small fields of view in some application scenarios, the calibration board's accuracy is limited, and its manufacturing cost is high. Zhang Zhengyou calibration and self-learning methods require mobile devices to acquire multiple images, which is difficult to implement for infrared monitoring equipment already fixed in complex environments, especially in complex industrial production sites where it is difficult to move the infrared monitoring equipment without interfering with production. Infrared imaging reflects the infrared thermal radiation of an object's surface but not three-dimensional spatial information, making direct linear transformation difficult to implement. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for correcting radial and tangential distortion in infrared images, which solves the problem that traditional Zhang Zhengyou calibration methods and self-learning methods require mobile devices to acquire multiple images.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for correcting radial and tangential distortion in infrared images, comprising the following steps:

[0006] S1. Select key feature points from the infrared image and clarify the relative positional relationship between the key feature points.

[0007] Among them, key feature points refer to points in infrared images whose relative positional relationship is clearly known or measurable.

[0008] S2. Based on the relative positional relationship between the key feature points in S1, construct a system of equations based on the polynomial distortion (correction) model describing radial and tangential distortion, and solve for the unknown distortion coefficients of the distortion model through the system of equations.

[0009] Among them, the polynomial distortion (correction) model describing radial and tangential distortion should be easily substituted into the distortion coordinates (u). d v d ) Calculate the corresponding distortion-free coordinates (u) u v u The model (in pixel coordinates) is shown below:

[0010]

[0011] In the formula, k i (i = 1, 2, ...) are the radial distortion coefficients, p i (i = 1, 2, ...) represents the tangential distortion coefficients, and the corresponding terms are the radial and tangential distortion terms; (u c v c The coordinates are the center coordinates of the image. The pixel coordinate system has the top left corner of the image as the origin, the positive v-axis points from the origin to the bottom left corner of the image, and the positive u-axis points from the origin to the top right corner of the image.

[0012] Among them, the equations must be screened to ensure that the system of equations has a solution.

[0013] S3. Establish the distortion coefficient and expected pixel distance variables as optimization variables. Based on an inequality constraint objective optimization strategy with two-layer decision-making, optimize the distortion coefficient by using the expected pixel distance value and the distortion coefficient value obtained by solving the system of equations.

[0014] S3-1. Based on the relative positional relationship of key feature points, construct the inequality constraints and objective function for the inequality-constrained objective optimization problem;

[0015] S3-2. Starting from the initial values ​​of the optimization variables obtained, optimize the distortion coefficient based on the nonlinear inequality constraint objective optimization method.

[0016] Optionally, the calculation method for the nonlinear inequality constraint objective optimization in step S3-2 includes, but is not limited to, the Lagrange-Newton method.

[0017] S3-3. Make a second judgment on the optimization results. If the objective function value does not reach the expected value at the optimization endpoint, give the current distortion coefficient value a small perturbation and repeat S3-2. If the objective function value reaches the expected value, take the current optimized value of the distortion coefficient as the distortion coefficient value used for final correction.

[0018] The distortion coefficients are perturbed as follows:

[0019] X0(ep)=δX(ep).

[0020] In the formula, δ is the perturbation coefficient, ep is the target optimization generation, X0(ep) is the initial value of the distortion coefficient in the ep-th generation optimization, and X(ep) is the optimized distortion coefficient value obtained in the ep-1-th generation optimization. Optionally, δ = 0.99.

[0021] S4. Based on the final distortion coefficient values ​​and the corresponding distortion model, correct the infrared image and calculate the mapping from all original distorted coordinates to distortion-free coordinates. Non-integer coordinates are rounded down and assigned to the corresponding distortion-free pixels to obtain a distortion-free infrared image that retains all the original distorted infrared image pixels.

[0022] It should be understood that during the reverse correction of image distortion, the stretching of the infrared image causes empty spots to appear in the distortion-free image. Due to the change in image size, the number of pixels in the original image is limited, and the larger the distortion of the image area, the more severe the stretching during correction, and the denser the distribution of empty spots.

[0023] S5. Based directly on the distortion-free image, fill in the empty points using the mean or weighted mean of the non-empty point values ​​in the neighborhood of the empty point. For empty points with completely empty neighborhoods, repeat the filling operation until the empty point is filled.

[0024] S5-1. Identify empty and non-empty points in a distortion-free image;

[0025] S5-2. Calculate the pixel value of the empty point based on the non-empty points in the neighborhood of the empty point according to specific rules, including but not limited to averaging or weighted averaging the non-empty points in the neighborhood.

[0026] Further optimization suggests that the selected neighborhood of an empty point should be an 8-neighborhood, but is not limited to 4-, 8-, or 16-neighborhoods.

[0027] S5-3. For empty points where all neighboring pixels are empty, repeat S5-1 to S5-2 until all target empty points are filled.

[0028] The target empty point refers to the empty point that is expected to be filled.

[0029] Further preferably, the relative positional relationship in step S1 includes the linear relationship between feature points and the expected pixel distance on the distortion-free image or in the real world. The expected pixel distance is calculated based on imaging device attributes such as focal length and pixel size, imaging distance, and the actual measured distance between feature points. The expected pixel distance calculation can be, but is not limited to, the following:

[0030]

[0031] In the formula, l is the measured distance of the feature point in the world coordinate system, H is the imaging height, f is the focal length, s is the pixel size, and d is the pixel distance of the feature point in the pixel coordinate system, i.e., the expected pixel distance.

[0032] Further optimization, in step S2, the equations constructed based on relative positional relationships to describe the linear relationships between feature points and the expected pixel distances must ensure that the constructed system of equations includes equations describing the expected pixel distance relationships, but should describe as many linear relationships as possible. The relationship description equations are shown below:

[0033]

[0034] Where, v u1 and v u2 Let u be the ordinate of two feature points on a distortion-free image that are expected to have the same v-axis coordinate. u1 and u u2 Let v be the x-coordinate of two feature points on a distortion-free image that are expected to have the same u-axis coordinate. u3 and v u4 The desired pixel distance along the v-axis in a distortion-free image is

[0035] d v The ordinates of the two feature points, u u3 and u u4 The desired pixel distance along the u-axis in a distortion-free image is d. u The x-coordinates of the two feature points

[0036] In step S3-1, the inequality construction is based on the construction of a system of equations. The construction pattern of the inequality constraints is shown below:

[0037]

[0038] In the formula, o i (i = 1, 2, ...) represents the allowable deviation threshold of the equation relationship set manually; η1 and η2 are the lower and upper bound range coefficients of the expected pixel distance, respectively; d1 and d2 are the expected pixel distance variables, with initial values ​​of d1 and d2, respectively. v and d u Other variables are defined as before.

[0039] Further optimization, η1∈(0.5,1), in,

[0040] P max d0 is the maximum value in the image size P1×P2, and d0 is the initial value of the expected pixel distance variable in the inequality.

[0041] Further optimization, the objective function O(X) in step S3-1 reflects the deviation between the linear relationship between the corrected feature points and the expected pixel distance, and is constructed as follows:

[0042]

[0043] In the formula, w i (i = 1, 2, 3, 4) are weighting coefficients; v u1，2 (i)(i=1,2,...,n) are the ordinate values ​​of the feature points on the same straight line perpendicular to the v axis in the distortion-free image;

[0044] u u1，2 (i) (i = 1, 2, ..., n) represents the x-coordinate of the feature points on the same straight line perpendicular to the u-axis in the distortion-free image; X refers to the optimization variable; n in each sub-formula is an independent variable, representing the number of feature point pairs that conform to the current relative position relationship.

[0045] This invention provides a method for correcting radial and tangential distortion in infrared images. It offers the following advantages:

[0046] 1. This invention can correct radial and tangential distortions in infrared images using only a single image. The method is flexible in application scenarios, highly feasible, and can achieve high-precision correction of large-distortion, low-resolution infrared images.

[0047] 2. This invention innovatively proposes a correction method that solves for distortion-free coordinates from distorted coordinates, and then performs gap filling solely based on the distortion-free image. By avoiding the complex nonlinear mapping calculation from all distortion-free coordinates to distorted coordinates, this method effectively improves the overall computational efficiency of a single distortion correction operation.

[0048] 3. This invention fills in empty points by using non-empty points within the neighborhood of empty points in a distortion-free image. Since the distance between distortion-free pixels better reflects the true distance compared to the distance between distorted pixels, pixels closer to the actual distance of the empty point contribute more to the empty point filling. Both theory and experiments have confirmed that, compared to the commonly used nearest neighbor interpolation method, this invention achieves higher quality restoration of empty point pixels. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the process of the present invention;

[0050] Figure 2 This is a schematic diagram of the process of filling empty points based on non-empty neighborhoods according to the present invention;

[0051] Figure 3 The results of radial and tangential distortion correction of the infrared image of the electrolytic cell of the present invention are as follows: Figure 3 -(a) is a schematic diagram of the original infrared image and key feature points (marked with "+" and "*"); 3-(b) is an infrared image after radial and tangential distortion correction (before filling in the gaps);

[0052] Figure 4 The image shown is an infrared image of an electrolytic cell after filling empty points with non-empty points in the neighborhood of empty points, as per the present invention. Figure 4 -(a) is the infrared image after the first filling. There are still some empty spots in the densely populated edge areas. 4-(b) is the infrared image of the electrolytic cell after the second filling. The edge empty spots have also been eliminated. Detailed Implementation

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] Example 1:

[0055] like Figure 1-2 As shown, this embodiment of the invention provides a method for correcting radial and tangential distortion in infrared images, comprising the following steps:

[0056] S1. Select key feature points in the infrared image, clarify the relative positional relationship between the key feature points, and solve for the expected pixel distance between some feature points;

[0057] S2. Based on the relative positional relationship between the key feature points in S1, construct a system of equations based on the polynomial distortion correction model describing radial and tangential distortion, and solve for the unknown distortion coefficients of the distortion model through the system of equations.

[0058] S3. Establish the distortion coefficient and expected pixel distance variables as optimization variables, and use the expected pixel distance value and the distortion coefficient value obtained by solving the system of equations as initial values. Based on an inequality constraint objective optimization strategy with two-layer decision, optimize the solution of the distortion coefficient.

[0059] S3-1. Based on the relative positional relationship of key feature points, construct the inequality constraints and objective function for the inequality-constrained objective optimization problem;

[0060] S3-2. Starting from the initial values ​​of the optimization variables obtained, optimize the distortion coefficient based on the nonlinear inequality constraint objective optimization method.

[0061] S3-3. Make a second judgment on the optimization results. If the objective function value does not reach the expected value at the optimization endpoint, give the current distortion coefficient value a small perturbation and repeat S3-2. If the objective function value reaches the expected value, take the current optimized value of the distortion coefficient as the distortion coefficient value used for final correction.

[0062] S4. Based on the final distortion coefficient values ​​and the corresponding distortion model, correct the infrared image and calculate the mapping from all original distorted coordinates to distortion-free coordinates. Non-integer coordinates are rounded down and assigned to the corresponding distortion-free pixels to obtain a distortion-free infrared image that retains all the original distorted infrared image pixels.

[0063] S5. Based directly on the distortion-free image, fill in the empty points using the mean or weighted mean of the non-empty point values ​​in the neighborhood of the empty point. For empty points with completely empty neighborhoods, repeat the filling operation until the empty point is filled.

[0064] S5-1. Identify empty and non-empty points in a distortion-free image;

[0065] S5-2. Calculate the pixel value of the empty point based on the non-empty points in the neighborhood of the empty point according to specific rules, including but not limited to averaging or weighted averaging the non-empty points in the neighborhood.

[0066] Among them, the recommended selection of the neighborhood of the empty point is an 8-neighborhood, but it is not limited to 4-, 8-, 16-neighborhoods, etc.

[0067] S5-3. For empty points where all neighboring pixels are empty, repeat S5-1 to S5-2 until all target empty points are filled, where target empty points refer to the empty points that are expected to be filled.

[0068] The key feature points in step S1 refer to points in the infrared image whose relative positional relationship is clearly known or measurable.

[0069] The expected pixel distance calculation in step S1 is based on the imaging focal length, pixel size, height, and actual measured distance.

[0070] The distortion model in step S2 is a polynomial distortion model, and the model formula in the pixel coordinate system is shown below:

[0071]

[0072] In the formula, k i (i = 1, 2, ...) are the radial distortion coefficients, p i (i = 1, 2, ...) represents the tangential distortion coefficients, and the corresponding terms are the radial and tangential distortion terms; (u c v c The coordinates are the center coordinates of the image. The pixel coordinate system has the top left corner of the image as the origin, the positive v-axis points from the origin to the bottom left corner of the image, and the positive u-axis points from the origin to the top right corner of the image.

[0073] The constraint objective optimization strategy in step S3 has dual determination, and the objective optimization process can be repeated.

[0074] Before each optimization loop, a perturbation is applied to the new optimization value as the initial value for the current optimization. The perturbation formula is shown below:

[0075] X0(ep)=δX(ep).

[0076] In the formula, δ is the disturbance coefficient, with a value range of δ∈(0.7,1); X0 is the initial value of the optimization variable, X is the value of the optimization variable, and ep is the optimization algebra.

[0077] The mapping in step S4 is a mapping from all distorted coordinates to undistorted coordinates. The undistorted image retains all the original distorted pixel information, but is not completely filled.

[0078] The empty point filling in step S5 is performed only based on the undistorted image, and no further operations are performed on the original distorted image.

[0079] Example 2:

[0080] like Figure 3-4 As shown, this invention provides a method for radial and tangential distortion correction of infrared images. Based on key feature points in a single infrared image, it calibrates unknown coefficients in the distortion model, eliminating the need for a calibration plate or moving imaging equipment. This expands the applicability of the method and enables high-precision correction of large-distortion, low-resolution infrared images. Furthermore, this invention uses a method of filling empty points in distortion-free images, avoiding numerous complex coordinate mapping solutions and efficiently and effectively restoring the pixel values ​​of empty points. The embodiments provided by this invention use infrared images of electrolytic cells in non-ferrous metallurgy (such as...) Figure 3 Taking distortion correction as an example (as shown in (a)), the content of the present invention will be further described in detail. The specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention; that is, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The execution process of the method in this embodiment is as follows:

[0081] S1. Select the corner points on the edges of the two electrolytic cells far from the image center in the infrared image as the key feature points for steps two and three, such as... Figure 3 As indicated by the "+" sign in -(a), select several points on the leftmost edge of the groove to supplement the key feature points in step three, such as... Figure 3 -(a) is indicated by the "*" symbol. Calculate the expected pixel distance d between adjacent and relative slot corner points. u and d v As shown below:

[0082]

[0083] In the formula, substituting the focal length f = 7.5 mm, pixel size s = 17 μm, forming height H ≈ 8,100 mm, and actual distance l between adjacent and relative corner points, we find the values ​​for the focal length f = 7.5 mm, pixel size s = 17 μm, forming height H ≈ 8,100 mm, and actual distance l between adjacent and relative corner points.u ≈488.02 pixels and l v ≈4,000mm. Therefore, d u ≈57.19 pixels and d v ≈488.01 pixels.

[0084] S2. Select radial and tangential distortion models, construct a system of equations based on the distortion models and the relative positions of feature points, and solve for the distortion coefficients as initial values ​​for optimization.

[0085] The radial and tangential distortion models are shown below:

[0086]

[0087] The distortion model is simplified as follows:

[0088]

[0089] In the formula, f(P|(u) d v d ), (u c v c The equation describes the relationship between undistorted coordinates and distortion coefficients, where P refers to all unknown distortion coefficients to be calibrated (k). 1～4 and p 1～4 For any distorted point in any infrared image, its coordinates (u) d v d ) and image center coordinates (u c v c ) is a known quantity.

[0090] In step two, for the upper edge slot corner point (u) among the key feature points u (i), v u (i) (i = 1, 2, ..., n1) and the lower edge slot corner point (u u (i), v u (i))(i=n1+1,n1+2,...,2n1), the relative positional relationships of the feature points on which the equation is constructed include:

[0091] Points at the same edge slot corner have the same v-coordinate, and the equation is constructed as follows:

[0092] v u (i)=v u (j), (i≠j; i, j=1, 2, ..., n1 or n1+1, n1+2, ..., 2n1).

[0093] Relative slot corner points have the same u-coordinate, and the equation is constructed as follows:

[0094] uu (i)=u u (i+n1), (i=1, 2,..., n1).

[0095] The expected pixel distance between adjacent slot corner points is d u The equation is constructed as follows:

[0096] u u (i+1)=u u (i)+d u , (i=1, 2,..., n1-1 or n1+1, n1+2,..., 2n1-1).

[0097] The expected pixel distance between relative slot corner points is d v The equation is constructed as follows:

[0098] v u (i+n1)=v u (j)+d v , (i = 1, 2, ..., n1).

[0099] The distortion coefficients are solved using eight nonlinear correlation equations from the above equations. In this embodiment, the following values ​​are obtained: k1 = 1.6721e-06, k2 = 5.3061e-11, k3 = -5.0835e-16, k4 = 2.1574e-21, p1 = 3.7630e-06, p2 = 2.3795e-05, p3 = -1.3128e+04, p4 = 7.4121e-10.

[0100] Thus, the initial values ​​of the optimization variables, including the distortion coefficients and the expected imaging distance between feature points, are obtained: X0(0) = {P0, d...} u d v}, where P0 is the set of distortion coefficient values ​​calculated based on the system of equations. It should be understood that, due to the bias in the selection of feature points, X0(0) is a rough estimate of the values ​​of each variable and cannot accurately describe the ideal correction result.

[0101] S3. Optimize the optimization variables based on an inequality constraint optimization strategy with two-level decision-making. S3-1. Optimize the upper edge slot corner points (u) among the key feature points. u (i), v u (i) (i = 1, 2, ..., n1), lower edge slot corner point (u u (i), v u (i))(i=n1+1,n1+2,...,2n1) and the feature point ((u) of the leftmost groove edge u (i), v uThe relative positions of (i) (i = 2n1+1, 2n1+2, ..., n2) are used to construct inequality constraints, as shown below:

[0102]

[0103] In the formula, o i (i = 1, 2, 3, 4, 5) are the set deviation thresholds, which are set to 5, 5, 5, 5, 3 respectively in this embodiment; the lower and upper bound range coefficients η1 and η1 are set to 0.5 and 1.1 respectively.

[0104] The objective function O(X) for measuring the deviation of the expected relative positional relationship is constructed as follows:

[0105]

[0106] In the formula, X is the value of the optimization variable, and w i (i = 1, 2, 3, 4) are the weighting coefficients for each item.

[0107] It should be understood that O1(X) calculates the linear deviation of the feature points located on the edges of the two horizontal straight lines, O2(X) calculates the linear deviation of the feature points on the leftmost vertical straight edge of the electrolytic cell, and O3(X) and O4(X) calculate the deviations of the distance between adjacent cell corner points and the distance between the relative cell corner points of the two columns at the outer end of the electrolytic cell, respectively.

[0108] S3-2: The nonlinear inequality-constrained objective optimization problem is calculated based on the Lagrange-Newton method. The objective function is minimized to obtain the optimization variable value X(eq). Here, eq is the optimization algebra, and one complete optimization process is one generation.

[0109] S3-3: The upper limit of the expected deviation O e =5pix, if O(X(eq))>O e Give the optimization variable value X(eq) a perturbation to obtain a new initial optimization value, and repeat the objective optimization step S3-2 based on the new initial value; if O(X(eq)) < O e The optimization process is then complete, and the final distortion coefficient values ​​are obtained. The perturbation formula for the optimization variables is shown below:

[0110] X0(ep)=δX(ep).

[0111] In the formula, δ is the disturbance coefficient. In this embodiment, δ = 0.99.

[0112] In this embodiment, after one optimization of the target, the optimized distortion coefficients are: k1 = 1.4464e-06, k2 = 5.0681e-11, k3 = -4.5343e-16, k4 = 1.8860e-21, p1 = 1.4464e-06, p2 = -1.0384e-05, p3 = -1.1342e-04, p4 = 6.3948e-10, d u =56.0510, d v =478.2053.

[0113] S4. Infrared image correction based on optimized distortion coefficients and corresponding distortion models.

[0114] Specifically, based on the optimized distortion coefficient values ​​and the corresponding distortion model, the distortion-free pixel coordinate mappings of all distorted pixel coordinates are calculated, resulting in a distortion-free image that retains all distorted pixel information. The results are as follows: Figure 3 As shown in (b), the mapping relationship is solved as follows:

[0115] (u u (i), v u (i))=Round(f u (P,(u d v d ), (u c v c )), f v (P,(u d v d ), (u c v c ))).

[0116] In the formula, Round(·) is the rounding function.

[0117] It should be understood that step four only considers the mapping of all original distorted coordinates to coordinates in the undistorted image; the mapping does not include all pixels in the undistorted image. During distortion correction, the image undergoes deformation and scaling, resulting in empty pixels in the corrected image.

[0118] S5. Fill in the empty points based on the non-empty points in the neighborhood of the empty points on the distortion-free image.

[0119] S5-1. By mapping, all non-empty points and empty points on the distortion-free image are simply identified, and the coordinate set of all empty points is obtained.

[0120] A={p(u i v j )=NaN, i∈[1, P1], j∈[1, P2]|(u i v j )},(i,j∈R), where

[0121] p(u i v j ) Identify coordinates (u i v j The pixel value of ).

[0122] S5-2. For each empty point, sequentially search for non-empty points within its 8-neighborhood, calculate the average of the non-empty points, and use this average to fill the target empty point. The result is as follows: Figure 4 As shown in (a). Below:

[0123]

[0124] Among them, Mean(B ij ) represents the set B ij Find the mean of the elements in B. ij Empty point (u) i v j The set of non-empty point values ​​in the 8-neighborhood of ).

[0125] S5-3. For empty points within an 8-neighborhood that are all empty points, repeat the above steps once to fill all empty points and obtain the final corrected image, as shown in the figure. Figure 4 As shown in -(b).

[0126] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for correcting radial and tangential distortion in infrared images, characterized in that: Includes the following steps: S1. Select key feature points in the infrared image, clarify the relative positional relationship between the key feature points, and solve for the expected pixel distance between some feature points; S2. Based on the relative positional relationship between the key feature points in S1, construct a system of equations based on the polynomial distortion correction model describing radial and tangential distortion, and solve for the unknown distortion coefficients of the distortion model through the system of equations. S3. Establish the distortion coefficient and expected pixel distance variables as optimization variables, and use the expected pixel distance value and the distortion coefficient value obtained by solving the system of equations as initial values. Based on an inequality constraint objective optimization strategy with two-layer decision, optimize the solution of the distortion coefficient. S3-1. Based on the relative positional relationship of key feature points, construct the inequality constraints and objective function for the inequality-constrained objective optimization problem; S3-2. Starting from the initial values ​​of the optimization variables obtained, optimize the distortion coefficient based on the nonlinear inequality constraint objective optimization method. S3-3. Make a second judgment on the optimization results. If the objective function value does not reach the expected value at the optimization endpoint, give the current distortion coefficient value a small perturbation and repeat S3-2. If the objective function value reaches the expected value, take the current optimized value of the distortion coefficient as the distortion coefficient value used for final correction. S4. Based on the final distortion coefficient values ​​and the corresponding distortion model, correct the infrared image and calculate the mapping from all original distorted coordinates to distortion-free coordinates. Non-integer coordinates are rounded down and assigned to the corresponding distortion-free pixels to obtain a distortion-free infrared image that retains all the original distorted infrared image pixels. S5. Based directly on the distortion-free image, fill in the empty points using the mean or weighted mean of the non-empty point values ​​in the neighborhood of the empty point; for empty points with completely empty neighborhoods, perform the filling operation repeatedly until the empty point is filled. S5-1. Identify empty and non-empty points in a distortion-free image; S5-2. Calculate the pixel value of the empty point based on the non-empty points in the neighborhood of the empty point according to specific rules, including but not limited to averaging or weighted averaging the non-empty points in the neighborhood. Among them, the recommended selection of the neighborhood of the empty point is an 8-neighborhood, but it is not limited to 4-, 8-, or 16-neighborhoods; S5-3. For empty points where all neighboring pixels are empty, repeat S5-1 to S5-2 until all target empty points are filled, where target empty points refer to the empty points that are expected to be filled.

2. The method for correcting radial and tangential distortion of infrared images according to claim 1, characterized in that: The key feature points in step S1 refer to points in the infrared image whose relative positional relationship is clearly known or measurable.

3. The method for correcting radial and tangential distortion of infrared images according to claim 1, characterized in that: The expected pixel distance calculation in step S1 is based on the imaging focal length, pixel size, height, and actual measured distance.

4. The method for correcting radial and tangential distortion of infrared images according to claim 1, characterized in that: The distortion model in step S2 is a polynomial distortion model, and the model formula in the pixel coordinate system is as follows: In the formula, k i (i = 1, 2, ...) are the radial distortion coefficients, p i (i = 1, 2, ...) represents the tangential distortion coefficients, and the corresponding terms are the radial and tangential distortion terms; (u c v c ) represents the coordinates of the image center; the pixel coordinate system has the upper left corner of the image as the origin, the positive v-axis points from the origin to the lower left corner of the image, and the positive u-axis points from the origin to the upper right corner of the image.

5. The method for correcting radial and tangential distortion of infrared images according to claim 1, characterized in that: The constraint target optimization strategy in step S3 has dual determination, and the target optimization process can be repeated cyclically.

6. The method for correcting radial and tangential distortion of infrared images according to claim 5, characterized in that: Before each optimization cycle, a perturbation is applied to the new optimization value as the initial value for the current optimization. The perturbation formula is shown below: X0(ep)=δX(ep). In the formula, δ is the disturbance coefficient, with a value range of δ∈(0.7,1); X0 is the initial value of the optimization variable, X is the value of the optimization variable, and ep is the optimization algebra.

7. The method for correcting radial and tangential distortion of infrared images according to claim 1, characterized in that: The mapping in step S4 is a mapping from all distorted coordinates to undistorted coordinates. The undistorted image retains all the original distorted pixel information, but is not completely filled.

8. The method for correcting radial and tangential distortion of infrared images according to claim 1, characterized in that: The empty point filling in step S5 is performed only based on the undistorted image, and no further operations are performed on the original distorted image.

Images