Image correction method and system based on nonlinear distortion distribution elliptical fisheye lens, and medium

Abstract

The invention provides an image correction method and system based on a nonlinear distortion distribution elliptical fisheye lens, and a medium. The method comprises the following steps: establishing a nonlinear mixed distortion imaging model based on a mapping relation between image coordinates and a spherical direction; defining a distortion vector, and performing fitting processing on the distortion vector based on a two-dimensional B-spline function to obtain a distortion function; the method comprises the following steps: acquiring images in a five-surface three-dimensional angle calibration space based on an elliptical fisheye lens, analyzing lens imaging parameters, and generating parameter values of a distortion function; analyzing parameter values of the distortion function based on a nonlinear mixed distortion imaging model to generate calibration parameters, and remapping pixel points in the distorted image based on the calibration parameters; performing image reconstruction on the remapped distorted image to obtain a corrected image; improving the sampling density of a target area by constructing a nonlinear mixed distortion imaging model; the system adapts to complex monitoring environments such as galleries and entrances and exits; and spatial distortion is flexibly expressed through a B spline, so that the modeling capability is improved.

Description

Technical Field

[0001] This application relates to the field of image correction technology, and more specifically, to an image correction method, system, and medium based on a nonlinear distortion distribution elliptical fisheye lens. Background Technology

[0002] Fisheye lenses are widely used in panoramic video surveillance, traffic management, security patrol, and robot navigation. Traditional fisheye lenses primarily employ angularly symmetric optical systems with a circular imaging plane, and their distortion functions are based on fixed projection models such as equidistant, equisolid, or orthographic projections. These models generally suffer from the following problems:

[0003] Angular symmetry limits scene flexibility: distortion is uniformly distributed in all directions, making it difficult to specifically enhance certain directions (such as entrance passages);

[0004] Low imaging accuracy in critical areas: Because pixels are evenly distributed across the field of view, critical areas cannot obtain higher sampling density;

[0005] The imaging model is rigid: it cannot express dissimilar transformations, resulting in inaccurate image correction.

[0006] Traditional calibration methods are insufficient to handle complex asymmetric distortions. Summary of the Invention

[0007] The purpose of this application is to provide an image correction method, system, and medium based on a nonlinear distortion distribution elliptical fisheye lens. This method improves the sampling density of the target area by constructing a nonlinear hybrid distortion imaging model; it adapts to complex monitoring environments, such as corridors and entrances / exits; and it flexibly expresses spatial distortion through B-splines, thereby improving modeling capabilities.

[0008] This application also provides an image correction method based on a nonlinear distortion distribution elliptical fisheye lens, including:

[0009] A nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation;

[0010] Define a distortion vector, and fit the distortion vector to a two-dimensional B-spline function to obtain the distortion function;

[0011] Based on the acquisition of images in a five-dimensional angular calibration space using an elliptical fisheye lens, the lens imaging parameters are analyzed to generate the parameter values ​​of the distortion function.

[0012] Based on the nonlinear hybrid distortion imaging model, the parameter values ​​of the distortion function are analyzed to generate calibration parameters, and the pixels in the distorted image are remapped based on the calibration parameters.

[0013] The remapped distorted image is then reconstructed to obtain the corrected image.

[0014] Optionally, in the image correction method based on an elliptical fisheye lens with nonlinear distortion distribution described in the embodiments of this application, a nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation, specifically including:

[0015] Define the mapping relationship between the spherical orientation of an elliptical fisheye lens and the ideal image coordinates, and generate a mapping dataset based on the mapping relationship between the spherical orientation and the ideal image coordinates;

[0016] Construct a nonlinear hybrid distortion function, which includes a radial distortion function and a tangential distortion function;

[0017] Select an initial model framework, iteratively train the initial model framework based on the mapping dataset, and generate radial and tangential distortion values;

[0018] The distortion difference is obtained by comparing the radial distortion threshold and the tangential distortion threshold with the radial distortion value and the tangential distortion value, respectively.

[0019] The model parameters are dynamically optimized based on the distortion difference until the model converges, resulting in a nonlinear hybrid distortion imaging model.

[0020] Optionally, in the image correction method based on an elliptical fisheye lens with nonlinear distortion distribution described in this application embodiment, a distortion vector is defined, and the distortion vector is fitted based on a two-dimensional B-spline function to obtain a distortion function, specifically including:

[0021] The image plane is divided into a regular grid, with each grid node corresponding to a sample point, resulting in a number of sample points;

[0022] Obtain the ideal and actual coordinates of each sample point;

[0023] The actual coordinates of the sample points are used as input parameters for the two-dimensional B-spline function. The deviation between the actual coordinates and the ideal undistorted coordinates is calculated to generate a distortion vector.

[0024] The distortion vector is decomposed into radial distortion components and tangential distortion components, and radial distortion component functions and tangential distortion component functions are constructed.

[0025] The distortion function is obtained by fitting the radial distortion component function and the tangential distortion component function.

[0026] Optionally, in the image correction method based on an elliptical fisheye lens with nonlinear distortion distribution described in this application embodiment, the method involves acquiring images within a five-dimensional angular calibration space using the elliptical fisheye lens, analyzing lens imaging parameters, and generating parameter values ​​for the distortion function. Specifically, this includes:

[0027] Set the elliptical fisheye lens at the center of the five-dimensional angle calibration space and keep the elliptical fisheye lens horizontal;

[0028] Acquire images from several different angles and simultaneously record the pose information of the elliptical fisheye lens;

[0029] Extract image feature points, analyze the corner points of the image feature points, and perform position matching of corner points in images from different angles to obtain matching information;

[0030] Lens imaging parameters are obtained by analyzing matching information based on a nonlinear hybrid distortion imaging model.

[0031] The distortion function is input based on the lens imaging parameters, and the parameter values ​​of the distortion function are generated.

[0032] Optionally, in the image correction method based on a nonlinear distortion distribution elliptical fisheye lens described in this application embodiment, calibration parameters are generated by analyzing the parameter values ​​of the distortion function based on a nonlinear hybrid distortion imaging model, and the pixels in the distorted image are remapped based on the calibration parameters, specifically including:

[0033] The parameter values ​​of the distortion function are analyzed based on the nonlinear hybrid distortion imaging model, and the radial distortion parameter values ​​and tangential distortion parameter values ​​are analyzed based on the parameter values ​​of the distortion function.

[0034] The radial offset of a pixel is obtained by comparing the radial distortion parameter value with the standard radial distortion parameter value.

[0035] Generate radial calibration parameters based on the radial offset of each pixel;

[0036] The tangential distortion parameter value is compared with the standard tangential distortion parameter value to obtain the tangential offset of the pixel.

[0037] Tangential calibration parameters are generated based on the tangential offset of each pixel.

[0038] The final calibration parameters are obtained by weighted fusion of radial and tangential calibration parameters.

[0039] The pixels in the distorted image are remapped based on the final calibration parameters.

[0040] Optionally, in the image correction method based on a nonlinear distortion distribution elliptical fisheye lens described in this application embodiment, the remapped distorted image is reconstructed to obtain a corrected image, specifically including:

[0041] Obtain the image coordinates of the remapped distorted image and convert the image coordinates into normalized camera coordinates;

[0042] Convert the normalized camera coordinates to the original distorted image coordinates;

[0043] The corresponding position of the normalized camera coordinates in the original distorted image is found based on the inverse operation of the distortion function;

[0044] The normalized camera coordinates at the same location are compared with the original distorted coordinates to obtain the coordinate difference.

[0045] The corrected image is obtained by filling in the pixel values ​​of the distorted image based on the coordinate difference.

[0046] Secondly, embodiments of this application provide an image correction system based on a nonlinear distortion distribution elliptical fisheye lens. The system includes a memory and a processor. The memory includes a program for an image correction method based on a nonlinear distortion distribution elliptical fisheye lens. When the program for the image correction method based on a nonlinear distortion distribution elliptical fisheye lens is executed by the processor, it performs the following steps:

[0047] A nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation;

[0048] Define a distortion vector, and fit the distortion vector to a two-dimensional B-spline function to obtain the distortion function;

[0049] Based on the acquisition of images in a five-dimensional angular calibration space using an elliptical fisheye lens, the lens imaging parameters are analyzed to generate the parameter values ​​of the distortion function.

[0050] Based on the nonlinear hybrid distortion imaging model, the parameter values ​​of the distortion function are analyzed to generate calibration parameters, and the pixels in the distorted image are remapped based on the calibration parameters.

[0051] The remapped distorted image is then reconstructed to obtain the corrected image.

[0052] Optionally, in the image correction system based on an elliptical fisheye lens with nonlinear distortion distribution described in this application embodiment, a nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation, specifically including:

[0053] Define the mapping relationship between the spherical orientation of an elliptical fisheye lens and the ideal image coordinates, and generate a mapping dataset based on the mapping relationship between the spherical orientation and the ideal image coordinates;

[0054] Construct a nonlinear hybrid distortion function, which includes a radial distortion function and a tangential distortion function;

[0055] Select an initial model framework, iteratively train the initial model framework based on the mapping dataset, and generate radial and tangential distortion values;

[0056] The distortion difference is obtained by comparing the radial distortion threshold and the tangential distortion threshold with the radial distortion value and the tangential distortion value, respectively.

[0057] The model parameters are dynamically optimized based on the distortion difference until the model converges, resulting in a nonlinear hybrid distortion imaging model.

[0058] Optionally, in the image correction system based on an elliptical fisheye lens with nonlinear distortion distribution described in this application embodiment, a distortion vector is defined, and the distortion function is obtained by fitting the distortion vector based on a two-dimensional B-spline function, specifically including:

[0059] The image plane is divided into a regular grid, with each grid node corresponding to a sample point, resulting in a number of sample points;

[0060] Obtain the ideal and actual coordinates of each sample point;

[0061] The actual coordinates of the sample points are used as input parameters for the two-dimensional B-spline function. The deviation between the actual coordinates and the ideal undistorted coordinates is calculated to generate a distortion vector.

[0062] The distortion vector is decomposed into radial distortion components and tangential distortion components, and radial distortion component functions and tangential distortion component functions are constructed.

[0063] The distortion function is obtained by fitting the radial distortion component function and the tangential distortion component function.

[0064] Thirdly, embodiments of this application also provide a computer-readable storage medium, which includes an image correction method program based on a nonlinear distortion distribution elliptical fisheye lens. When the image correction method program based on a nonlinear distortion distribution elliptical fisheye lens is executed by a processor, it implements the steps of the image correction method based on a nonlinear distortion distribution elliptical fisheye lens as described in any of the preceding claims.

[0065] As can be seen from the above, the image correction method, system, and medium based on an elliptical fisheye lens with nonlinear distortion distribution provided in this application establish a nonlinear hybrid distortion imaging model based on the mapping relationship between image coordinates and spherical direction; define a distortion vector, and fit the distortion vector based on a two-dimensional B-spline function to obtain a distortion function; acquire images in a five-dimensional angular calibration space based on the elliptical fisheye lens, analyze lens imaging parameters, and generate parameter values ​​for the distortion function; analyze the parameter values ​​of the distortion function based on the nonlinear hybrid distortion imaging model to generate calibration parameters, and remap the pixels in the distorted image based on the calibration parameters; reconstruct the distorted image using the remapped image to obtain a corrected image; improve the sampling density of the target area by constructing a nonlinear hybrid distortion imaging model; adapt to complex monitoring environments, such as corridors and entrances / exits; and flexibly express spatial distortion through B-splines to improve modeling capabilities. Attached Figure Description

[0066] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 A flowchart of an image correction method based on a nonlinear distortion distribution elliptical fisheye lens provided in this application embodiment;

[0068] Figure 2 A flowchart of a nonlinear hybrid distortion imaging model construction method for an image correction method based on a nonlinear distortion distribution elliptical fisheye lens provided in this application embodiment;

[0069] Figure 3 A flowchart illustrating the distortion function acquisition process of an image correction method based on a nonlinear distortion distribution elliptical fisheye lens provided in this application embodiment;

[0070] Figure 4 A ray projection structure diagram of an image correction system based on a nonlinear distortion distribution elliptical fisheye lens provided in an embodiment of this application;

[0071] Figure 5 This is a five-dimensional spatial diagram illustrating the image correction system based on a nonlinear distortion distribution elliptical fisheye lens provided in an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0073] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0074] Please refer to Figure 1 , Figure 1 This is a flowchart of an image correction method based on an elliptical fisheye lens with nonlinear distortion distribution, as described in some embodiments of this application. This image correction method based on an elliptical fisheye lens with nonlinear distortion distribution is used in a terminal device and includes the following steps:

[0075] S101, a nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation;

[0076] S102, Define the distortion vector, and fit the distortion vector based on the two-dimensional B-spline function to obtain the distortion function;

[0077] S103, based on the elliptical fisheye lens, acquires images in a five-dimensional angular calibration space, analyzes lens imaging parameters, and generates parameter values ​​for the distortion function;

[0078] S104, Based on the nonlinear hybrid distortion imaging model, analyze the parameter values ​​of the distortion function to generate calibration parameters, and remap the pixels in the distorted image based on the calibration parameters;

[0079] S105, the remapped distorted image is reconstructed to obtain the corrected image.

[0080] It should be noted that the distortion vector (Δx, Δy) is defined to include both radial and tangential components:

[0081]

[0082] D r (θ,φ): Radial distortion component; represents the amount of distortion offset in the direction (i.e., radial) outward from the center of the image. It is mainly determined by the imaging geometry of the lens (especially the nonlinear stretching or compression caused by non-ideal lenses). It is related to the polar angle θ and azimuth angle φ of the incident light and reflects the changes in the image from the center to the edge due to spherical projection.

[0083] D t (θ,φ): Tangential distortion components, representing tangential offset perpendicular to the radial direction, mainly originating from slight assembly misalignment of lens components, non-coaxiality of lens elements, etc. It is also a function of the polar angle θ and azimuth angle φ, used to describe asymmetric distortion along the tangential direction.

[0084] It employs a non-angularly symmetrical lens structure, with an effective imaging plane that satisfies a two-dimensional elliptical geometric relationship:

[0085]

[0086] in:

[0087] x represents the horizontal coordinate of the image (unit: mm or pixels);

[0088] y represents the vertical coordinate of the image (unit: mm or pixels);

[0089] 'a' represents the length of the semi-major axis of the ellipse in the horizontal direction (unit: mm);

[0090] b represents the length of the semi-minor axis of the ellipse in the vertical direction (unit: mm).

[0091] The lens imaging function is modeled based on the spherical orientation (θ, φ), where:

[0092] θ∈[0,π / 2] represents the polar angle (the angle between the ray and the principal axis of the lens);

[0093] φ∈[0,2π) represents the azimuth angle (the direction of the light ray projected onto the horizontal plane).

[0094] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a nonlinear hybrid distortion imaging model construction method for an image correction method based on an elliptical fisheye lens with nonlinear distortion distribution, as described in some embodiments of this application. According to embodiments of the present invention, a nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation, specifically including:

[0095] S201, Define the mapping relationship between the spherical orientation of the elliptical fisheye lens and the ideal image coordinates, and generate a mapping dataset based on the mapping relationship between the spherical orientation and the ideal image coordinates;

[0096] S202, Construct a nonlinear hybrid distortion function, which includes a radial distortion function and a tangential distortion function;

[0097] S203, Select the initial model framework, iteratively train the initial model framework based on the mapping dataset, and generate radial distortion values ​​and tangential distortion values;

[0098] S204, Based on the radial distortion threshold and the tangential distortion threshold, the radial distortion value and the tangential distortion value are compared respectively to obtain the distortion difference value;

[0099] S205, based on the distortion difference, dynamically optimizes the model parameters until the model converges, thus obtaining a nonlinear hybrid distortion imaging model.

[0100] It should be noted that the mapping relationship between image coordinates (x, y) and spherical directions (θ, φ) is as follows:

[0101]

[0102] in:

[0103] R(θ,φ) represents the nonlinear polar radius function, which describes the distortion caused by the combined action of polar angle and azimuth angle;

[0104] α(φ) represents the azimuth scaling function along the x-direction;

[0105] β(φ) represents the azimuth scaling function along the y-direction;

[0106] cos(φ) and sin(φ) represent the azimuth components.

[0107] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating the distortion function acquisition process of an image correction method based on an elliptical fisheye lens with a nonlinear distortion distribution, as described in some embodiments of this application. According to embodiments of the present invention, a distortion vector is defined, and the distortion vector is fitted using a two-dimensional B-spline function to obtain the distortion function, specifically including:

[0108] S301, the image plane is divided into a regular grid, and each grid node corresponds to a sample point, resulting in a number of sample points;

[0109] S302, obtain the ideal coordinates and actual coordinates of each sample point;

[0110] S303 uses the actual coordinates of the sample points as input parameters for the two-dimensional B-spline function, calculates the deviation between the actual coordinates and the ideal undistorted coordinates, and generates a distortion vector.

[0111] S304, decompose the distortion vector into radial distortion components and tangential distortion components, and construct radial distortion component functions and tangential distortion component functions;

[0112] S305, the distortion function is obtained by fitting the radial distortion component function and the tangential distortion component function.

[0113] It should be noted that a two-dimensional B-spline function is used for fitting, as shown in the following formula:

[0114]

[0115] Denotes the basis function of the i-th k-th polar angle direction spline;

[0116] Represent the j-th l-th order azimuth direction spline basis function;

[0117] These represent the control point parameters corresponding to radial and tangential distortions.

[0118] m and n represent the number of control points.

[0119] According to an embodiment of the present invention, based on an elliptical fisheye lens acquiring images in a five-dimensional angular calibration space, analyzing lens imaging parameters, and generating parameter values ​​for a distortion function, the specific steps include:

[0120] Set the elliptical fisheye lens at the center of the five-dimensional angle calibration space and keep the elliptical fisheye lens horizontal;

[0121] Acquire images from several different angles and simultaneously record the pose information of the elliptical fisheye lens;

[0122] Extract image feature points, analyze the corner points of the image feature points, and perform position matching of corner points in images from different angles to obtain matching information;

[0123] Lens imaging parameters are obtained by analyzing matching information based on a nonlinear hybrid distortion imaging model.

[0124] The distortion function is input based on the lens imaging parameters, and the parameter values ​​of the distortion function are generated.

[0125] It should be noted that a five-sided stereoscopic calibration space is constructed to comprehensively acquire the spatial distortion response of the fisheye lens. The five-sided calibration structure is as follows:

[0126] Front plane (Z+);

[0127] Left and right planes (X-, X+);

[0128] The upper and lower planes are (Y+, Y-).

[0129] Each plane is fitted with a target surface having a precise two-dimensional calibration pattern (such as a checkerboard pattern), forming a three-dimensional point set. The corresponding pixel points are obtained through image acquisition.

[0130] Define an optimization function to fit the set of control points.

[0131]

[0132] in:

[0133] This represents the complete imaging map, which includes the elliptic mapping function and the distortion correction function.

[0134] N represents the total number of calibration points;

[0135] This represents the set of control point parameters (B-spline + ellipse axis length + nonlinear polar radius function parameters, etc.).

[0136] According to an embodiment of the present invention, calibration parameters are generated by analyzing the parameter values ​​of the distortion function based on a nonlinear hybrid distortion imaging model, and pixels in the distorted image are remapped based on the calibration parameters, specifically including:

[0137] The parameter values ​​of the distortion function are analyzed based on the nonlinear hybrid distortion imaging model, and the radial distortion parameter values ​​and tangential distortion parameter values ​​are analyzed based on the parameter values ​​of the distortion function.

[0138] The radial offset of a pixel is obtained by comparing the radial distortion parameter value with the standard radial distortion parameter value.

[0139] Generate radial calibration parameters based on the radial offset of each pixel;

[0140] The tangential distortion parameter value is compared with the standard tangential distortion parameter value to obtain the tangential offset of the pixel.

[0141] Tangential calibration parameters are generated based on the tangential offset of each pixel.

[0142] The final calibration parameters are obtained by weighted fusion of radial and tangential calibration parameters.

[0143] The pixels in the distorted image are remapped based on the final calibration parameters.

[0144] It should be noted that calibration parameters include internal parameters and external parameters. Internal parameters include basic intrinsic parameters and distortion parameters.

[0145] The basic internal parameters include the following parameters:

[0146] Focal length: fx, fy (considering the characteristics of an ellipse, the focal lengths are different in the two directions);

[0147] Principal points: cx, cy (image principal point offset);

[0148] Tilt factor: γ (pixel axis non-orthogonality);

[0149] Distortion parameters include:

[0150] Radial coefficients: k1, k2, k3;

[0151] Tangential coefficients: p1, p2;

[0152] Centrifugal coefficients: d1, d2, d3, d4;

[0153] Thin prism coefficients: s1, s2;

[0154] Extrinsics include:

[0155] Rotation matrix: R∈SO(3) (a 3×3 orthogonal matrix describing the camera pose);

[0156] Translation vector: t∈R3 (describes the camera position).

[0157] According to an embodiment of the present invention, image reconstruction is performed on the remapped distorted image to obtain a corrected image, specifically including:

[0158] Obtain the image coordinates of the remapped distorted image and convert the image coordinates into normalized camera coordinates;

[0159] Convert the normalized camera coordinates to the original distorted image coordinates;

[0160] The corresponding position of the normalized camera coordinates in the original distorted image is found based on the inverse operation of the distortion function;

[0161] The normalized camera coordinates at the same location are compared with the original distorted coordinates to obtain the coordinate difference.

[0162] The corrected image is obtained by filling in the pixel values ​​of the distorted image based on the coordinate difference.

[0163] It should be noted that image reconstruction is essentially a reverse mapping. Starting from the corrected ideal image coordinates, the corresponding position in the original distorted image is found through the inverse operation of the distortion function, and then the pixel values ​​are filled by interpolation.

[0164] Common methods for pixel value interpolation include:

[0165] Bilinear interpolation: Calculated using a weighted average of the surrounding 4 pixels (smooth but with blurred edges);

[0166] Nearest neighbor interpolation: takes the value of the nearest pixel (clear edges but prone to jagged edges);

[0167] Bicubic interpolation: uses 16 neighboring pixels for fitting (high fidelity but computationally intensive).

[0168] For the actually acquired distorted image I d (x,y), construct the inverse mapping function F -1 (x,y)→(θ,φ), obtaining the distortion-free target image coordinates (x,y) ′ ,y ′ ):

[0169] (x ′ ,y ′ )=G(θ,φ),where(θ,φ)=F -1 (x,y)

[0170] The reconstructed corrected image is given by the following formula:

[0171] I c (x ′,y ′ ) = Interp[I d [x,y)],(x,y)=F(θ,φ)

[0172] in:

[0173] I c (x ′ ,y ′ () indicates the corrected image;

[0174] Interp represents a bilinear or B-spline interpolation function;

[0175] F represents the original imaging function;

[0176] F -1 This represents the inverse numerical function.

[0177] like Figures 4-5 As shown, in a second aspect, embodiments of this application provide an image correction system based on a nonlinear distortion distribution elliptical fisheye lens. The system includes a memory and a processor. The memory includes a program for an image correction method based on a nonlinear distortion distribution elliptical fisheye lens. When the program for the image correction method based on a nonlinear distortion distribution elliptical fisheye lens is executed by the processor, it performs the following steps:

[0178] A nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation;

[0179] Define a distortion vector, and fit the distortion vector to a two-dimensional B-spline function to obtain the distortion function;

[0180] Based on the acquisition of images in a five-dimensional angular calibration space using an elliptical fisheye lens, the lens imaging parameters are analyzed to generate the parameter values ​​of the distortion function.

[0181] Based on the nonlinear hybrid distortion imaging model, the parameter values ​​of the distortion function are analyzed to generate calibration parameters, and the pixels in the distorted image are remapped based on the calibration parameters.

[0182] The remapped distorted image is then reconstructed to obtain the corrected image.

[0183] It should be noted that the distortion vector (Δx, Δy) is defined to include both radial and tangential components:

[0184]

[0185] in:

[0186] D r (θ,φ): Radial distortion function;

[0187] D t(θ,φ): Tangential distortion function.

[0188] It employs a non-angularly symmetrical lens structure, with an effective imaging plane that is elliptical and satisfies a two-dimensional elliptical geometric relationship:

[0189]

[0190] in:

[0191] x represents the horizontal coordinate of the image (unit: mm or pixels);

[0192] y represents the vertical coordinate of the image (unit: mm or pixels);

[0193] 'a' represents the length of the semi-major axis of the ellipse in the horizontal direction (unit: mm);

[0194] b represents the length of the semi-minor axis of the ellipse in the vertical direction (unit: mm).

[0195] The lens imaging function is modeled based on the spherical orientation (θ, φ), where:

[0196] θ∈[0,π / 2] represents the polar angle (the angle between the ray and the principal axis of the lens);

[0197] φ∈[0,2π) represents the azimuth angle (the direction of the light ray projected onto the horizontal plane).

[0198] According to an embodiment of the present invention, a nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation, specifically including:

[0199] Define the mapping relationship between the spherical orientation of an elliptical fisheye lens and the ideal image coordinates, and generate a mapping dataset based on the mapping relationship between the spherical orientation and the ideal image coordinates;

[0200] Construct a nonlinear hybrid distortion function, which includes a radial distortion function and a tangential distortion function;

[0201] Select an initial model framework, iteratively train the initial model framework based on the mapping dataset, and generate radial and tangential distortion values;

[0202] The distortion difference is obtained by comparing the radial distortion threshold and the tangential distortion threshold with the radial distortion value and the tangential distortion value, respectively.

[0203] The model parameters are dynamically optimized based on the distortion difference until the model converges, resulting in a nonlinear hybrid distortion imaging model.

[0204] The mapping relationship between image coordinates (x, y) and spherical directions (θ, φ) is as follows:

[0205]

[0206] in:

[0207] R(θ,φ) represents the nonlinear polar radius function, which describes the distortion caused by the combined action of polar angle and azimuth angle;

[0208] α(φ) represents the azimuth scaling function along the x-direction;

[0209] β(φ) represents the azimuth scaling function along the y-direction;

[0210] cos(φ) and sin(φ) represent the azimuth components.

[0211] According to an embodiment of the present invention, a distortion vector is defined, and a distortion function is obtained by fitting the distortion vector based on a two-dimensional B-spline function, specifically including:

[0212] The image plane is divided into a regular grid, with each grid node corresponding to a sample point, resulting in a number of sample points;

[0213] Obtain the ideal and actual coordinates of each sample point;

[0214] The actual coordinates of the sample points are used as input parameters for the two-dimensional B-spline function. The deviation between the actual coordinates and the ideal undistorted coordinates is calculated to generate a distortion vector.

[0215] The distortion vector is decomposed into radial distortion components and tangential distortion components, and radial distortion component functions and tangential distortion component functions are constructed.

[0216] The distortion function is obtained by fitting the radial distortion component function and the tangential distortion component function.

[0217] It should be noted that a two-dimensional B-spline function is used for fitting, as shown in the following formula:

[0218]

[0219] Denotes the basis function of the i-th k-th polar angle direction spline;

[0220] Represent the j-th l-th order azimuth direction spline basis function;

[0221] These represent the control point parameters corresponding to radial and tangential distortions.

[0222] m and n represent the number of control points.

[0223] According to an embodiment of the present invention, based on an elliptical fisheye lens acquiring images in a five-dimensional angular calibration space, analyzing lens imaging parameters, and generating parameter values ​​for a distortion function, the specific steps include:

[0224] Set the elliptical fisheye lens at the center of the five-dimensional angle calibration space and keep the elliptical fisheye lens horizontal;

[0225] Acquire images from several different angles and simultaneously record the pose information of the elliptical fisheye lens;

[0226] Extract image feature points, analyze the corner points of the image feature points, and perform position matching of corner points in images from different angles to obtain matching information;

[0227] Lens imaging parameters are obtained by analyzing matching information based on a nonlinear hybrid distortion imaging model.

[0228] The distortion function is input based on the lens imaging parameters, and the parameter values ​​of the distortion function are generated.

[0229] It should be noted that a five-sided stereoscopic calibration space is constructed to comprehensively acquire the spatial distortion response of the fisheye lens. The five-sided calibration structure is as follows:

[0230] Front plane (Z+);

[0231] Left and right planes (X-, X+);

[0232] The upper and lower planes are (Y+, Y-).

[0233] Each plane is fitted with a target surface having a precise two-dimensional calibration pattern (such as a checkerboard pattern), forming a three-dimensional point set. The corresponding pixel points are obtained through image acquisition.

[0234] Define an optimization function to fit the set of control points.

[0235]

[0236] in:

[0237] This represents the complete imaging map, which includes the elliptic mapping function and the distortion correction function.

[0238] N represents the total number of calibration points;

[0239] This represents the set of control point parameters (B-spline + ellipse axis length + nonlinear polar radius function parameters, etc.).

[0240] According to an embodiment of the present invention, calibration parameters are generated by analyzing the parameter values ​​of the distortion function based on a nonlinear hybrid distortion imaging model, and pixels in the distorted image are remapped based on the calibration parameters, specifically including:

[0241] The parameter values ​​of the distortion function are analyzed based on the nonlinear hybrid distortion imaging model, and the radial distortion parameter values ​​and tangential distortion parameter values ​​are analyzed based on the parameter values ​​of the distortion function.

[0242] The radial offset of a pixel is obtained by comparing the radial distortion parameter value with the standard radial distortion parameter value.

[0243] Generate radial calibration parameters based on the radial offset of each pixel;

[0244] The tangential distortion parameter value is compared with the standard tangential distortion parameter value to obtain the tangential offset of the pixel.

[0245] Tangential calibration parameters are generated based on the tangential offset of each pixel.

[0246] The final calibration parameters are obtained by weighted fusion of radial and tangential calibration parameters.

[0247] The pixels in the distorted image are remapped based on the final calibration parameters.

[0248] It should be noted that calibration parameters include internal parameters and external parameters. Internal parameters include basic intrinsic parameters and distortion parameters.

[0249] The basic internal parameters include the following parameters:

[0250] Focal length: fx, fy (considering the characteristics of an ellipse, the focal lengths are different in the two directions);

[0251] Principal points: cx, cy (image principal point offset);

[0252] Tilt factor: γ (pixel axis non-orthogonality);

[0253] Distortion parameters include:

[0254] Radial coefficients: k1, k2, k3;

[0255] Tangential coefficients: p1, p2;

[0256] Centrifugal coefficients: d1, d2, d3, d4;

[0257] Thin prism coefficients: s1, s2;

[0258] Extrinsics include:

[0259] Rotation matrix: R∈SO(3) (a 3×3 orthogonal matrix describing the camera pose);

[0260] Translation vector: t∈R3 (describes the camera position).

[0261] According to an embodiment of the present invention, image reconstruction is performed on the remapped distorted image to obtain a corrected image, specifically including:

[0262] Obtain the image coordinates of the remapped distorted image and convert the image coordinates into normalized camera coordinates;

[0263] Convert the normalized camera coordinates to the original distorted image coordinates;

[0264] The corresponding position of the normalized camera coordinates in the original distorted image is found based on the inverse operation of the distortion function;

[0265] The normalized camera coordinates at the same location are compared with the original distorted coordinates to obtain the coordinate difference.

[0266] The corrected image is obtained by filling in the pixel values ​​of the distorted image based on the coordinate difference.

[0267] It should be noted that image reconstruction is essentially a reverse mapping. Starting from the corrected ideal image coordinates, the corresponding position in the original distorted image is found through the inverse operation of the distortion function, and then the pixel values ​​are filled by interpolation.

[0268] Common methods for pixel value interpolation include:

[0269] Bilinear interpolation: Calculated using a weighted average of the surrounding 4 pixels (smooth but with blurred edges);

[0270] Nearest neighbor interpolation: takes the value of the nearest pixel (clear edges but prone to jagged edges);

[0271] Bicubic interpolation: uses 16 neighboring pixels for fitting (high fidelity but computationally intensive).

[0272] For the actually acquired distorted image I d (x,y), construct the inverse mapping function F -1 (x,y)→(θ,φ), obtaining the distortion-free target image coordinates (x,y) ′ ,y ′ ):

[0273] (x ′ ,y ′ )=G(θ,φ),where(θ,φ)=F -1 (x,y)

[0274] The reconstructed corrected image is given by the following formula:

[0275] I c (x ′ ,y ′ ) = Interp[I d [x,y)],(x,y)=F(θ,φ)

[0276] in:

[0277] I c (x ′ ,y ′ () indicates the corrected image;

[0278] Interp represents a bilinear or B-spline interpolation function;

[0279] F represents the original imaging function;

[0280] F -1 This represents the inverse numerical function.

[0281] A third aspect of the present invention provides a computer-readable storage medium including an image correction method program based on a nonlinear distortion distribution elliptical fisheye lens. When the image correction method program based on a nonlinear distortion distribution elliptical fisheye lens is executed by a processor, it implements the steps of the image correction method based on a nonlinear distortion distribution elliptical fisheye lens as described in any of the above claims.

[0282] This invention discloses an image correction method, system, and medium based on an elliptical fisheye lens with nonlinear distortion distribution. It establishes a nonlinear hybrid distortion imaging model based on the mapping relationship between image coordinates and spherical orientation; defines a distortion vector; fits the distortion vector using a two-dimensional B-spline function to obtain a distortion function; acquires images in a five-dimensional angular calibration space using the elliptical fisheye lens; analyzes lens imaging parameters; generates parameter values ​​for the distortion function; generates calibration parameters based on the parameter values ​​of the distortion function using the nonlinear hybrid distortion imaging model; remaps pixels in the distorted image using the calibration parameters; reconstructs the distorted image using the remapped image to obtain the corrected image; improves the sampling density of the target area by constructing a nonlinear hybrid distortion imaging model; adapts to complex monitoring environments such as corridors and entrances / exits; and flexibly expresses spatial distortion using B-splines, improving modeling capabilities.

[0283] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0284] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0285] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0286] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0287] Alternatively, if the integrated units of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. An image correction method based on a nonlinear distortion distribution elliptical fisheye lens, characterized in that, include: A nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation; Define a distortion vector, and fit the distortion vector to a two-dimensional B-spline function to obtain the distortion function; Based on the acquisition of images in a five-dimensional angular calibration space using an elliptical fisheye lens, the lens imaging parameters are analyzed to generate the parameter values ​​of the distortion function. Based on the nonlinear hybrid distortion imaging model, the parameter values ​​of the distortion function are analyzed to generate calibration parameters, and the pixels in the distorted image are remapped based on the calibration parameters. The remapped distorted image is then reconstructed to obtain the corrected image.

2. The image correction method based on a nonlinear distortion distribution elliptical fisheye lens according to claim 1, characterized in that, A nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation, specifically including: Define the mapping relationship between the spherical orientation of an elliptical fisheye lens and the ideal image coordinates, and generate a mapping dataset based on the mapping relationship between the spherical orientation and the ideal image coordinates; Construct a nonlinear hybrid distortion function, which includes a radial distortion function and a tangential distortion function; Select an initial model framework, iteratively train the initial model framework based on the mapping dataset, and generate radial and tangential distortion values; The distortion difference is obtained by comparing the radial distortion threshold and the tangential distortion threshold with the radial distortion value and the tangential distortion value, respectively. The model parameters are dynamically optimized based on the distortion difference until the model converges, resulting in a nonlinear hybrid distortion imaging model.

3. The image correction method based on a nonlinear distortion distribution elliptical fisheye lens according to claim 2, characterized in that, Define a distortion vector, and fit the distortion vector using a two-dimensional B-spline function to obtain the distortion function, which specifically includes: The image plane is divided into a regular grid, with each grid node corresponding to a sample point, resulting in a number of sample points; Obtain the ideal and actual coordinates of each sample point; The actual coordinates of the sample points are used as input parameters for the two-dimensional B-spline function. The deviation between the actual coordinates and the ideal undistorted coordinates is calculated to generate a distortion vector. The distortion vector is decomposed into radial distortion components and tangential distortion components, and radial distortion component functions and tangential distortion component functions are constructed. The distortion function is obtained by fitting the radial distortion component function and the tangential distortion component function.

4. The image correction method based on a nonlinear distortion distribution elliptical fisheye lens according to claim 3, characterized in that, Images are acquired using an elliptical fisheye lens within a five-dimensional angular calibration space. Lens imaging parameters are analyzed, and distortion function parameter values ​​are generated, specifically including: Set the elliptical fisheye lens at the center of the five-dimensional angle calibration space and keep the elliptical fisheye lens horizontal; Acquire images from several different angles and simultaneously record the pose information of the elliptical fisheye lens; Extract image feature points, analyze the corner points of the image feature points, and perform position matching of corner points in images from different angles to obtain matching information; Lens imaging parameters are obtained by analyzing matching information based on a nonlinear hybrid distortion imaging model. The distortion function is input based on the lens imaging parameters, and the parameter values ​​of the distortion function are generated.

5. The image correction method based on a nonlinear distortion distribution elliptical fisheye lens according to claim 4, characterized in that, Based on the nonlinear hybrid distortion imaging model, the parameter values ​​of the distortion function are analyzed to generate calibration parameters. Based on these calibration parameters, pixels in the distorted image are remapped, specifically including: The parameter values ​​of the distortion function are analyzed based on the nonlinear hybrid distortion imaging model, and the radial distortion parameter values ​​and tangential distortion parameter values ​​are analyzed based on the parameter values ​​of the distortion function. The radial offset of a pixel is obtained by comparing the radial distortion parameter value with the standard radial distortion parameter value. Generate radial calibration parameters based on the radial offset of each pixel; The tangential distortion parameter value is compared with the standard tangential distortion parameter value to obtain the tangential offset of the pixel. Tangential calibration parameters are generated based on the tangential offset of each pixel. The final calibration parameters are obtained by weighted fusion of radial and tangential calibration parameters. The pixels in the distorted image are remapped based on the final calibration parameters.

6. The image correction method based on a nonlinear distortion distribution elliptical fisheye lens according to claim 5, characterized in that, The remapped distorted image is then reconstructed to obtain the corrected image, specifically including: Obtain the image coordinates of the remapped distorted image and convert the image coordinates into normalized camera coordinates; Convert the normalized camera coordinates to the original distorted image coordinates; The corresponding position of the normalized camera coordinates in the original distorted image is found based on the inverse operation of the distortion function; The normalized camera coordinates at the same location are compared with the original distorted coordinates to obtain the coordinate difference. The corrected image is obtained by filling in the pixel values ​​of the distorted image based on the coordinate difference.

7. An image correction system based on a nonlinear distortion distribution elliptical fisheye lens, characterized in that, The system includes a memory and a processor. The memory contains a program for image correction based on a nonlinear distortion distribution elliptical fisheye lens. When the program for image correction based on a nonlinear distortion distribution elliptical fisheye lens is executed by the processor, it performs the following steps: A nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation; Define a distortion vector, and fit the distortion vector to a two-dimensional B-spline function to obtain the distortion function; Based on the acquisition of images in a five-dimensional angular calibration space using an elliptical fisheye lens, the lens imaging parameters are analyzed to generate the parameter values ​​of the distortion function. Based on the nonlinear hybrid distortion imaging model, the parameter values ​​of the distortion function are analyzed to generate calibration parameters, and the pixels in the distorted image are remapped based on the calibration parameters. The remapped distorted image is then reconstructed to obtain the corrected image.

8. The image correction system based on a nonlinear distortion distribution elliptical fisheye lens according to claim 7, characterized in that, A nonlinear hybrid distortion imaging model is established based on the mapping relationship between image coordinates and spherical orientation, specifically including: Define the mapping relationship between the spherical orientation of an elliptical fisheye lens and the ideal image coordinates, and generate a mapping dataset based on the mapping relationship between the spherical orientation and the ideal image coordinates; Construct a nonlinear hybrid distortion function, which includes a radial distortion function and a tangential distortion function; Select an initial model framework, iteratively train the initial model framework based on the mapping dataset, and generate radial and tangential distortion values; The distortion difference is obtained by comparing the radial distortion threshold and the tangential distortion threshold with the radial distortion value and the tangential distortion value, respectively. The model parameters are dynamically optimized based on the distortion difference until the model converges, resulting in a nonlinear hybrid distortion imaging model.

9. The image correction system based on a nonlinear distortion distribution elliptical fisheye lens according to claim 8, characterized in that, Define a distortion vector, and fit the distortion vector using a two-dimensional B-spline function to obtain the distortion function, which specifically includes: The image plane is divided into a regular grid, with each grid node corresponding to a sample point, resulting in a number of sample points; Obtain the ideal and actual coordinates of each sample point; The actual coordinates of the sample points are used as input parameters for the two-dimensional B-spline function. The deviation between the actual coordinates and the ideal undistorted coordinates is calculated to generate a distortion vector. The distortion vector is decomposed into radial distortion components and tangential distortion components, and radial distortion component functions and tangential distortion component functions are constructed. The distortion function is obtained by fitting the radial distortion component function and the tangential distortion component function.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes an image correction method program based on a nonlinear distortion distribution elliptical fisheye lens. When the image correction method program based on a nonlinear distortion distribution elliptical fisheye lens is executed by a processor, it implements the steps of the image correction method based on a nonlinear distortion distribution elliptical fisheye lens as described in any one of claims 1 to 6.

Images