A method for correcting distortion of a fisheye lens panoramic image

Abstract

The application provides a fisheye lens panoramic image distortion correction method, and relates to the technical field of data processing, which comprises the following steps: correcting the spatial configuration by using a steady-state disturbance correction vector to obtain a steady-state configuration; mapping and transforming each pixel point in the original panoramic image according to the steady-state configuration to obtain a first corrected image; performing radial distortion correction on each pixel point in the first corrected image according to the first corrected image and a radial distortion component to obtain a radial corrected image; performing tangential distortion correction on the radial corrected image according to the radial corrected image and a tangential distortion component to obtain a tangential corrected image; and performing balanced processing on the edge brightness and the definition of the tangential corrected image to obtain a target corrected image. The application can improve the geometric consistency and visual quality of the panoramic image.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for distortion correction of panoramic images using fisheye lenses. Background Technology

[0002] In drone aerial photography applications, fisheye lenses are often used to capture images with a wide field of view. Fisheye lenses can achieve ultra-wide-angle imaging, with some models boasting a field of view of up to 245°, covering a large area of ​​scene information in a single image. However, in practical use, due to the optical design characteristics of fisheye lenses, the panoramic images captured often exhibit significant geometric distortion in the edge areas. For example, when shooting urban building complexes using a fisheye lens with a 245° field of view, the building facades or road markings that originally had regular straight-line structures at the edges of the image often appear as irregular lines in the final image. The same degree of arc curvature alters the true geometric shape of the target object. Existing distortion correction methods are mostly based on globally unified mapping models, such as obtaining fixed distortion parameters through a calibration plate and then performing a global transformation on the entire image. When processing such ultra-wide-angle fisheye images, these methods may have certain limitations in correcting local nonlinear distortions introduced by the large field of view in the edge regions. This manifests as slight curvature remaining in the straight structures of some edge regions after correction, or a certain degree of stretching and deformation of local textures during the correction process. This affects the accuracy of subsequent image stitching and the target recognition effect based on geometric features to some extent. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a distortion correction method for panoramic images with fisheye lenses, which can improve the geometric consistency and visual quality of panoramic images.

[0004] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0005] In a first aspect, a distortion correction method for panoramic images using fisheye lenses is provided, the method comprising:

[0006] Step 1: Acquire the original panoramic image captured by the fisheye lens; extract the imaging parameters and distortion parameters based on the original panoramic image.

[0007] Step 2: Based on the imaging parameters, establish the spatial configuration between pixels in the original panoramic image;

[0008] Step 3: Set three calibration points in the original panoramic image and connect them to form a state-phase domain. Divide the state-phase domain into multiple heterogeneous topographic primitives according to the pixel density within the domain. Calculate the planar area of ​​each heterogeneous topographic primitive. Determine the topographic constraint coefficient based on the union area between the planar areas of each heterogeneous topographic primitive and the planar areas of adjacent heterogeneous topographic primitives. Calculate the distortion manifold displacement field based on the relative displacement of pixels within the heterogeneous topographic primitives and the topographic constraint coefficient. Set the initial spatial mapping perturbation based on the distortion manifold displacement field and iteratively update it until convergence to obtain the steady-state perturbation correction vector. Use the steady-state perturbation correction vector to correct the spatial configuration to obtain the steady-state configuration.

[0009] Step 4: Based on the steady-state configuration, perform mapping transformation on each pixel in the original panoramic image to obtain the first corrected image; based on the first corrected image and the radial distortion component, perform radial distortion correction on each pixel in the first corrected image to obtain the radial corrected image.

[0010] Step 5: Based on the radially corrected image and the tangential distortion component, perform tangential distortion correction on the radially corrected image to obtain the tangentially corrected image; perform edge brightness and sharpness equalization processing on the tangentially corrected image to obtain the target corrected image.

[0011] In a second aspect, a computing device includes:

[0012] One or more processors;

[0013] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0014] Thirdly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0015] The above-described solution of the present invention has at least the following beneficial effects:

[0016] By setting multiple calibration points in the original panoramic image to construct the state-phase domain, combining pixel sparseness to divide heterogeneous morphological primitives, introducing morphological constraint coefficients to quantify the spatial correlation between primitives, and then constructing a distortion manifold displacement field and obtaining a steady-state perturbation correction vector through iterative optimization, the spatial configuration can be accurately corrected. This design can accurately capture the local nonlinear distortion characteristics of the edge region of an ultra-wide-angle fisheye lens (such as a field of view of up to 245°), effectively solving the problems of residual bending of straight structures and local texture stretching deformation in the edge region. Attached Figure Description

[0017] Figure 1This is a schematic flowchart of a distortion correction method for panoramic images using fisheye lenses, provided by an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the process of establishing the spatial configuration between pixels in the original panoramic image based on imaging parameters, provided by an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the convergence process of the iterative update of the displacement field of the distorted manifold provided in an embodiment of the present invention.

[0020] Figure 4 This is a diagram showing the curvature of the image edge structure before radial distortion correction, provided by an embodiment of the present invention.

[0021] Figure 5 This is a diagram showing the curvature of the edge structure of an image after radial distortion correction, provided by an embodiment of the present invention. Detailed Implementation

[0022] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0023] like Figure 1 As shown, an embodiment of the present invention proposes a distortion correction method for panoramic images using fisheye lenses, the method comprising the following steps:

[0024] Step 1: Acquire the original panoramic image captured by the fisheye lens; extract the imaging parameters and distortion parameters based on the original panoramic image.

[0025] Step 2: Based on the imaging parameters, establish the spatial configuration between pixels in the original panoramic image;

[0026] Step 3: Set three calibration points in the original panoramic image and connect them to form a state-phase domain. Divide the state-phase domain into multiple heterogeneous topographic primitives according to the pixel density within the domain. Calculate the planar area of ​​each heterogeneous topographic primitive. Determine the topographic constraint coefficient based on the union area between the planar areas of each heterogeneous topographic primitive and the planar areas of adjacent heterogeneous topographic primitives. Calculate the distortion manifold displacement field based on the relative displacement of pixels within the heterogeneous topographic primitives and the topographic constraint coefficient. Set the initial spatial mapping perturbation based on the distortion manifold displacement field and iteratively update it until convergence to obtain the steady-state perturbation correction vector. Use the steady-state perturbation correction vector to correct the spatial configuration to obtain the steady-state configuration.

[0027] Step 4: Based on the steady-state configuration, perform mapping transformation on each pixel in the original panoramic image to obtain the first corrected image; based on the first corrected image and the radial distortion component, perform radial distortion correction on each pixel in the first corrected image to obtain the radial corrected image.

[0028] Step 5: Based on the radially corrected image and the tangential distortion component, perform tangential distortion correction on the radially corrected image to obtain the tangentially corrected image; perform edge brightness and sharpness equalization processing on the tangentially corrected image to obtain the target corrected image.

[0029] In this embodiment of the invention, a state-phase domain is constructed by setting multiple calibration points in the original panoramic image. Heterogeneous morphological primitives are divided by pixel sparseness, and morphological constraint coefficients are introduced to quantify the spatial correlation between primitives. Then, a distortion manifold displacement field is constructed and a steady-state perturbation correction vector is obtained through iterative optimization, thereby achieving accurate correction of the spatial configuration. This design can accurately capture the local nonlinear distortion characteristics of the edge region of an ultra-wide-angle fisheye lens (such as a field of view of up to 245°), effectively solving the problems of residual bending of straight structures and local texture stretching deformation in the edge region.

[0030] In a preferred embodiment of the present invention, step 1 involves acquiring the original panoramic image captured by the fisheye lens; based on the original panoramic image, imaging parameters and distortion parameters are extracted. The imaging parameters include focal length, aperture, maximum image height, field of view, principal ray angle, optical back focal length, total optical length, and resolution. The distortion parameters include radial distortion components and tangential distortion components. Specifically, this embodiment uses an ultra-wide-angle fisheye lens for image acquisition, which is suitable for industrial aerial photography scenarios. The lens has a maximum field of view of 245°, supports up to 12 megapixels of imaging, a focal length between 0.85mm and 2.50mm, an aperture between 2.0 and 2.8, a maximum image height between 2.89mm and 8.00mm, and a principal ray angle (CRA) of 3.54°. The optical back focal length (BFL) ranges from 2.38mm to 5.52mm, and the total optical length (TCL) ranges from 12.79mm to 42.16mm. The fisheye lens is mechanically assembled with the image acquisition device, ensuring the lens optical axis is perpendicular to the image sensor target surface and that the lens optical center is perfectly aligned with the sensor target surface's geometric center to prevent displacement, tilting, or loosening. After assembly, the image acquisition device is activated, capturing a single frame at the preset frame rate and exposure parameters to obtain the original panoramic image without distortion correction, geometric transformation, cropping, or compression. The original panoramic image is stored in an RGB three-channel bitmap format, fully preserving all pixel and brightness information acquired by the sensor.

[0031] Imaging parameters are directly extracted from the pixel distribution of the original panoramic image and the physical structure of the lens. The focal length is the vertical distance from the optical center of the lens to the image plane, and the aperture F-number is determined by the ratio of the lens's effective aperture to the focal length, specifically according to the formula F= The calculation is as follows: f is the lens focal length, D is the physical size of the effective aperture, and the maximum image height is the diameter of the maximum imaging circle formed by the lens on the sensor surface. This value is determined by the lens optical structure and the sensor size. The field of view includes the diagonal field of view, the horizontal field of view, and the vertical field of view. The diagonal field of view is directly determined by the lens optical design and focal length parameters. The horizontal and vertical field of view are calculated according to the formulas... and Calculate, where For horizontal field of view, For vertical field of view, Hs is the effective imaging surface width of the sensor, f is the effective imaging surface height of the sensor, and f is the lens focal length. The principal ray angle CRA is the angle between the principal ray and the optical axis at the maximum image height position. This angle is determined by actual measurement of the lens optical structure. The optical back focal length BFL is the vertical distance from the surface of the last optical element of the lens to the sensor imaging surface. The total optical length TCL is the total length from the surface of the first optical element of the lens to the sensor imaging surface. The resolution is the number of horizontal and vertical pixels in the original panoramic image. The number of horizontal and vertical pixels is directly determined by reading the sensor specifications of the image acquisition device.

[0032] Distortion parameters are extracted based on the pixel distortion distribution characteristics of the original panoramic image. Canny edge detection is performed on the original panoramic image to extract naturally existing linear structures. Least squares line fitting is then performed on all pixels on the extracted linear structures to obtain the expression U= for the ideal line. x+ In the formula, x is the horizontal coordinate of a pixel on the linear structure, which is the independent variable of the ideal line; U is the theoretical vertical coordinate of a pixel on the linear structure calculated based on the ideal line, which is the dependent variable of the ideal line. The slope of the ideal straight line; For the ideal linear intercept; for each actual pixel on a linear structure. According to the formula Calculate the offset distance of the pixel from the ideal straight line. In the formula The horizontal coordinates of the actual pixel. Given the vertical coordinates of the actual pixels, iterate through all the pixels involved in the calculation to obtain the set of offset distances. The radial distance from the pixel to the center of the image As the independent variable, with offset distance Using the radial offset distance as the dependent variable, a quadratic polynomial fit is performed to obtain the radial distortion expression. In the formula Here, r represents the radial distortion offset, and r is the radial distance from the pixel to the image center. The first-order radial distortion coefficient, The second-order radial distortion coefficient is used to determine the radial distortion component. A bivariate quadratic polynomial is fitted to the offset distance in the tangential direction to obtain the tangential distortion expression. In the formula The offset is the tangential distortion, where u is the horizontal coordinate of the pixel and v is the vertical coordinate of the pixel. The first tangential distortion coefficient, The second tangential distortion coefficient is used to determine the tangential distortion component. All distortion coefficients are calculated by directly fitting the pixel offset data of the original panoramic image.

[0033] like Figure 2 As shown, in another preferred embodiment of the present invention, step 2, establishing the spatial configuration between pixels in the original panoramic image based on imaging parameters, may include:

[0034] Step 201: Based on the focal length and field of view, convert the image coordinates of each pixel into the corresponding incident ray angle to obtain angular distribution data. Specifically, this includes: using the original panoramic image obtained in step 101 as the processing object, and combining the focal length and field of view parameters extracted in step 101, performing coordinate-to-incident ray angle conversion calculations on all pixels in the image point by point; establishing a two-dimensional Cartesian coordinate system with the geometric center point of the original panoramic image as the origin, and the horizontal axis as the u-axis and the vertical axis as the v-axis; traversing each pixel in the original panoramic image and reading the corresponding pixel coordinates (u, v), where the unit of pixel coordinates is pixels, where u represents the value of the pixel on the horizontal axis of the image, and v represents the value of the pixel on the vertical axis of the image; for any pixel... For each pixel, the straight-line distance from the pixel to the origin is calculated directly; this distance is the pixel distance from the pixel to the image center origin. Based on the image's horizontal and vertical resolutions, the maximum straight-line distance from the image center to the diagonal vertex is calculated; this distance is the maximum pixel distance from the image center to the diagonal vertex. The normalized radial distance is obtained by comparing the distance from the pixel to the center with the maximum pixel distance. The normalized radial distance is constrained to a value between 0 and 1, with 0 corresponding to the image center and 1 corresponding to the diagonal edge. The polar angle of the incident ray is obtained by multiplying the normalized radial distance by half the diagonal field of view; the polar angle represents the angle between the incident ray and the lens optical axis. The azimuth angle of the incident ray corresponding to the pixel is calculated using the four-quadrant arctangent function, with the following formula: In the formula This represents the azimuth angle of the incident ray in the plane perpendicular to the optical axis, and its value ranges from [value missing]. , The vertical coordinate value of the pixel. The horizontal coordinate value of the pixel is used as the reference. The above coordinate transformation calculation is repeated for all pixels in the original panoramic image to convert the image coordinates of each pixel into a unique incident light angle. All incident light angle data are stored in the order of the original pixel positions to form angle distribution data that corresponds one-to-one with the pixels of the original image.

[0035] Step 202: Based on the maximum image height and the principal ray angle, perform image plane projection mapping on each incident angle in the angle distribution data to obtain the initial coordinate set of all pixels. Specifically, this includes: using the angle distribution data obtained in step 201 as input data, and combining the maximum image height and principal ray angle parameters extracted in step 101, performing image plane projection mapping calculation point by point for each set of incident ray angles; establishing a physical coordinate system with the intersection of the lens optical axis and the image plane as the origin, and the horizontal axis of the image plane as the x-axis and the vertical axis as the y-axis, with the unit of the physical coordinates being millimeters, and this coordinate system being consistent with the actual imaging plane of the fisheye lens; for each polar angle value in the angle distribution data, performing deviation correction in combination with the principal ray angle CRA to obtain the corrected projection polar angle, calculated using the following formula: In the formula This indicates the corrected projection polar angle, in degrees. The polar angle of the incident ray before correction is expressed in degrees. CRA represents the principal ray angle of the fisheye lens at the maximum image height, also in degrees. The maximum physical projection radius of the image plane is obtained by multiplying the maximum physical projection radius by the sine of the corrected polar angle. The formula is as follows: In the formula This represents the projection radius of a pixel onto the image plane, in millimeters. The maximum image height of the fisheye lens is expressed in millimeters. The initial horizontal and vertical coordinates of the pixel on the image plane are obtained by multiplying the projection radius by the cosine and sine of the azimuth angle, respectively, using the following formulas: In the formula This represents the initial horizontal coordinates of a pixel on the image plane, in millimeters. This represents the initial vertical coordinates of a pixel on the image plane, in millimeters. The initial coordinates represent the azimuth angle of the incident light ray; the initial coordinates are the physical projection positions without depth correction; the above projection mapping calculation is performed on all incident light ray angles in the angle distribution data to obtain the initial physical coordinates corresponding to each incident angle; all initial physical coordinates are stored one by one according to the original image pixel positions to form a complete set of initial coordinates.

[0036] Step 203: Based on the optical back focal length and total optical length, perform spatial depth correction on the initial coordinate set, calculate the spatial distance and directional relationship between adjacent pixels point by point, and establish the spatial configuration between pixels. Specifically, this includes: using the initial coordinate set obtained in step 202 as the processing object, combining the optical back focal length and total optical length parameters extracted in step 101, performing three-dimensional spatial depth correction on the initial coordinates and constructing the spatial topological relationship between pixels; calculating the depth reference offset by the difference between the total optical length and the optical back focal length, the calculation formula is as follows: In the formula This represents the depth reference offset from the equivalent optical center of the lens to the image plane, expressed in millimeters. This refers to the total optical length of a fisheye lens, expressed in millimeters. The optical back focal length of the fisheye lens is expressed in millimeters. For any set of two-dimensional initial coordinates in the initial coordinate set, the depth reference offset is assigned as the depth coordinate, expanding the two-dimensional initial coordinates into three-dimensional initial coordinates. Fine-tuning calculations are performed on the depth coordinates using the incident polar angle obtained in step 201. The final depth coordinates are obtained by multiplying the original depth coordinates by the cosine of the polar angle, using the following formula: In the formula This represents the final depth coordinates after correction for the incident angle, in millimeters. This represents the initial 3D depth coordinates, in millimeters. The polar angle of the incident light ray is expressed in degrees. The depth correction process described above is applied to all pixels in the initial coordinate set to obtain the depth-corrected 3D spatial coordinates of all pixels. All pixels in the original panoramic image are traversed, and any pixel and its eight neighboring pixels are selected. The straight-line distance between the two pixels in 3D space is directly calculated; this distance represents the true spatial interval between adjacent pixels. A spatial direction vector is constructed using the 3D coordinate difference between adjacent pixels. The ratio of the spatial direction vector to the corresponding spatial distance is used to obtain the unit direction vector, which represents the spatial pointing relationship between two pixels. The 3D spatially corrected coordinates of all pixels, the spatial distance between adjacent pixels, and the unit direction vector are integrated and stored according to the pixel position relationship of the original image, forming the spatial configuration between pixels.

[0037] This embodiment converts pixel coordinates into incident light angles in steps, performs image plane projection mapping and spatial depth correction, and combines specific parameters such as lens focal length, field of view, and maximum image height to establish a precise pixel spatial configuration, effectively avoiding distortion correction deviations caused by inaccurate spatial configuration, and adapting to the optical structural characteristics of the lens.

[0038] In a preferred embodiment of the present invention, step 3 involves setting three calibration points in the original panoramic image and connecting them to form a state-phase domain. The state-phase domain is then divided into multiple heterogeneous topographic primitives based on the pixel density within it. The planar area of ​​each heterogeneous topographic primitive is calculated, and a topographic constraint coefficient is determined based on the union area between the planar areas of the primitives and those of adjacent heterogeneous topographic primitives. A distorted manifold displacement field is calculated based on the relative displacement of pixels within the heterogeneous topographic primitive and the topographic constraint coefficient. An initial spatial mapping perturbation is set based on the distorted manifold displacement field and iteratively updated until convergence to obtain a steady-state perturbation correction vector. The steady-state perturbation correction vector is used to correct the spatial configuration to obtain a steady-state configuration, which may include:

[0039] Step 301: Select three calibration points in the original panoramic image. The first calibration point is located in the image center region, the second calibration point is located in the image edge region, and the third calibration point is located in the transition region between the center region and the edge region. Specifically, using the original panoramic image obtained in step 101 as the processing object, and based solely on the image's horizontal resolution, vertical resolution, and maximum image height, automatically calculate and determine the coordinates of the three calibration points. Let the total horizontal pixel count of the original panoramic image be W, and the total vertical pixel count of the original panoramic image be H. The first calibration point is selected at the geometric center of the image, which is the region with the least image distortion, and its pixel coordinates ( , Determine according to the following formula: In the formula The horizontal pixel coordinates of the first calibration point; The first calibration point is the vertical pixel coordinate; the second calibration point is selected at the outermost edge of the effective imaging circle of the image, which is the area with the most obvious image distortion, and is based on the maximum image height obtained in step 101. Using the corresponding imaging circle boundary as a reference, edge pixels are selected along the diagonal direction of the image, and their coordinates are ( , Determine according to the following formula: In the formula The horizontal pixel coordinates of the second calibration point; The vertical pixel coordinates of the second calibration point; This represents the number of pixels per millimeter horizontally of the image sensor, expressed in pixels per millimeter. The third calibration point is selected in the transition area between the central and edge regions to characterize the distortion gradient characteristics; its coordinates are ( , The coordinates of the midpoint between the first and second calibration points are determined according to the following formula: In the formula The horizontal pixel coordinates of the third calibration point; The vertical pixel coordinates of the third calibration point, in pixels; The pixel coordinates of the first calibration point; These are the pixel coordinates of the second calibration point.

[0040] Step 302: Connect the three calibration points pairwise. The region enclosed by the three calibration points and the connecting lines is defined as the state-phase domain. Calculate the density between each pixel and its surrounding neighboring pixels within the state-phase domain to obtain the pixel sparsity. Specifically, this includes: connecting the three calibration points determined in step 301... Connect each pixel with a straight line, forming a closed triangular region. This region includes the three calibration points themselves and all pixels within it. This closed region is defined as the state-phase domain, which completely covers the three regions with different distortion features: the image center, transition, and edge. Traverse each pixel (u, v) within the state-phase domain, selecting a 3×3 eight-neighbor area centered on the current pixel. An eight-neighbor area refers to the eight adjacent pixels centered on the current pixel in the horizontal, vertical, and diagonal directions (top, bottom, left, right, top-left, top-right, bottom-left, bottom-right). Count the number of valid pixels actually existing within this neighborhood. The theoretical maximum number of pixels in a 3×3 neighborhood. Using 9 as a baseline, the density of the current pixel is calculated according to the following formula: (u,v)= In the formula This represents the density value of the current pixel. This represents the theoretical maximum number of pixels that can be contained within a 3×3 neighborhood, and is fixed at 9. This represents the actual number of valid pixels within the current pixel's 3×3 eight-neighborhood, expressed in pixels.

[0041] Step 303: Divide the state-phase domain into multiple sub-regions according to the pixel density from high to low. Each sub-region is defined as a heterogeneous morphology primitive, specifically including: the density distribution data obtained in step 302. Based on this, all pixels in the phase domain are sorted from largest to smallest according to their sparsity values, and pixels with sparsity differences less than a preset gradient threshold are selected. Furthermore, pixels that are spatially continuous and connected are grouped into the same set. Based on spatial connectivity, the state-phase domain is divided into multiple non-overlapping, gapless, and completely covered sub-regions. Within each sub-region, the pixel sparseness is uniform and the distortion characteristics are consistent. There are significant differences in sparseness and distortion characteristics between different sub-regions. Each independent sub-region is defined as a heterogeneous morphological primitive. ,in Let i represent the i-th heterogeneous morphology primitive, where i is the index of the heterogeneous morphology primitive, which increases sequentially starting from 1, and n is the total number of heterogeneous morphology primitives obtained by the final segmentation within the state-phase domain.

[0042] Step 304: Calculate the projected area of ​​each heterogeneous shape primitive on the image plane. For any pair of adjacent heterogeneous shape primitives, calculate the union area of ​​the two heterogeneous shape primitive plane areas. Divide the union area by the sum of the areas of the two heterogeneous shape primitives to obtain the shape constraint coefficient. Specifically, this includes: for each heterogeneous shape primitive obtained from the segmentation in step 303... Traverse all pixel coordinates on the primitive boundary ( The shoelace formula is used to calculate the projected area of ​​the primitive on the image plane. The calculation formula is: In the formula, Let be the projected area of ​​the i-th heterogeneous morphological primitive on the image plane; To perform an accumulation operation on all pixels on the primitive boundary; This represents the total number of pixels on the boundary of the current heterogeneous topography primitive; Let be the horizontal coordinate of the b-th pixel on the primitive boundary; Let be the vertical coordinate of the b-th pixel on the primitive boundary; Let be the horizontal coordinate of the (b+1)th pixel on the primitive boundary; Let be the vertical coordinate of the (b+1)th pixel on the primitive boundary; it is agreed that when hour, and To achieve boundary closure; select any pair of adjacent primitives with a common boundary from all primitives. and Calculate the union of the planar areas of two primitives. The union area is the total area covered by the merged primitives; calculate the shape constraint coefficient of this set of adjacent primitives. In the formula, The morphological constraint coefficient is the coefficient between the i-th and j-th adjacent heterogeneous morphological primitives. Let be the area of ​​the union of the i-th and j-th heterogeneous morphological primitives on the image plane; The projected area of ​​the i-th heterogeneous shape primitive is expressed in square pixels. Let be the projected area of ​​the j-th heterogeneous morphological primitive.

[0043] Step 305: Within each heterogeneous topography primitive, extract the relative displacement of each pixel relative to its surrounding neighboring pixels; calculate the weighted displacement value of each pixel by weighting the relative displacement according to the topography constraint coefficient. Specifically, this includes: within each heterogeneous topography primitive... Internally, iterate through each pixel. Using the geometric center of the 3×3 eight neighboring pixels of the current pixel as the ideal reference position, the coordinate deviation of the current pixel relative to the ideal reference center is calculated to obtain the relative displacement of the pixel. Its horizontal component Longitudinal component They are respectively: In the formula This represents the lateral relative displacement of a pixel. This represents the vertical relative displacement of a pixel. v represents the actual horizontal coordinate of the pixel; v represents the actual vertical coordinate of the pixel. It is the average of the horizontal coordinates of all pixels in the 3×3 eight-neighborhood of the pixel. The average of the vertical coordinates of all pixels in the 3×3 eight-neighborhood of the pixel; the shape constraint coefficient corresponding to the primitive obtained in step 304 is called. ,by The relative displacements are weighted and calculated to obtain the weighted displacement values. The calculation formula is: In the formula This is the weighted displacement value of the current pixel after constraint weighting; (u,v) represents the relative displacement of the current pixel; It represents the shape constraint coefficient corresponding to the heterogeneous shape primitive to which the current pixel belongs.

[0044] Step 306: Accumulate the weighted displacement values ​​of all pixels within the same heterogeneous topography primitive, calculate the average value based on the total number of pixels within the primitive, and obtain the average distortion contribution. Based on the spatial positions of all heterogeneous topography primitives in the state-phase domain, synthesize the average distortion contributions of each primitive to obtain the distortion manifold displacement field. Specifically, this includes: for a single heterogeneous topography primitive... Accumulate the weighted displacement values ​​of all pixels within the primitive to obtain the total weighted displacement sum, and then count the total number of pixels within the primitive. The average distortion contribution of the primitive is calculated according to the following formula: In the formula The average distortion contribution of the i-th heterogeneous morphology unit; It is the sum of the weighted displacement values ​​of all pixels within the i-th heterogeneous morphology primitive; This represents the weighted displacement value of the p-th pixel within the i-th primitive; Let be the total number of pixels contained within the i-th heterogeneous topographic primitive; p is the index of the pixels within the primitive, increasing sequentially from 1 to the total number of pixels; to ensure that the displacement field of the distorted manifold remains continuously differentiable, without boundary jumps or singular abrupt changes throughout the entire state-phase domain, this embodiment introduces a complex function encirclement integral algorithm to spatially synthesize and smoothly reconstruct the average distortion contribution of each primitive; any spatial position within the state-phase domain is mapped to a complex plane variable, expressed as: In the formula, z is the position variable on the complex plane; x is the horizontal physical coordinate of the pixel in the state-phase domain; The imaginary unit satisfies =-1; y is the vertical physical coordinate of the pixel in the state-phase domain; each heterogeneous topography primitive is regarded as a simply connected region on the complex plane, and its average distortion contribution is used as the complex-valued distortion density function in the region. The closed boundary of the state-phase domain is used as the confinement path for integration. Confinement path integration is performed on all primitive regions to achieve global smooth fusion of distortion contribution. The integration formula is as follows: In the formula The displacement field of the distorted manifold, reconstructed by the complex plane after convolution of the complex function; 1 is the fixed coefficient in the numerator of the integral formula; For the fixed coefficient terms in the Cauchy integral formula for complex functions; For the closed enclosure Complex integral operators; The closed integral encirclement formed by the outer boundary of the state-phase domain; To perform an accumulation operation on all heterogeneous morphology primitives; n is the total number of heterogeneous morphology primitives obtained by segmentation within the state-phase domain; Let be the region indicator function for the i-th heterogeneous morphology primitive, taking a value of 1 inside the primitive and a value of 0 outside the primitive; For closed integral enclosed path The integral complex variable on the complex plane; z is the target position variable of the distorted manifold value to be solved on the complex plane, which is dimensionless; For integral complex variables The differential unit; the complex numerical result obtained by the boundary integral is mapped from the complex plane back to the two-dimensional image plane, and the real part is extracted to obtain the real numerical distortion manifold displacement field. (x,y).

[0045] Step 307: Use the distorted manifold displacement field as the initial quantity of the spatial mapping perturbation, setting the initial quantity of the spatial mapping perturbation to be equal to the displacement vector of each position point in the distorted manifold displacement field; substitute the initial quantity of the spatial mapping perturbation into the iterative optimization process, calculate the spatial mapping error under the current perturbation in each iteration, and update the perturbation quantity according to the direction of the error gradient if the spatial mapping error is greater than a preset threshold; repeat the iteration until the change in the perturbation quantity is less than the preset threshold, and obtain the converged steady-state perturbation correction vector; superimpose the steady-state perturbation correction vector onto the coordinates of each pixel point in the spatial configuration, so that the spatial distance and direction relationship between adjacent pixels in the spatial configuration conforms to the distortion-free state, and obtain the steady-state configuration, specifically including: reconstructing the distorted manifold displacement field obtained by the complex function coaxial integral in step 306. (x, y) are directly assigned the initial values ​​of the space mapping perturbation. (x, y) is used to make the initial perturbation distribution completely consistent with the actual image distortion distribution. The assignment formula is: The spatial mapping distortion correction problem is constructed as an energy functional extremum problem with respect to the perturbation vector V(x,y), and the energy functional expression is: In the formula, Let V represent the extremum energy functional to be solved with the perturbation vector V as the functional variable; In the state-phase domain The double integral operator on; This represents the state-phase domain region corresponding to distortion correction; Indicates the current disturbance vector Under the influence of this action, the actual spatial coordinates obtained by pixel mapping; Represents the ideal spatial coordinates of a pixel in a distortion-free state; This represents the squared error term between the current mapped coordinates and the ideal coordinates, used to characterize the distortion correction deviation; Represents a two-dimensional plane integral infinitesimal element; This represents the smoothing regularization coefficient, which is set to a value of [value missing] in this embodiment. This is used to balance the correction accuracy and the smoothness of the displacement field; Indicates the disturbance vector The spatial gradient operator is used to characterize the spatial rate of change of the perturbation vector; Let represent the squared magnitude of the gradient of the perturbation vector space, used to constrain the smoothness of the perturbation vector; by performing variational differentiation on the above energy functional, the Euler-Lagrange extremum condition is obtained, expressed as follows: In the formula Represents the energy functional For the disturbance vector The variational derivative; 2 represents the fixed coefficients obtained from the variational operation; Indicates the pixel mapping coordinates under the current perturbation; Represents the ideal, distortion-free coordinates of a pixel; Represents the smoothing regularization coefficient; Represents the two-dimensional Laplace operator; Indicates the disturbance vector The result after Laplace operation; based on the variational extremum condition, a functional extremum variational iterative scheme is constructed, and the perturbation vector update rule for the k-th iteration is as follows:

[0046] In the formula, This represents the perturbation vector after the (k+1)th iteration update; This represents the perturbation vector in the k-th iteration; This represents the iteration step size coefficient, which is set to 0.05 in this embodiment. It is used to control the iteration convergence speed and stability. Represents the ideal, distortion-free coordinates of a pixel; This represents the pixel mapping coordinates obtained during the k-th iteration; Represents the smoothing regularization coefficient; Denotes the perturbation vector of the k-th iteration. The result after Laplace operation; preset iteration convergence threshold. The iteration termination condition is as follows:

[0047] In the formula, This represents a double integral over the state-phase domain; This represents the absolute difference between two consecutive iterations of the perturbation vector; This represents the iteration convergence threshold, which is set to a value in this embodiment. When the iteration termination condition is met, the iteration process converges, and the converged steady-state perturbation correction vector is output. The steady-state disturbance correction vector is then superimposed point by point onto the three-dimensional spatial coordinates of each pixel in the spatial configuration constructed in step 203 to obtain the corrected three-dimensional coordinates, expressed as: In the formula This represents the pixel's three-dimensional spatial coordinates after correction by the steady-state perturbation correction vector; This represents the original three-dimensional spatial coordinates of the pixel before correction; This represents the steady-state perturbation correction vector obtained through iterative convergence. The corrected spatial coordinates ensure that the spatial distance and orientation relationship between adjacent pixels fully conforms to the ideal distortion-free physical rules, eliminating the positional offset and geometric distortion caused by distortion. The final stable distortion-free spatial structure is the steady-state configuration.

[0048] In this embodiment, by calculating the shape constraint coefficient and weighted displacement value, the distortion calculation is made to better fit the local features, thereby improving the accuracy of distortion characterization.

[0049] In a preferred embodiment of the present invention, step 4, which involves performing a mapping transformation on each pixel in the original panoramic image according to the steady-state configuration to obtain a first corrected image; and performing radial distortion correction on each pixel in the first corrected image according to the first corrected image and the radial distortion component to obtain a radially corrected image, may include:

[0050] Step 401: Based on the spatial coordinates of all pixels in the steady-state configuration, extract the pixels located at the edges of the steady-state configuration, fit the spatial coordinates of the edge pixels to a plane, and define the plane as the distortion-free image plane. Specifically, this includes: based on the three-dimensional spatial coordinates of all pixels in the steady-state configuration, traverse all coordinate points and extract the edge pixels located on the outermost contour; let the three-dimensional coordinates of the edge pixels be: ,in These are the three-dimensional spatial coordinates of the i-th edge pixel. Let be the horizontal, vertical, and depth coordinates of the i-th edge pixel in three-dimensional space, all in millimeters; The total number of edge pixels; using the least squares method for plane fitting, the plane equation is: Using the least squares criterion, the objective function is to minimize the sum of squared distances from all edge points to the plane. By solving the above least squares problem, the plane parameters a are obtained. The plane determined by this is the distortion-free image plane.

[0051] Step 402: Construct a sphere with the spatial coordinates of each pixel in the stable configuration as its center and a preset length as its radius; define a quad bounding box on the distortion-free image plane, the quad bounding box being enclosed by four boundary lines, specifically including: using the three-dimensional coordinates of each pixel in the stable configuration as its center. Construct a sphere with a center at a predetermined length R and a radius of 1; the equation of the sphere is: Preset radius Determined based on the ratio of the maximum image height of the fisheye lens to the image resolution: = In the formula, K is a scaling factor, ranging from 2 to 5; on the distortion-free image plane, the four-sided bounding box is defined according to the total number of horizontal pixels W and the total number of vertical pixels H, and the equations of the four boundary lines are: ,in These correspond to the four boundary lines on the left, right, bottom, and top of the bounding box, respectively.

[0052] Step 403: Traverse the sphere corresponding to each original pixel point, calculate the distance from the center of the sphere to the line containing each boundary line. If the distance is greater than the radius of the sphere, it is determined that the sphere does not overlap with the boundary line. If the distance from the center of the sphere to all boundary lines is not greater than the radius of the sphere, or the center of the sphere is located within the corner area of ​​the bounding box, it is determined that the sphere overlaps with the bounding box. When an overlap is determined, record all candidate positions within the bounding box that overlap with the sphere. Specifically, this includes: traversing the spheres constructed for each pixel point in step 402, and for each sphere, calculating the perpendicular distance from the center of the sphere to the four boundary lines of the four-sided bounding box. The specific calculation method is: calculate the perpendicular distance from the center of the sphere to the left boundary line of the bounding box, the perpendicular distance from the center of the sphere to the right boundary line of the bounding box, and the perpendicular distance from the center of the sphere to the bottom boundary line of the bounding box. The four perpendicular distances from the center of the sphere to the upper boundary line of the bounding box are calculated precisely using the perpendicular distance calculation formula to ensure the accuracy of the calculation results. If any of the four calculated perpendicular distances is greater than the preset radius of the sphere, it is determined that the sphere does not overlap with the corresponding boundary line, and therefore the sphere does not overlap with the entire quad bounding box. If all four perpendicular distances are less than or equal to the preset radius of the sphere, or if the center of the sphere is located within the area of ​​the four corner points of the quad bounding box, it is determined that the sphere and the quad bounding box have a spatial overlap relationship. When it is determined that the sphere and the quad bounding box have an overlap relationship, all positions that have spatial overlap with the sphere within the effective range of the quad bounding box are searched point by point, and all these overlapping positions are taken as candidate positions.

[0053] Step 404: For each candidate position, calculate the intersection region between the distortion-free image plane and the sphere. The intersection region is a circular plane. Take the coordinates of the center point of the circular plane as the mapping coordinates from the current original pixel to the distortion-free image pixel. After calculating the mapping coordinates for all original pixels, a complete mapping relationship from each pixel position in the original panoramic image to the coordinates of the distortion-free image plane is obtained. Resample each pixel in the original panoramic image according to the mapping relationship to obtain the first corrected image. Specifically, this includes: for each candidate position... Calculate the intersection region between the plane and the sphere in the distortion-free image; solve the equations of the sphere and the plane simultaneously: The circular planar region formed by the intersection of the two is obtained by solving the problem. The coordinates of the center point of this circular plane are taken as the mapping coordinates from the current original pixel to the distortion-free image plane. ( ) ,in These are the mapped coordinates of the current pixel. The sum of the horizontal and vertical coordinates of all candidate positions; after calculating the mapping coordinates of all original pixels, a complete mapping relationship is established from the original image coordinates to the plane coordinates of the distortion-free image; according to this mapping relationship, the original panoramic image is resampled pixel by pixel, and the grayscale and color information of the original pixels are transferred to the corresponding mapping coordinates to obtain the first corrected image.

[0054] Step 405: Extract the radial distortion component from the distortion parameters. The radial distortion component includes a first-order radial distortion coefficient and a second-order radial distortion coefficient. For each pixel in the first corrected image, calculate the radial distance from the pixel to the image center. Calculate the radial distortion offset based on the radial distortion coefficient and the radial distance. Move the current coordinates of the pixel radially inward by the radial distortion offset to obtain the radially corrected coordinates. Specifically, this includes: extracting the radial distortion component from the distortion parameters, including the first-order radial distortion coefficient. and second-order radial distortion coefficient For each pixel in the first corrected image, with the geometric center of the image as the reference point... Using the origin as the starting point, calculate the radial distance: The radial distortion offset is calculated based on the radial distortion model: ; Move the current coordinates of the pixel along the radial direction pointing to the image center by the corresponding offset to obtain the radially corrected coordinates: ,in , These are the radially corrected x and y coordinates; , It is the radial unit direction vector.

[0055] Step 406: Resample the first corrected image based on the radially corrected coordinates to obtain a radially corrected image. Specifically, this includes: using the radially corrected coordinates (X... rc ,Y rc Based on the positioning, the first corrected image is resampled point by point; for positions with non-integer coordinates, bilinear interpolation is used to calculate the pixel gray value: I(X,Y)=(1-t)(1-s)I 11 +(1-t)sI 12 +t(1-s)I 21 +tsI 22 Where I(X,Y) is the grayscale value obtained after interpolation; I 11 ,I 12 ,I 21 ,I 22 t and s are the gray values ​​of the four vertices in the interpolation neighborhood; t and s are the interpolation weight coefficients, both ranging from 0 to 1; after completing the full image resampling in the above manner, the radially corrected image is obtained.

[0056] This embodiment is based on fitting a distortion-free image plane with a three-dimensional steady-state configuration. It uses a sphere intersection mapping method to establish a one-to-one correspondence between the original pixels and the ideal plane, which can completely restore the distortion distribution characteristics of the fisheye lens in the center, transition and edge regions. The corrected image is free from stretching, distortion and local deformation, and the pixel position mapping relationship is stable and reliable.

[0057] In a preferred embodiment of the present invention, step 5, which involves performing tangential distortion correction on the radially corrected image based on the radially corrected image and the tangential distortion component to obtain a tangentially corrected image, and then performing edge brightness and sharpness equalization processing on the tangentially corrected image to obtain a target corrected image, may include:

[0058] Step 501: Extract the tangential distortion component from the distortion parameters. The tangential distortion component includes a first tangential distortion coefficient and a second tangential distortion coefficient. For each pixel in the radially corrected image, calculate the horizontal and vertical offsets of the tangential distortion based on the pixel's horizontal and vertical coordinates, the first tangential distortion coefficient, and the second tangential distortion coefficient. Specifically, this includes: extracting the tangential distortion component from the distortion parameters, including the first tangential distortion coefficient. Second tangential distortion coefficient For each pixel in the radially corrected image, calculate the tangential distortion offset in the horizontal and vertical directions: ,in Current coordinates of the pixel This represents the radial distance from a pixel to the center of the image.

[0059] Step 502: Shift the x-coordinate of each pixel horizontally by a horizontal offset to obtain the tangentially corrected x-coordinate; shift the y-coordinate of each pixel vertically by a vertical offset to obtain the tangentially corrected y-coordinate; resample the radially corrected image based on the tangentially corrected x-coordinates and tangentially corrected y-coordinates of all pixels to obtain the tangentially corrected image. Specifically, this includes: superimposing the corresponding tangential offset onto the original coordinates of each pixel to obtain the tangentially corrected coordinates. ,in , The horizontal and vertical coordinates are the results of tangential correction. Using the corrected coordinates of all pixels as the positioning reference, a pixel-by-pixel resampling operation is performed on the radially corrected image. Specifically, each pixel in the radially corrected image is traversed, and its grayscale value and color information are assigned to a new image array according to the tangentially corrected coordinates. For positions with non-integer corrected coordinates, bilinear interpolation is used to select four neighboring integer coordinate pixels around the non-integer coordinate. The pixel value corresponding to the position is obtained by weighting the grayscale and color information of the neighboring pixels, ensuring smooth image edges without jagged edges or breaks. After completing the information assignment and interpolation calculation for all pixels in the above manner, the tangentially corrected image with tangential distortion correction is finally obtained.

[0060] Step 503: Calculate the local average brightness of each pixel in the tangentially corrected image; divide the original brightness value of each pixel by the local average brightness to obtain a brightness adjustment coefficient; scale the original brightness value of each pixel according to the brightness adjustment coefficient to obtain a brightness-equalized image. Specifically, this includes: selecting a 5×5 local neighborhood centered on each pixel in the tangentially corrected image, and calculating the average brightness of all pixels within that neighborhood. ,in It is the average brightness of the local neighborhood corresponding to the current pixel; The relative coordinates within the neighborhood are ( The pixel brightness value; The values ​​are -2, -1, 0, 1, and 2; calculate the brightness adjustment coefficient: ,in ' is the global average brightness of the entire image. The original brightness value of the current pixel is scaled using a brightness adjustment factor: ,in The pixel brightness value after brightness equalization; The original brightness value of the corresponding pixel in the tangential correction image is used; after the brightness adjustment is completed point by point, the brightness-equalized image is obtained.

[0061] Step 504: Calculate the grayscale gradient value between each pixel and its surrounding neighboring pixels in the brightness-equalized image; and calculate the average grayscale gradient of all pixels in the entire image; for local areas where the grayscale gradient is lower than the average, increase the grayscale difference between adjacent pixels in the local area; for local areas where the grayscale gradient is higher than the average, decrease the grayscale difference between adjacent pixels in the local area, thus obtaining the target corrected image. Specifically, this includes: calculating the grayscale gradient value between each pixel and its 8 neighboring pixels in the brightness-equalized image. ,in, This represents the grayscale gradient value of the current pixel. This refers to the horizontal grayscale gradient component; The vertical grayscale gradient component is represented by the grayscale gradient of all pixels in the entire image, and the overall average value is calculated. ,in The image contains the total number of pixels horizontally and vertically; the image is optimized according to its gradient distribution, specifically for grayscale gradients. In localized areas, increasing the grayscale difference between adjacent pixels enhances regional detail and clarity; for grayscale gradients... In local areas, the grayscale difference between adjacent pixels is reduced to achieve local smoothing and noise reduction; after the whole image optimization process is completed, a target corrected image with complete geometric distortion correction, uniform brightness distribution, and moderate clarity is obtained.

[0062] This embodiment further compensates for tangential distortion on the basis of geometric correction, eliminates the eccentric distortion and asymmetric offset caused by assembly and lens processing errors in the image, and makes the corrected image more geometrically accurate and visually consistent. It adopts a local neighborhood brightness equalization method to adaptively correct the uneven brightness problems such as dark corners at the edges and excessive brightness in the center of the fisheye image, and improves the overall brightness uniformity while preserving image details.

[0063] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0064] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0065] I. Overview of Experimental Conditions

[0066] This experimental example uses a panoramic aerial image of a city's building complex captured by a 245-degree ultra-wide-angle fisheye lens as the simulation object. The original panoramic image has a horizontal resolution of 4096 pixels and a vertical resolution of 3072 pixels. The fisheye lens imaging parameters include: focal length 2.8mm, aperture f / 2.0, maximum image height 5.67mm, field of view 245 degrees, principal ray angle 122.5 degrees, optical back focal length 3.5mm, total optical length 12.8mm, and effective resolution 12 megapixels. The image sensor has a horizontal dimension of 6.17mm and a vertical dimension of 4.55mm, a horizontal pixel density of 664 pixels / mm, and a vertical pixel density of 675 pixels / mm. In the distortion parameters, the radial distortion component includes a first-order radial distortion coefficient k1=-0.35 and a second-order radial distortion coefficient k2=0.12, and the tangential distortion component includes a first tangential distortion coefficient p1=0.002 and a second tangential distortion coefficient p2=-0.001. The coordinates of the three calibration points are: center calibration point (2048, 1536), edge calibration point (3580, 530), and transition calibration point (2814, 1033). The preset gradient threshold for heterogeneous morphology primitive partitioning is 0.08. The preset threshold for iterative convergence is a displacement change of less than 0.01 pixels.

[0067] II. Experimental Procedures and Results

[0068] Step 3: Using the distorted manifold displacement field as the initial quantity of the spatial mapping perturbation, perform iterative updates. For example... Figure 3 As shown, the initial maximum displacement vector amplitude is 8.0 pixels and the average displacement vector amplitude is 5.0 pixels. After 30 iterations, they converge to 0.15 pixels and 0.08 pixels respectively. The displacement change is lower than the preset convergence threshold of 0.01 pixels, thus obtaining the steady-state disturbance correction vector. The spatial configuration is corrected using the correction vector to obtain the steady-state configuration. The original panoramic image is then mapped and transformed according to the steady-state configuration to obtain the first corrected image. Radial distortion correction is performed on the first corrected image using radial distortion coefficients (k1=-0.35, k2=0.12), as follows: Figure 4 , 5 As shown, the curvature of the straight-line structure at the image edge is reduced from a significant arc (maximum deviation 15 pixels) before correction to near-straight lines (maximum deviation 2 pixels). Tangential distortion correction is performed on the radially corrected image using tangential distortion coefficients (p1=0.002, p2=-0.001).

[0069] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for distortion correction of panoramic images using fisheye lenses, characterized in that, The method includes: Step 1: Acquire the original panoramic image captured by the fisheye lens; extract the imaging parameters and distortion parameters based on the original panoramic image. Step 2: Based on the imaging parameters, establish the spatial configuration between pixels in the original panoramic image; Step 3: Set three calibration points in the original panoramic image and connect them to form a state-phase domain. Divide the state-phase domain into multiple heterogeneous topographic primitives according to the pixel density within the domain. Calculate the planar area of ​​each heterogeneous topographic primitive. Determine the topographic constraint coefficient based on the union area between the planar areas of each heterogeneous topographic primitive and the planar areas of adjacent heterogeneous topographic primitives. Calculate the distortion manifold displacement field based on the relative displacement of pixels within the heterogeneous topographic primitives and the topographic constraint coefficient. Set the initial spatial mapping perturbation based on the distortion manifold displacement field and iteratively update it until convergence to obtain the steady-state perturbation correction vector. Use the steady-state perturbation correction vector to correct the spatial configuration to obtain the steady-state configuration. Step 4: Based on the steady-state configuration, perform mapping transformation on each pixel in the original panoramic image to obtain the first corrected image; based on the first corrected image and the radial distortion component, perform radial distortion correction on each pixel in the first corrected image to obtain the radial corrected image. Step 5: Based on the radially corrected image and the tangential distortion component, perform tangential distortion correction on the radially corrected image to obtain the tangentially corrected image; perform edge brightness and sharpness equalization processing on the tangentially corrected image to obtain the target corrected image.

2. The distortion correction method for panoramic images using fisheye lenses according to claim 1, characterized in that, The imaging parameters include focal length, aperture, maximum image height, field of view, principal ray angle, optical back focal length, total optical length, and resolution. The distortion parameters include radial distortion components and tangential distortion components.

3. The distortion correction method for panoramic images using fisheye lenses according to claim 2, characterized in that, Based on the imaging parameters, the spatial configuration between pixels in the original panoramic image is established, including: Based on the focal length and field of view, the image coordinates of each pixel are converted into the corresponding incident light angle to obtain the angle distribution data; Based on the maximum image height and the principal ray angle, image plane projection mapping is performed on each incident angle in the angle distribution data to obtain the initial coordinate set of all pixels. Based on the optical back focal length and total optical length, the initial coordinate set is spatially depth-corrected, and the spatial distance and directional relationship between adjacent pixels are calculated point by point to establish the spatial configuration between pixels.

4. The distortion correction method for panoramic images using fisheye lenses according to claim 3, characterized in that, Three calibration points are set in the original panoramic image and connected to form a state-phase domain. The state-phase domain is divided into multiple heterogeneous morphological primitives according to the pixel density within the state-phase domain. Calculate the planar area of ​​each heterogeneous morphological unit, and determine the morphological constraint coefficient based on the union area between the planar areas of each heterogeneous morphological unit and the planar areas of adjacent heterogeneous morphological units, including: Three calibration points are selected in the original panoramic image. The first calibration point is located in the central region of the image, the second calibration point is located in the edge region of the image, and the third calibration point is located in the transition region between the central region and the edge region. Connect the three calibration points in pairs, and define the region enclosed by the three calibration points and the connecting lines as the state-phase domain; calculate the density between each pixel and its surrounding neighboring pixels in the state-phase domain to obtain the pixel density. The state-phase domain is divided into multiple sub-regions according to the pixel density from high to low, and each sub-region is defined as a heterogeneous morphology primitive. Calculate the projected area of ​​each heterogeneous shape primitive on the image plane. For any pair of adjacent heterogeneous shape primitives, calculate the union area of ​​the plane areas of the two heterogeneous shape primitives. Divide the union area by the sum of the areas of the two heterogeneous shape primitives to obtain the shape constraint coefficient.

5. The distortion correction method for panoramic images using fisheye lenses according to claim 4, characterized in that, The distorted manifold displacement field is calculated based on the relative displacement of pixels within the heterogeneous morphology primitive and the morphology constraint coefficient; the initial spatial mapping perturbation is set based on the distorted manifold displacement field and iteratively updated until convergence is performed to obtain the steady-state perturbation correction vector. The spatial configuration is corrected using the steady-state perturbation correction vector to obtain the steady-state configuration, including: Within each heterogeneous topography primitive, the relative displacement of each pixel relative to its surrounding neighboring pixels is extracted; the relative displacement is weighted according to the topography constraint coefficient to obtain the weighted displacement value of each pixel. The weighted displacement values ​​of all pixels within the same heterogeneous topography primitive are summed, and the average value is calculated based on the total number of pixels within the primitive to obtain the average distortion contribution. The average distortion contribution of each primitive is synthesized according to the spatial position of all heterogeneous topography primitives in the state-phase domain to obtain the distortion manifold displacement field. The distorted manifold displacement field is used as the initial quantity of the spatial mapping perturbation. The initial quantity of the spatial mapping perturbation is set to be equal to the displacement vector of each position point in the distorted manifold displacement field. The initial quantity of the spatial mapping perturbation is substituted into the iterative optimization process. In each iteration, the spatial mapping error under the current perturbation is calculated. If the spatial mapping error is greater than a preset threshold, the perturbation quantity is updated according to the direction of the error gradient. The iteration is repeated until the change in the perturbation quantity is less than the preset threshold, and the converged steady-state perturbation correction vector is obtained. The steady-state perturbation correction vector is superimposed on the coordinates of each pixel point in the spatial configuration so that the spatial distance and direction relationship between adjacent pixels in the spatial configuration conforms to the distortion-free state, and the steady-state configuration is obtained.

6. The distortion correction method for panoramic images using fisheye lenses according to claim 5, characterized in that, Based on the steady-state configuration, a mapping transformation is performed on each pixel in the original panoramic image to obtain the first corrected image, including: Based on the spatial coordinates of all pixels in the steady-state configuration, the pixels located at the edge of the steady-state configuration are extracted, and the spatial coordinates of the edge pixels are fitted into a plane, which is defined as the distortion-free image plane. Construct a sphere with the spatial coordinates of each pixel in the steady-state configuration as the center and a preset length as the radius; define a quad bounding box on the distortion-free image plane, which is enclosed by four boundary lines; Iterate through the sphere corresponding to each original pixel, calculate the distance from the center of the sphere to the line containing each boundary line. If the distance is greater than the radius of the sphere, it is determined that the sphere does not overlap with the boundary line. If the distance from the center of the sphere to all boundary lines is not greater than the radius of the sphere, or the center of the sphere is located in the corner area of ​​the bounding box, it is determined that the sphere overlaps with the bounding box. When it is determined that there is an overlap, record all candidate positions within the bounding box that overlap with the sphere. For each candidate location, the intersection region between the distortion-free image plane and the sphere is calculated. The intersection region is a circular plane, and the coordinates of the center point of the circular plane are taken as the mapping coordinates from the current original pixel to the distortion-free image pixel. After calculating the mapping coordinates for all original pixels, a complete mapping relationship from each pixel position in the original panoramic image to the coordinates of the distortion-free image plane is obtained. Each pixel in the original panoramic image is resampled according to the mapping relationship to obtain the first corrected image.

7. The distortion correction method for panoramic images using fisheye lenses according to claim 6, characterized in that, Based on the first corrected image and the radial distortion component, radial distortion correction is performed on each pixel in the first corrected image to obtain a radially corrected image, including: Radial distortion components are extracted from the distortion parameters. The radial distortion components include first-order radial distortion coefficients and second-order radial distortion coefficients. For each pixel in the first corrected image, the radial distance from the pixel to the image center is calculated. The radial distortion offset is calculated based on the radial distortion coefficients and the radial distance. The current coordinates of the pixel are moved radially inward by the radial distortion offset to obtain the radially corrected coordinates. Based on the radially corrected coordinates, the first corrected image is resampled to obtain the radially corrected image.

8. The distortion correction method for panoramic images using fisheye lenses according to claim 7, characterized in that, Based on the radially corrected image and the tangential distortion component, tangential distortion correction is performed on the radially corrected image to obtain the tangentially corrected image; the edge brightness and sharpness are then balanced on the tangentially corrected image to obtain the target corrected image, including: The tangential distortion component is extracted from the distortion parameters. The tangential distortion component includes a first tangential distortion coefficient and a second tangential distortion coefficient. For each pixel in the radially corrected image, the offset of the tangential distortion in the horizontal direction and the offset in the vertical direction are calculated based on the horizontal and vertical coordinates of the pixel and the first and second tangential distortion coefficients. The x-coordinate of each pixel is shifted horizontally by an offset to obtain the tangentially corrected x-coordinate; the y-coordinate of each pixel is shifted vertically by an offset to obtain the tangentially corrected y-coordinate; based on the tangentially corrected x-coordinates and tangentially corrected y-coordinates of all pixels, the radially corrected image is resampled to obtain the tangentially corrected image. Calculate the local average brightness of each pixel in the tangentially corrected image; divide the original brightness value of each pixel by the local average brightness to obtain the brightness adjustment coefficient; scale the original brightness value of each pixel according to the brightness adjustment coefficient to obtain the brightness-equalized image; Calculate the grayscale gradient value between each pixel and its surrounding neighboring pixels in the brightness-equalized image; calculate the average grayscale gradient of all pixels in the entire image; for local areas where the grayscale gradient is lower than the average value, increase the grayscale difference between adjacent pixels in the local area; for local areas where the grayscale gradient is higher than the average value, decrease the grayscale difference between adjacent pixels in the local area, thus obtaining the target corrected image.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images