A geometric correction method for aerial dome images based on flight attitude data
By establishing a reference mapping and weight matrix in the production of aerial dome images, and synchronously updating attitude parameters and local deformations, the problem of mutual masking between attitude parameter calibration and local mesh deformation compensation is solved, reducing projection errors and redundant corrections.
Patent Information
- Application Number
- CN202611519790.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-29
- Publication Date
- 2026-11-17
AI Technical Summary
In existing technologies for producing aerial dome images, attitude parameter calibration and compensation for local mesh deformation mutually mask each other, leading to redundant compensation and projection errors, and making it impossible to effectively update attitude parameters and local deformation constraints synchronously.
By establishing a reference mapping from the original pixels to the spherical coordinates, and using the derivative of the inverse spherical mapping and measurement error to form a weight matrix, the response matrix of the residual to the time and orientation parameters to be estimated is calculated. Combined with the total local deformation correction of the local mesh, the weighted orthogonal equation is satisfied, and the attitude parameters and local deformation are updated synchronously.
It reduces orientation errors caused by time mismatch and single-frame attitude approximation, limits repeated correction of local meshes, ensures clear compensation division between attitude parameters and local deformation, and reduces the masking and repeated correction of attitude errors.
Smart Images

Figure CN123118418A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image geometry processing and dome projection technology, and in particular to a method for geometric correction of aerial dome projection images based on flight attitude data. Background Technology
[0002] Aerial dome image production requires converting wide-angle or fisheye images captured by the flight platform into source material suitable for dome display. Changes in the flight platform's pitch, roll, and yaw will alter the camera's shooting direction; when the camera is mounted on a gimbal, its actual attitude is also affected by the gimbal's rotation. For cameras using a rolling shutter, different lines within the same frame have different exposure times, and the time deviation between attitude data and image data will further cause position-dependent projection errors. Therefore, dome image correction not only needs to determine the target viewing direction but also needs to ensure that pixel exposure times, actual camera attitude, and dome mapping correspond to each other.
[0003] Existing processing methods can utilize attitude data to establish geometric mappings and reduce residual alignment errors through local mesh deformation. However, when attitude parameter calibration and local meshing are performed separately, the same residual may be interpreted by both orientation parameter changes and mesh displacement. As a result, the local mesh may absorb uncorrected attitude errors, causing pixel alignment results to mask inaccuracies in attitude parameters. When recalibrating attitude parameters subsequently, continuing to use the existing mesh may cause redundant compensation, manifesting as local contour swaying, unnecessary scaling, or changes in orientation reference.
[0004] Furthermore, the pixel displacements at different positions of the dome screen source do not have entirely the same angular meaning. Directly using a uniform pixel distance to compare attitude compensation and local compensation may cause the error distribution between the two to lose its consistent geometric basis. Therefore, it is necessary to establish clear compensation constraints for the residuals involving both attitude parameters and local deformation, and to update these constraints and local deformations synchronously after parameter changes. Summary of the Invention
[0005] The purpose of this invention is to provide a geometric correction method for aerial dome images based on flight attitude data, so as to solve the problems of mutual masking between attitude parameter calibration and compensation for the same residual by local mesh deformation, and repeated compensation caused by using existing local deformation after attitude update.
[0006] To achieve the above objectives, this invention provides a geometric correction method for aerial dome images based on flight attitude data, comprising: establishing a reference mapping from the original pixels to the dome coordinates based on the line-by-line exposure time, the fuselage attitude, and the relative attitude of the camera; establishing a reference direction based on the camera optical center at the corresponding exposure time and static control points determined independently of the current correction segment; calculating the reprojection residual between the reference direction and the reference mapping on the same set of control points; establishing a weight matrix using the derivative of the dome inverse mapping and the measurement error; forming a response matrix based on the response of the residual to the increments of the time and direction parameters to be estimated; representing the total local deformation with the correction of all vertices of the local mesh, such that the interpolation correction at the control points and the response matrix satisfy a weighted orthogonal equation; reconstructing the exposure time, camera optical center, reference mapping, reference direction, residual, weight matrix, and response matrix for each candidate parameter, and resolving the total local deformation under the new weighted orthogonal constraints; adopting the candidate parameters and the total local deformation only when the residual target, mesh geometry, and the angle error of the verification points separated by the control point identity all meet the preset acceptance conditions, and outputting the dome image by resampling the original image once.
[0007] In some implementations, the pixel exposure time is determined based on the first line exposure start time, residual time deviation, normalized line number, whole frame readout time, and single line exposure duration; the normalized line number is determined based on the actual readout direction, and the parameters to be estimated include residual time deviation and direction deviation, while the whole frame readout time uses an independent calibration value; the camera unit line of sight is converted into a unit direction in the spherical coordinate system through camera mounting rotation, gimbal rotation, camera body attitude, direction deviation rotation, and target spherical screen rotation.
[0008] In some implementations, static control points are provided by a georegistration model that is independently measured or established and pre-frozen from mapping data outside the current correction segment; the reference direction of the control point is determined by the camera optical center position at the corresponding pixel exposure time, and the optical center position and reference direction are recalculated synchronously when the parameters to be estimated change, so as to distinguish the actual camera position change from the attitude error correction.
[0009] In some implementations, pixel displacement is converted into unit direction increments by the derivative of the inverse spherical mapping, and a spherical error metric is formed by combining the input measurement error and the matching confidence. The same metric and control point set are used for attitude update and local deformation solution. When the metric or set changes, the response matrix and deformation constraints are reconstructed synchronously.
[0010] In some implementations, the parameters to be estimated are scaled and normalized, and the response matrix of the complete residual link is calculated through parameter perturbation; the parameters that can be updated are determined based on the singular values of the weighted response matrix and the parameter uncertainty, and the time deviation is frozen when the time deviation and the orientation deviation cannot be reliably distinguished; for attitude response modes that can be calculated but are not allowed to be updated, the corresponding local deformation constraints are still maintained.
[0011] In some implementations, a triangular mesh is created on the original image, and the local correction at the control point is obtained by centroid interpolation. The total local deformation is solved under weighted residual, spatial smoothing and amplitude regularization constraints. The weighted orthogonal constraint between the total local deformation and the attitude response space is established for all current deformations to prevent historical deformations from retaining compensations that can be explained by attitude parameters.
[0012] In some implementations, the same static verification control point in adjacent frames is used as the object, and the reference direction is calculated based on the real optical center of each frame. The error of the final mapping relative to the respective reference direction is compared in the same world coordinate system. When there is no reliable independent static control point, the cross-frame check is not performed, and the deformation degrees of freedom of the unobserved area are restricted.
[0013] In some implementations, for each candidate parameter step size, the baseline mapping, reference direction, residual, weight matrix, response matrix, and deformation constraint are regenerated, and the total local deformation is resolved. In the same candidate step size comparison, the solution point, evaluation weight, and regularization matrix are fixed. If the solution point exceeds the effective calibration domain, geometric feasibility, or the fixed evaluation target is not met, the candidate parameter and candidate mesh are rejected, the step size is reduced, and the solution is resolved.
[0014] In some implementations, verification data, which is separated from the solution data by control point identity, is used to check the spherical angle error, and the grid orientation, local scaling, non-adjacent triangle overlap, coverage, and weighted orthogonality error of the reference mapping and the final mapping are checked. The parameters and grids that pass the check are recorded as the same acceptance version, and the composite mapping is directly applied to the original image. After one image resampling, the dome image and effective pixel mask are output.
[0015] Compared with the prior art, the present invention has the following beneficial effects: synthesizing the flight attitude, gimbal attitude and camera installation relationship according to the actual exposure time can reduce the direction error caused by time mismatch and single-frame attitude approximation; using a local mesh subject to spatial smoothing and amplitude constraints can reduce the residual projection error unexplained by the attitude model and limit local folding and excessive stretching; furthermore, the attitude parameter update and the total local deformation share the same response matrix and spherical error metric, so that the two have a clear division of compensation in the observable response space, and the total local deformation is re-solved when the parameters change, thereby reducing the masking of attitude error by the local mesh and the repeated correction caused by the superposition of historical compensation by the new attitude mapping. Attached Figure Description
[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention. Figure 2 This is a schematic diagram illustrating the correspondence between line-by-line exposure times and flight attitude in this invention; Figure 3 This is a schematic diagram of the attitude transformation chain from the original pixel to the spherical screen coordinates in this invention; Figure 4 A schematic diagram showing the static control point, camera optical center, and reference direction of this invention; Figure 5 This is a schematic diagram illustrating the establishment of the spherical error measurement and attitude response matrix of the present invention; Figure 6 This is a schematic diagram of the weighted orthogonal constraint between local deformation and attitude response of the triangular mesh in this invention; Figure 7 This is a schematic diagram illustrating the synchronous recalculation of candidate attitude parameters and total local deformation in this invention. Figure 8 This is a schematic diagram illustrating the joint acceptance logic of parameters and mesh in this invention; Figure 9 This is a schematic diagram of the final mapping composite and one resampling to generate the dome image in this invention. Detailed Implementation
[0018] Exemplary embodiments of the present invention will be described below. Features in the various embodiments can be combined without contradiction; the exemplary values are used to illustrate an executable configuration, and the specific values can be predetermined according to camera calibration accuracy, image resolution, and dome screen output requirements.
[0019] See Figure 1-9 This embodiment provides a geometric correction method for aerial dome images based on flight attitude data. The method is executed by an image processing device with memory and processor. The processing object is aerial images that retain the original exposure sequence, and the output is a dome image source and an effective pixel mask. The processor can perform correction offline after the aerial photography is completed, or it can perform segmented processing after obtaining continuous image segments that meet the calculation conditions.
[0020] This embodiment takes a fisheye camera covering the target hemispherical field of view as an example. The processing in the method is divided into a line-by-line attitude mapping and residual response calculation step group, and a local geometric correction step group constrained by the response. The former step group provides the reference mapping, reprojection residual, response matrix and spherical error metric. The latter step group uses the above-mentioned shared data to limit local deformation and participates in the adoption or rejection of parameter updates through the feasibility check of candidate mappings. The two step groups are synchronously recalculated by the same parameter update.
[0021] Here, the dome source is an angled image organized according to a predetermined viewing direction, and the dome coordinate correction is distinguished from the installation calibration of the projector; this embodiment does not require converting all objects in the image to a new common shooting center, nor does it consider translational parallax when lacking depth, unshot areas, or severe motion blur as content that can be recovered solely through geometric mapping.
[0022] In the first step, the processor receives the original aerial frame, the first line exposure start time, the readout direction, the full frame readout time, and the single line exposure duration, and receives the timestamped body attitude, gimbal attitude, and position data. When the camera is rigidly mounted on the body, fixed mounting rotation is used instead of the combination of gimbal rotation and camera mounting rotation. When the camera is mounted on a rotatable gimbal, the synchronously recorded gimbal angle or attitude sensor data rigidly connected to the camera is used. The body attitude cannot be directly regarded as the camera attitude.
[0023] Before implementation, the camera lens, camera mounting relationship, and optical center arm are calibrated; the mapping from the original camera pixels to the unit line of sight in the camera coordinate system is denoted as... ,in These are the original pixel coordinates. For column coordinates, Row coordinates, superscript This indicates transpose, with pixel coordinates in pixels.
[0024] For example, using a radial fisheye model that has been affinely normalized in two dimensions, the calculation of the pixel to the unit line of sight is as follows: in, It is a reversible second-order affine matrix for the camera, used to compensate for pixel aspect ratio and skew. Let the principal point coordinates of the camera be... For normalized image plane coordinates, For its two components, For the normalized radius, Let be the angle between the incident ray and the camera's optical axis. For image plane azimuth, For radial calibration coefficients, Describing the Euclidean norm, This represents the two-parameter arctangent function. During calibration, calibration board images at different distances and orientations are used to perform least-squares fitting on the above parameters, requiring the radius function to be strictly monotonic within the effective field of view. During runtime, the bisection method is used to solve the problem within the calibration angle range. , Take the unit line of sight directly .
[0025] For lenses that cannot be fully described by the radial model described above, a pre-calibrated pixel-to-unit line-of-sight lookup table can be used. Each lookup table entry is established by the known spatial direction and the corresponding original pixel, and the line of sight at non-integer pixels is obtained by local interpolation. Pixels outside the calibration range do not participate in calibration or effective output. The above calculation directly uses the original pixel coordinates and does not generate a resampled distortion-free image first.
[0026] Let the world coordinate system, fuselage coordinate system, gimbal coordinate system, camera coordinate system, and dome screen coordinate system be respectively... Represents the rotation matrix. The unified meaning is to define the coordinate system Vector transformation to coordinate system All coordinate systems use the right-hand rule; this subscript convention only indicates the direction of coordinate transformation, not the method / step group.
[0027] The image clock is pre-converted to the time reference of the attitude data. For example, the clock correspondence is established using the trigger pulses jointly recorded by the camera exposure trigger output and the attitude acquisition end. If there is no hardware trigger output, the offline calibration is completed using a light source with a known flashing timing and a synchronous recording signal, and the remaining synchronization error range is determined. At the same time, the whole frame readout time is calibrated according to the sensor readout mode or flashing stripes. The encapsulation timestamp of the video container is only used for subsequent calculations when its relationship with the actual exposure start point is known.
[0028] No. Frame number The exposure time represented by a row pixel is determined by the following formula: in, For frame index, This is the exposure start time for the first exposure line of the frame that has been converted to the attitude time reference. The remaining time bias to be estimated. This represents the interval between the start times of the exposure from the first exposure row to the last exposure row. The above time values are all in seconds and represent the duration of a single-line exposure. The normalized row number calculated according to the actual exposure order; the number of image rows. When the value is not less than 2, read the value from top to bottom. Read from bottom to top ,in Let the row coordinates be zero-based; if there is only one row, simply set... Global shutter capture .
[0029] This embodiment will As an independent calibration quantifier, a set of residual time deviations and three directional deviations are estimated within the same continuous segment to avoid simultaneously opening multiple highly correlated timing parameters; for segments with changes in exposure mode, clock jumps, or installation status, they are re-segmented and processed using the corresponding calibration data, and the time relationships of previous segments cannot be continued.
[0030] The unit quaternions in the attitude data are treated to have the same sign, so that the inner product of adjacent quaternions is non-negative, and then spherical linear interpolation is used to obtain the result. The attitude at any given time; position data is linearly interpolated over time, and the sampling interval must meet the predetermined position error budget; if the exposure time falls outside the attitude or position recording range, the recording has gaps exceeding the allowable interval, or the expected projection displacement during the exposure period causes severe blurring, the corresponding observation is excluded, and the corresponding frame retains a data insufficiency indicator.
[0031] In the second step, the target viewing direction is fixed and a reference map is generated. The target viewing direction can be determined by the shooting script or by a pre-determined smooth viewing trajectory, but it remains unchanged during the parameter solution and acceptance process to avoid absorbing the direction deviation that should be estimated by changing the target viewing direction.
[0032] Define the estimated parameter vector and orientation deviation rotation of the current segment as follows: in, These are the small angular directional deviation components along the three axes of the world coordinate system, calculated in radians. To meet The third-order antisymmetric matrix, Represents any three-dimensional vector. Represents the cross product of vectors. Represents the matrix index. The rotation matrix represents the corresponding directional deviation; the directional deviation is used to compensate for the slow-changing direction calibration error of the current segment and does not represent arbitrary frame-by-frame rotation degrees of freedom.
[0033] Form a spherical coordinate system unit direction for the original pixels: in, For a moment The fuselage rotates to the world coordinate system. This refers to the rotation from the gimbal to the main body. Rotate the calibrated camera onto the gimbal. For the first A fixed rotation from the frame world coordinate system to the target spherical screen coordinate system. This refers to the unit direction of the pixel in the target dome screen coordinate system; for rigid mounting, Instead of a fixed camera, the body rotates.
[0034] Using an equidistant fisheye master covering a hemisphere as the output format, the mapping from the unit direction of the dome screen to the master coordinates is as follows: in, For any unit direction of the dome screen, The angle between this direction and the zenith axis of the dome screen. It is the azimuth angle. The two-dimensional pixel coordinates of the center of the circular master. The effective radius of the master circle. The angular to radial pixel ratio of equidistant projection. Pi For dome projection functions, A baseline mapping from the original pixels to the dome master; output preserved. direction, Direct mapping to Avoid using undefined azimuth angles.
[0035] In the third step, an independent geometric reference is established. Preferably, fixed markers, building corners, or static points in a georeferencing model with error descriptions obtained from surveying instruments are used, whose world coordinates are in the same coordinate system as the flight positioning data. The model is established from surveying data outside the current correction segment and frozen before this correction. The pixel correspondence of the current segment, the attitude parameters to be estimated, or the reverse estimation and rewriting of the control point coordinates of the current correction output are not used. If there is a common reference error between the model and the flight positioning, the common error is retained in the measurement error budget, and increasing the number of control points is not regarded as eliminating this error.
[0036] Static features in the original image are matched with reference images or texture models of control points using descriptors. This is combined with continuous frame tracking, bidirectional matching, and prior position filtering to identify correspondences. For example, candidates with a ratio of the nearest to the second nearest descriptor distance of less than 0.75 are used as initial matches. Matches with a backlash of more than one original pixel in the forward and backward tracking are then excluded. Finally, geometric consistency filtering is performed according to the prior attitude error range. The filtering range covers the allowable changes in time and orientation parameters to prevent the incorrect exclusion of valid points with real attitude deviations before calibration.
[0037] To verify the correction effect, all reliable control points are divided into a solution set and a verification set according to their status, for example, 80% and 20% respectively. All observations of the same control point in different frames can only be included in one set. Verification points are not included in parameter estimation or grid fitting to avoid using other frame observations of the same point as independent verification. It is preferable to check the coordinates of independently remeasured verification points. If the solution point and the verification point share a set of surveying benchmarks, the common benchmark error should be included in the acceptance. Absolute direction accuracy cannot be claimed solely based on the grouping results. Moving objects, unstable points caused by reflection, occluded points, and severely blurred points are not considered as control points.
[0038] The position of the camera's optical center is determined by the position and orientation of the same pixel at the exposure time. When using a mounting method with a lever arm, it can be calculated using the following formula: in, The world coordinates of the camera's optical center. The world position of the fuselage as a reference point. The fixed lever arm, located at the gimbal rotation center and expressed in the body coordinate system, is positioned relative to this positioning reference point. The fixed lever arm, which points from the center of rotation of the gimbal to the optical center of the camera, is represented in the gimbal coordinate system. Both the position and the lever arm are in meters. The above formula makes the directional deviation act on both the line of sight and the direction of the lever arm. When the independent positioning device has directly provided the position of the optical center of the camera, this position can be directly used for recording, but the value is still taken according to the actual exposure time.
[0039] No. The reference direction, reference master coordinates, and residuals for each control point observation are as follows: in, For observation index, The world coordinates of the static control point to which this observation belongs. For the frame in which it is located, To correspond to the original pixels, For reference unit direction, For reference to the master coordinates, This represents the 2D reprojection residual in pixels; the same control point can have multiple observation indices, but its world coordinates remain consistent, and the solution set will be within... The residuals of each observation are stacked sequentially as follows: dimensional vector ,in To solve for the total number of observations.
[0040] After the residual time deviation is updated, the exposure time, optical center position, and reference direction are recalculated simultaneously. When the direction deviation affects the arm, the arm direction is also recalculated using the same candidate parameters. This preserves the normal line-of-sight changes caused by the camera's actual translation, without forcing all static objects to be fixed to the master pixel position of the previous frame.
[0041] In the fourth step, a spherical error metric and response matrix shared by both step groups are generated; for the effective master coordinates... The inverse mapping of the above isometric projection is: in, The master's two-dimensional coordinates, Its displacement relative to the center of the master plate. These are the two components of the displacement. Radial pixel distance, For the corresponding angle, This is the inverse mapping from the master coordinates to the unit direction; in Time to take And use the continuous limit to calculate the derivative.
[0042] In the current parameters At the corresponding reference coordinates, the metrics and weights are generated: in, This is a 3x2 matrix of inverse mapping derivatives, which converts minute displacements of master pixels into unit direction increments. It is a second-order spherical metric matrix. For located Match confidence, The standard deviation of the angle observed is... To observe the weighted block, This is a block-diagonal weight matrix. This indicates a diagonal combination of blocks. This is the whitening matrix obtained through Cholesky decomposition of each weighted block; at the center of the master image. The continuous limit is obtained by taking the first two rows respectively. , The third row is zero, so there is no need to calculate the azimuth derivative at the center.
[0043] In this embodiment, observations that have undergone reliability screening can be uniformly selected. When using non-equal weighted confidence levels, the confidence levels are pre-defined on the independent calibration segments according to the correspondence between matching backlash and error, and remain unchanged within a linear solution cycle. Correction cannot be made by temporarily reducing the weight of large residual points.
[0044] The standard deviation of the angle is determined by the propagation of input errors such as control points, positioning, and pixel extraction; let This is a vector composed of independent measurements involved in the observation, including at least the original pixel coordinates, control point coordinates, and positioning quantities. Let its covariance matrix be... To predict the difference between the unit direction and the reference unit direction The derivative of can be taken as: in, For the operation of the largest eigenvalue of a matrix, The lower limit of angular noise is determined from the independent calibration residuals to avoid zero variance weighting; propagation is calculated at the current parameter, and the parameter to be estimated itself is not repeatedly included as known independent noise. The correlation error of the same sensor or the same control point should be included in the covariance or processed by a conservative upper bound; when the upper bound of the input error is already greater than the angular deviation to be identified, the observation is not used for fine calibration.
[0045] To compare the responses of parameters with different dimensions, scale normalization is performed with the current parameter as the center: in, For the current linearization parameters, For the dimensionless parameter increment in four dimensions, It is a diagonal scaling matrix. For time scale, In terms of angular scale, Represents a diagonal matrix. for The response matrix is a four-row matrix; in this embodiment, we take... Second, The radians correspond to 1 millisecond and 0.1 degrees, respectively. The negative sign before the response matrix matches the residual using the definition of "reference coordinates minus predicted coordinates".
[0046] The response matrix can be calculated automatically through differentiation or using central difference; when using central difference, the first... Listed as: in, Indexes for parameter columns 1 to 4, For the first Four-dimensional unit basis vectors For dimensionless difference step size, Indicates the first List; This embodiment takes and compare the adoption and The response results are only adopted when the relative difference is less than the predetermined numerical tolerance, for example, 1%; the exposure time, all rotations, optical center position, reference direction and dome projection are recalculated for each perturbation, and the derivative cannot be taken only with respect to the last two-dimensional rotation matrix.
[0047] In a linear solution, the original corresponding points, current parameters, target viewing direction, and weights are fixed, and reconstruction is performed synchronously after changes in the sample set or weights. And the deformation constraints described later; for points close to the effective hemisphere boundary, the derivative can be calculated in a slightly extended angular domain, but only points that are still within the effective output domain are used for acceptance, and points outside the lens calibration range are not extrapolated.
[0048] In the fifth step, the identifiability of the parameters is verified; singular value decomposition is performed on the weighted response matrix. in, The matrix consisting of the retained left singular vectors. This is the diagonal matrix corresponding to the singular values. The matrix is composed of the corresponding right singular vectors, with the singular values arranged in descending order; when generating the constraint basis, directions with purely numerical zeros are removed, for example, those with values less than the largest singular value are removed. The direction of the multiple is checked, and the stability of the difference is verified. The number of columns retained is denoted as... The rank threshold value is not equivalent to the parameter accuracy meeting the requirements.
[0049] When determining the parameters that can be updated, the ratio of the minimum to the maximum singular value in the weighted response matrix composed of the columns that can be updated is checked separately, for example, requiring it to be no less than Meanwhile, the parameter uncertainty is estimated based on the linear covariance of weighted least squares and the upper bound of independent measurement error. Three times the standard deviation should be lower than the preset parameter tolerance. For example, the tolerance for time parameter is 0.5 milliseconds and the tolerance for angle parameter is 0.05 degrees. Common measurement error cannot be ignored simply by increasing the number of repeated observations.
[0050] When the angular velocity in a segment is approximately constant over a long period, the time deviation may produce the same or similar response as the fixed direction deviation. The processor fixes the time deviation as a priori calibration value and only updates the direction deviation that can be reliably identified. If the accuracy condition is still not met, the priori parameters are retained. The identification test cannot be replaced by increasing the number of iterations. For attitude response modes that can be reliably established, even if their parameters are fixed or have reached the allowable boundary, the subsequent local mesh must still be constrained by the response mode to avoid the mesh taking over the prohibited attitude updates.
[0051] In the sixth step, local meshes and overall local deformations are established; triangular meshes are created within the effective field of view of each original frame, with each vertex associated with a two-dimensional master coordinate correction. The vertex corrections of all frames are arranged into a vector in a fixed order. Its dimension is ,in This represents the total number of all grid vertices involved in the solution process.
[0052] For any original pixel belonging to one of the three vertices of the source triangle, calculate three centroid coefficients based on the pixel's position in the source image. The sum of these three coefficients is 1. Multiply each coefficient by the second-order identity matrix to form an interpolation block, thereby obtaining the two-dimensional local correction. ,in For the first Frame raw pixels The interpolation matrix; stack the interpolation matrices corresponding to the solution points sequentially to form... ,but This represents the local correction for all solution points. The source triangle attribution and interpolation coefficients are not reclassified as the current spherical coordinates change.
[0053] It is preferable to start with a coarser grid, for example, dividing the source image into 32 columns and 18 rows of grid cells and dividing each cell into triangles along the diagonal, and deleting nodes outside the effective fisheye region; for connected regions without reliable control points or static tracking support, fix their local correction to zero, and use zero correction boundaries or a smooth transition to zero for regions located outside the reliable observation convex hull. Only increase the grid degrees of freedom when the observation coverage allows, to avoid fitting a small number of control points with a large number of vertices.
[0054] Spatial difference matrix It consists of the correction difference between adjacent grid vertices. For equally spaced grids, the two-dimensional correction difference between two adjacent vertices is divided by the pixel scale. ,in Pixels; for unequal-pitched grids, further multiply by the ratio of the reference grid side length to the actual source grid side length to provide a comparable penalty for the same degree of spatial variation.
[0055] Due to the movement of the camera's optical center, the same static object in adjacent frames may have different true observation directions and different local aberrations. Therefore, the grid correction amount is not directly made equal, nor is the difference between the grid correction amounts of the two frames used as the default regularization term. When verifying cross-frame consistency, the reference direction corresponding to the actual optical center of each frame is used as the basis to check the final direction error.
[0056] In the seventh step, attitude compensation and local compensation are allocated using a shared response matrix, and the linearized solution is obtained using: in, The non-negative canonical coefficients of the spatial difference are... The magnitude regularization coefficients are strictly greater than zero, and all residual terms are dimensionless after the aforementioned scaling normalization; the zero correction condition for the fixed region is solved together with the above equation.
[0057] The weighted orthogonal equation means that the correction amount generated by the total local deformation at the solution point cannot include compensation in the attitude response space under the adopted spherical error metric; its limitation is the current total correction amount. It cannot be changed to only constrain the increment relative to the previous mesh; otherwise, the attitude compensation already absorbed in the previous mesh will still be retained.
[0058] Obtained using singular value decomposition The above constraints can be equivalently implemented as follows: in, This is the local deformation constraint matrix; the equivalence relation holds for the preserved numerical effective response space, and the original form is checked after solving. The normalization error is reduced to prevent numerical truncation from causing the omission of substantial constraints.
[0059] For smaller grids, the null space orthogonal basis of all linear equality constraints can be obtained through QR decomposition or singular value decomposition. and order ,in Let the coefficient vector be the null space vector; define the regular matrix: in, To and Identity matrices of the same dimension Let be the local deformation regularization matrix; when the weighted orthogonality constraint is precisely satisfied, the intersection term of the attitude term and the local deformation term is zero. Therefore, the attitude increment can be obtained by weighted least squares with parameterized boundaries and freezing conditions, and the mesh coefficients can be obtained by the following formula: The above equation is solved using a linear equation without explicitly finding its inverse; because Unobserved degrees of freedom in the null space are suppressed; when the dimension of the null space is zero, they are directly taken. For large meshes containing multiple frames, a dense null basis with a dimension close to the total number of mesh variables is not explicitly generated; instead, vertex variables that are fixed to zero are first eliminated, and the remaining free variables and their corresponding interpolation matrices and spatial difference matrices are denoted as follows: , and form: in, A sparse positive definite matrix of free variables. Let it be its identity matrix of the same order. The vector on the right side, The weighted orthogonal constraint matrix for the free variables. This is the local deformation solution without the constraint applied. For The matrix obtained by solving for the right-hand side of multiple columns. The dimension does not exceed the dimension of the attitude-preserving response. small matrix, For multipliers of equality constraints, The solution is the free variable solution after constraints; the corrections to the fixed vertices are zeroed out to restore the total and local deformations. .because , It is positive definite; sparse Cholesky decomposition or preconditional conjugate gradient method can be used to... Solve the problem and reuse the same decomposition or preconditioner. and .when Take directly ;when When the order is not fulfilled, The singular value decomposition is used to find the minimum norm solution of the small matrix that satisfies the equation, and the orthogonality tolerance is verified as described later. Unconverged approximate results are not used in the output. When resolving the total mesh at the candidate new parameters, the attitude increment is fixed to zero and the residuals and constraints corresponding to the new parameters are substituted into the above solution.
[0060] The parameter boundaries are set based on independent calibration error settings. For example, the residual time deviation is limited to ±10 milliseconds relative to the prior value, and the deviation components in the three directions are limited to ±1 degree relative to the prior value. When the boundary is reached but there is still a significant attitude mode residual, the residual is retained and calibration data check is triggered. The mesh is not allowed to absorb the error by relaxing the orthogonal constraints.
[0061] In the eighth step, the parameter update triggers a synchronous recalculation of the two step groups; the attitude response component of the shared residual and the remaining components can be expressed as: in, For whitening residue, For the components in the attitude response space, For its orthogonal complementary components, for An identity matrix of order 1; in actual operations, it can be multiplied sequentially by 1. and No construction required A dense projection matrix of order 1.
[0062] Perform a full solution when processing the fragment for the first time; during continuous processing, you can... As a trigger statistic for unit response dimension, its threshold is predetermined by independent stable segments using the same observation selection rules and error model, for example, taking the mean plus three times the standard deviation. When there is no effective response dimension, this statistic is not calculated and the process enters the degradation stage. The local grid is still solved based on the current data of the frame to be processed. This trigger is only used to decide whether to re-estimate the attitude parameters. Any actual adopted parameter change must trigger the overall grid to be re-evaluated. The old grid cannot be reused directly because the parameter change is small.
[0063] For the The attitude increments obtained in the next iteration form candidate parameters: in, For iterative index, For the current parameter, This is the current dimensionless increment. Candidate step size coefficients, Candidate parameters; step size according to The order of reduction, exemplarily reduced to a maximum of Candidate parameters must still be within the prior allowed range.
[0064] For each candidate parameter, the exposure time, optical center position, reference mapping, reference direction, residual, weight, and response matrix are recalculated from the original data. The total local deformation is then solved under the new weighted orthogonal constraints. The previous mesh can only be used as the initial value for numerical solution. If an iterative solver is used, the initial value should be projected onto the feasible space of the new constraints first. Finally, the new solution replaces the old mesh, rather than accumulating the new mesh increments to the old mesh.
[0065] To limit linearization errors, it can be required that the changes in baseline coordinates and reference coordinates caused by control points and grid check points in a single candidate update do not exceed 2 master pixels. If they exceed this, the step size should be reduced first. For larger initial deviations, the process should be carried out step by step through multiple re-linearizations, and image correction should not be repeatedly performed on the already corrected image.
[0066] The descent judgment of candidate results uses the fixed weights and regularization matrix of the current iteration, specifically comparing the following objectives: in, For the first The fixed evaluation objective for each iteration The whitening matrix and regularization matrix are determined at the beginning of the iteration; the candidate grid is obtained using the new constraints of the candidate parameters, but the merits of each candidate in the same round are compared with the above fixed evaluation target to prevent the candidate parameters from obtaining a false target decrease by changing the weights.
[0067] In the comparison of the same candidate step size, the identity of the control point and the original pixel position remain unchanged, and each solution point is required to be located in the lens calibration domain and the definable domain of the dome inverse mapping under the current parameters and candidate parameters. If a candidate makes the solution point invalid, the candidate is rejected and the dimension of the fixed evaluation target is not changed by deleting the point. If it is necessary to change the set of solution points, the current step size comparison ends, and a new round of comparison is started after reconstructing the residuals, weights, response matrix, interpolation matrix and deformation constraints with the new set.
[0068] If the candidate objective does not decrease or the candidate geometric mapping is infeasible, both the candidate parameters and the candidate total mesh are rejected, and the full candidate calculation is performed again after reducing the step size. If all candidates fail, the current parameters and mesh are maintained, and no part of the failed candidates is retained. When updating the weights or optimizing the mesh topology, the old residual decomposition and deformation constraints are also discarded, and the solution is resolved with the new complete data state.
[0069] For example, the maximum absolute value of the standardized parameter increments adopted in two consecutive adoptions is less than And the relative decrease of the fixed target is less than The iteration ends when the maximum number of outer iterations is reached (up to 12). Reaching the maximum number of iterations only indicates the end of the calculation; subsequent joint verification is still required, and it cannot be directly considered as the completion of the correction.
[0070] In the ninth step, the parameters and mesh are jointly validated; the candidate final mapping and the angle error of the validation point are defined as follows: in, The total local deformation corresponding to the candidate parameters. For candidate final mapping, To verify the directional angle error of the point, It is an inverse cosine function. Indicates the numerical value Truncation is performed within the range of -1 to 1 to eliminate floating-point rounding errors; in this formula When taking the validation set observations, the reference optical center position of the validation point is still calculated based on the current candidate exposure time.
[0071] The root mean square (RMS) and high quantile of the angle error are used as acceptance criteria; for example, the RMS of the angle error is no greater than [value missing]. radians, 95th percentile not greater than The radian is used as a set of preset requirements, and the input reference error is required to be significantly lower than the threshold used. When the original output error is significantly higher than the noise lower limit, an improvement threshold of at least 10% reduction in candidate error can be preset. When the original error is already at the noise level, no degradation judgment is adopted, and a fixed proportion improvement is not required.
[0072] For two reliable observations of the same static control point in adjacent frames within the validation set, the cross-frame orientation error can also be calculated: in, For the first The expression of the error vector of the final unit line of sight relative to the reference unit direction of the current frame in the world coordinate system for each verification observation. Convert the vector in the spherical coordinate system to a vector in the world coordinate system. For the same static verification control point, two observation indices in adjacent frames. The Euclidean norm of the difference between the error vectors of two frames; for each frame... Each measurement was independently calculated using the optical center at the actual exposure time of the frame and the world coordinates of the same static point. Therefore, the actual line-of-sight changes caused by camera movement are already included within their respective reference directions. The error propagation from the two measurements and the independently calibrated segments were predetermined. The upper limit; this check is only performed on verification points that have a reliable three-dimensional reference, are in the valid calibration domain in both frames and are not occluded. If there is no such observation, the cross-frame result is not used to make a pass / fail judgment, nor is the grid correction amount of the same static point in adjacent frames equal to replace this check.
[0073] The acceptance threshold is predetermined based on the output resolution, project angle accuracy, and independent calibration segments. The aforementioned values only represent the available configurations. The validation set is used to determine whether to allow the current calculation result. The regularization coefficients are not repeatedly adjusted, solution points are not filtered, or mesh degrees of freedom are not increased until the result passes the validation. If the acceptance fails, the parameters and mesh are rejected and the reason for the failure is recorded.
[0074] Geometric feasibility checks include at least local orientation and scaling checks; for the three vertices of the source triangle, two second-order edge matrices are formed using the reference mapping and the final mapping, respectively. and Each matrix has two columns representing two-dimensional edge vectors of the second and third vertices relative to the first vertex, and let: in, For the local linear transformation of the final triangle relative to the reference triangle, first check that the reference-mapped triangle maintains the orientation predetermined according to the calibration coordinates relative to the source triangle, and that it has not degenerated or formed an unacceptable non-adjacent overlap. If the check fails, further subdivide the verification or mark the corresponding region as invalid. This cannot be done solely based on... Reversible and valid reference mapping; when the reference triangle is valid, it requires And check its area ratio and outliers. For example, the area ratio is required to be no less than 0.2 and both outliers are between 0.5 and 2. This indicates determinant operations.
[0075] Checking only the orientation of each triangle cannot rule out overlap between different regions. Therefore, it is also necessary to check whether the boundaries of the effective mapping region are self-intersecting and whether non-adjacent source triangles have positive area overlap within the target master. Boundary contacts caused by shared edges or shared vertices are not considered overlaps. When unnecessary coverage conflicts are found, candidates are rejected or regions that cannot be reliably mapped are excluded from the effective output domain. An overlapping triangle cannot be arbitrarily selected to cover the problem.
[0076] Effective coverage is calculated by dividing the area of the master image that can be uniquely sampled from the original effective pixels by the area of the predetermined target output region. The denominator is fixed before acceptance. For example, it is required that the coverage decrease is no more than 1 percentage point relative to the current accepted version. The initial version is compared with the minimum coverage specified by the project and must cover the preset main viewing area. It is not possible to obtain a small geometric error simply by expanding the invalid mask.
[0077] Simultaneously verify the normalized orthogonal error: in, This is the relative amount by which the total local correction projection is projected to preserve the attitude response space. To perform the maximum value operation, in the formula All candidate versions are selected, and the acceptance tolerance is, for example, taken as... If the condition is no longer met after point-by-point truncation of the grid, changing the sample weights, or changing the topology, the solution is recalculated, and the truncated grid is not directly output.
[0078] After the above checks, the parameters, total grid, control point set, weights, response basis, target viewing direction, and effective area are recorded as the same acceptance version. If there are no reliable control points, the attitude response cannot be stably calculated, or the error exceeds the model's processing range, the independent calibration parameters are retained and the pure attitude mapping is reverted to zero. The total local correction is set to zero, while the degraded state is retained. The pure attitude mapping must also pass the effective domain and mapping uniqueness check. Otherwise, an invalid mask will be output for the unqualified area, and the degraded output will not be regarded as the result of the joint calibration of attitude and local deformation.
[0079] In step ten, the dome image is generated; for the parameters and grids that passed the same acceptance test, the final mapping is written as: in, and These are the parameters that passed the acceptance test and the total local deformation, indicated by the superscript. This indicates the selected value that passed the same acceptance test. This is the final mapping for this frame; the asterisks in the reference direction and reference coordinates are only used to indicate independent geometric references.
[0080] The vertices of the source triangle are projected onto the master image through final mapping. The centroid coordinates are calculated according to the target pixels within the target triangle coverage area. The original pixel positions are then calculated back from the same centroid coordinates. Pixel values are obtained using bilinear interpolation or a predetermined reconstruction filter. In the reduced area, anti-aliasing sampling is performed according to the local sampling ratio. All samples are directly read from the original image. Attitude correction and local deformation do not generate intermediate resampled images separately.
[0081] Since the attitude reference mapping may be nonlinear within the triangle, the precise final mapping at the midpoint, centroid, and additional checkpoints (if necessary) is compared with the triangulation results. When the maximum difference among the sampled points exceeds 0.25 master pixels, the triangle used for rasterization is subdivided. A maximum subdivision level is preset, for example, 8 levels, and the lower limit of the shortest side of the source triangle is set to 0.5 original pixels. If the sampling point tolerance is not met even at any limit, the region is marked as invalid to prevent subdivision from continuing indefinitely and to prevent the output of unchecked mappings. Internal checkpoints are added at locations with drastic error changes to detect folds. The local correction amount for newly sampled vertices is still obtained by interpolation from the accepted original mesh, without adding new deformation unknowns, thus not changing the already accepted compensation division. If it is necessary to increase the degrees of freedom of the optimized mesh, it must be solved and accepted again. For the subdivided sampled triangular mesh, orientation, scaling, non-adjacent overlap, and coverage checks are performed again. If the checks fail, the candidate mapping is not output.
[0082] Triangle clipping is performed on the effective hemispherical boundary, and an invalid mask is output for areas outside the original image, outside the calibration range, without a unique mapping, or without source image coverage. When a projector calibration table exists, it can be combined with the mapping of this embodiment before final sampling to obtain the projector source coordinates. However, the installation calibration of the projector does not participate in the error allocation between the attitude and local deformation of this embodiment.
[0083] The above method is further illustrated below with a set of implementation configurations; it can use raw aerial images at 60 frames per second, 800 Hz attitude data, 2160 lines of images, 12 milliseconds whole frame readout time, and a 4096 by 4096 pixel dome master. The single-line exposure duration is set to 1 millisecond or adjusted according to the shooting brightness. The estimated parameters are selected from a segment of about 2 seconds that includes varying angular velocities. The segment should have reliable control point observations across different exposure lines and different field of view directions. Whether it can be estimated is based on matrix sensitivity and measurement error checks.
[0084] For example, with top-down readout and a residual time offset of 4 milliseconds, the exposure times of the first row, the middle row with a normalized row number of 0.5, and the last row, relative to... The time intervals are 4.5 milliseconds, 10.5 milliseconds, and 16.5 milliseconds, respectively; the calculated camera poses for each row are used in the dome mapping, and the poses at intermediate moments cannot be directly used for the whole frame.
[0085] For example, the local regularization coefficient is taken These coefficients are frozen after being determined on independently calibrated segments; for static control points at a distance of approximately 500 meters, if the combined lateral standard deviation of the control point and the camera optical center is approximately 0.1 meters, then the corresponding directional standard deviation is approximately... The radius of radians corresponds to approximately 0.26 radial pixels on an equidistant master with a radius of 2048 pixels. This estimate must be combined with the pixel extraction error and cannot be used to conclude that all verification points have the same precision.
[0086] It is understandable that the aforementioned weighted orthogonality relation resolves the ambiguity of compensation allocation; under fixed samples, fixed weights, and the current linearization, the weighted inner product of pose compensation and local compensation satisfies: in, Represents the inner product with the same weight. For any vector of the same dimension; within this observable space, the two compensations will not simultaneously bear the same response component. After the attitude changes, the response space is regenerated and the total mesh is solved again, so that this division of labor is updated synchronously with the mapping state.
[0087] To check for repeated compensation of historical meshes under new pose states, let the states before and after parameter updates be labeled with their superscript numbers. and Identifier, definition: in, and These are the left singular vector basis and whitening matrix of the attitude response recalculated under the new parameters and the same set of control points. The interpolation matrix is determined by the original pixel positions and remains unchanged during this parameter update. For the total local deformation under the old parameters, The total local deformation is recalculated under the new parameters, with new residuals and new constraints. and These are the non-negative squared quantities projected from the old mesh and the recalculated mesh onto the new attitude response space, respectively. It still represents the Euclidean norm; the old grid only satisfies the orthogonality condition of the old state. In the new state, it may be greater than zero, while through the new constraints Reduced to within the numerical acceptance tolerance, and At the same time, the remaining compensable components are fitted according to the new residuals. It is not acceptable to simply overlay the old mesh onto the new datum mapping or to project the old mesh once as if the recalculation is complete.
[0088] Therefore, row-by-row attitude processing can reduce attitude timing error, local mesh processing can reduce residual projection error, and the two step groups share the response matrix and spherical error metric, and the total mesh is re-resolved by updating the same parameter, which further reduces the mutual masking of the two corrections and the repeated superposition of historical compensation; the synergistic effect has a clear data source, constraint relationship and execution action.
[0089] It should be noted that when the actual local error coincides with a certain attitude response mode, the physical cause cannot be determined unconditionally by mathematical orthogonal decomposition alone. In this embodiment, error attribution is limited by independent control points, measurement error budget, parameter boundaries and verification data. When it is impossible to reliably distinguish, unresolved residuals are retained or downgraded processing is adopted. All residuals in the orthogonal complement are not directly identified as lens error or actual parallax.
[0090] In other implementations, other dome source formats with defined forward and inverse mappings can be used, requiring only the establishment of the same spherical metric for attitude updates and local deformation solutions using the corresponding inverse mapping derivatives; alternatively, the above processing can be applied to multiple cameras separately, but each camera retains its own exposure sequence, mounting extrinsic parameters, and optical center position. For near-field multi-optical-center parallax, a reliable depth should be used for viewpoint transformation or a seam treatment that meets visibility conditions should be adopted, and it cannot be forcibly eliminated simply by using the same spherical orientation.
[0091] The above method can be stored in a non-volatile storage medium as a computer program. The processor reads the original image, records and calibrates the data synchronously, and executes it. Under the premise of maintaining the shared response constraints, the overall local deformation recalculation, and the joint acceptance relationship of the same version, the specific type of processor and the offline or segmented execution mode do not affect the implementation of the above method.
Claims
1. A geometric correction method for aerial dome images based on flight attitude data, characterized in that, include: Based on the line-by-line exposure time, fuselage attitude and camera relative attitude of the original aerial image, a reference mapping from the original pixels to the dome coordinates is generated, and a reference direction is generated with the camera optical center at the corresponding exposure time and the static control point determined independently of the current correction segment. Calculate the reprojection residuals of the reference direction and datum mapping on the same set of solution control points. A weight matrix is established using the inverse mapping derivative of the spherical screen and measurement error. The response matrix is constructed using the negative derivative of the residual with respect to the increment of the estimated parameter, including residual time and direction biases. ; A local mesh is created from the original image, with corrections applied to all mesh vertices. Representing total local deformation using an interpolation matrix Solve for local corrections and constraints at control points. ; For each candidate parameter to be estimated, recalculate the exposure time, camera optical center, reference mapping, and reference direction. , and And under the constraints of reconstruction, solve all the problems again. Only when the residual target, mesh geometry, and verification point angle errors separated by control point identity all meet the preset joint acceptance conditions will the candidate parameters to be estimated be jointly adopted. ; Using benchmark mapping adopted in the same time and The dome image is generated by resampling the original aerial image once.
2. The method according to claim 1, characterized in that, The line-by-line exposure time is determined based on the first line exposure start time after conversion to the attitude time reference, the residual time deviation, the normalized line number, the whole frame readout time, and the single line exposure duration. The normalized line number is determined by the number of image lines and the actual readout direction. The parameters to be estimated include the residual time deviation and the orientation deviation, and the whole frame readout time is an independent calibration quantity; The reference mapping is obtained by sequentially applying the camera-to-gimbal rotation, the gimbal-to-body rotation, the body-to-world coordinate system rotation, the rotation corresponding to the directional deviation, and the world coordinate system-to-spherical coordinate system rotation to the camera unit line of sight.
3. The method according to claim 2, characterized in that, The location of the static control points is provided by independent measurement or by a georegistration model established and pre-frozen from mapping data independent of the current correction segment; The position of the camera optical center is determined based on the timestamped position data of the progressive exposure time and the optical center lever arm; after the estimated parameters are updated, the position of the camera optical center is recalculated, and the reference direction is recalculated based on the direction from the static control point to the recalculated camera optical center.
4. The method according to claim 1, characterized in that, For each control point, calculate the derivative of the inverse spherical mapping at its reference spherical coordinates. ,Depend on The matching confidence level and angle measurement error of the control point form corresponding weight blocks, and the weight blocks are combined to form the weight matrix. ; In a linear solution, the calculation of the response matrix, parameter updates, and global local correction constraints use the same set of solution control points. The set or Rebuild synchronously when changes occur. and the aforementioned constraints.
5. The method according to claim 2, characterized in that, The residual time deviation and direction deviation are scaled and normalized, and the response matrix is determined by perturbing the parameters of the complete reprojection residual calculation chain. The parameters that can be updated are determined based on the singular values of the weighted response matrix and the parameter uncertainty. When the residual time deviation and the orientation deviation cannot be reliably distinguished, the residual time deviation is fixed and the identifiable orientation deviation is updated. For attitude response modes that can be reliably calculated but are prohibited from being updated, the total local deformation is still restricted from compensating for the attitude response mode.
6. The method according to claim 4, characterized in that, A triangular mesh is created on the original aerial image, and the interpolation matrix is determined by the centroid coordinates of the original pixels within the source triangle. The total local deformation vector is composed of the two-dimensional spherical coordinate corrections of the mesh vertices. Under the objective function, which includes weighted residual terms, spatial difference terms, and positive coefficient magnitude regularization terms, the total local deformation vector is solved in the feasible space defined by the weighted orthogonal equation. The weighted orthogonal equation applies to all mesh vertex corrections, not just to the current deformation increment.
7. The method according to claim 6, characterized in that, Establish cross-frame checks using verification observations from adjacent frames that belong to the same static control point but were not involved in the solution process; The reference direction of the static control point is calculated based on the camera optical center at the actual exposure time of each frame. The error vector of the final mapping direction of each frame relative to the corresponding reference direction is transformed to the same world coordinate system and compared. When the difference exceeds the predetermined error threshold, the candidate parameters to be estimated and the total local deformation are jointly rejected. The cross-frame check is not established for dynamic objects, occluded areas and areas without reliable reference directions.
8. The method according to claim 6, characterized in that, The update of the parameters to be estimated adopts a candidate step size. For each candidate step size, the reference mapping, reference direction, reprojection residual, response matrix and weight matrix are recalculated, and the total local deformation is resolved under the reconstructed deformation constraints. In a candidate step size comparison, the set of control points and the weight matrix and regularization matrix used for evaluation are fixed. When the candidate mapping fails the geometric feasibility check, the control points exceed the effective calibration domain, or the fixed evaluation target does not decrease, the candidate parameters and candidate total local deformation are rejected and the step size is reduced. The original total local deformation is only used as the initial value for the solution.
9. The method according to claim 8, characterized in that, The joint acceptance includes: Using the verification data separated by control point identity from the solution data, the angular error between the final mapping direction and the reference direction is calculated; Check the orientation, local scaling, overlap of non-adjacent triangles, effective coverage, and weighted orthogonal error of total local deformation of the reference and final mapped triangles; When the angle error, geometric feasibility, and weighted orthogonal error all meet the preset conditions, the parameters to be estimated, the total local deformation, and the corresponding response matrix and weight matrix are recorded as the same acceptance version.
10. The method according to claim 9, characterized in that, The benchmark mapping that passed the joint acceptance test and the total local deformation are combined to form the final mapping from the original pixels to the spherical screen coordinates. The target pixel coverage is calculated based on the mapped triangle, and the original pixel sampling coordinates are calculated back. The original aerial image is then resampled once. When the triangular interpolation error of the reference mapping exceeds the preset tolerance, the triangles used for resampling are subdivided within the preset maximum subdivision level. The local correction amount of the newly sampled vertices is obtained by interpolation of the accepted mesh. Areas that still do not meet the tolerance after reaching the maximum subdivision level are marked as invalid. Invalid pixel masks are output for areas outside the effective range of the dome screen, outside the effective range of the original image, and areas lacking reliable mapping coverage.
