A Method for Low-Altitude Digital Acquisition and Data Fusion of Unmanned Aerial Vehicles for Multi-Measurement Integration

Abstract

This invention relates to the field of image data processing technology and discloses a method for low-altitude digital acquisition and data fusion of UAVs for multi-measurement integration. The method includes: first, unifying the reference image and the image to be stitched into an elevation coordinate system and adaptively dividing it into multiple repeating units; then, extracting the normal brightness offset field between the repeating units; using the normal brightness offset field to perform normal position compensation on the image to be stitched, and simultaneously constructing a stitching cost map to generate an optimal stitching seam with edge avoidance; next, performing unit-level exposure correction and image fusion along the optimal stitching seam, using repeating units as units, to obtain an initial stitched image; finally, calculating the centroid residual to update the normal brightness offset field, and using the updated normal brightness offset field to re-execute position compensation, stitching seam generation, and image fusion operations to output the final stitched image. This invention effectively eliminates the visual misalignment problem during periodic elevation reconstruction and improves the quality of data fusion.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and more specifically, to a method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) for multi-measurement integration. Background Technology

[0002] In recent years, with the popularization of UAV multi-angle oblique photography technology, it has been widely used in tasks such as building facade detail acquisition, high-precision 3D reconstruction, and curtain wall semantic extraction. When acquiring low-altitude images of periodic facade units such as glass curtain walls, metal louvers, vertical keels, and horizontal pressure plates, it is often necessary to fuse and stitch images taken by UAVs from different perspectives.

[0003] Existing image stitching algorithms typically assume that texture is largely conserved across different views, or attribute matching errors primarily to differences in single-plane projection and general local deformations. However, curtain walls and louvered facades often exhibit material homogeneity, weak texture, and extremely high sensitivity to directional reflection and environmental mapping. Under the combined influence of external solar azimuth, sky brightness, and the geometric differences between the two views from drones, strong directional reflections cause the specular peak positions within the same curtain wall or louver unit to shift laterally. Simultaneously, due to the setback depth of the keel and louvers, changing viewing angles causes the visible boundary sequence to self-occlude and rotate. This results in the brightest position within each facade unit no longer coinciding with the true geometric center from different viewing angles.

[0004] When existing technologies process such periodic and homogeneous scenes, they are highly susceptible to mistaking the reflected light migration driven by line-of-sight differences for actual geometric displacement, and mistaking occlusion rotation for newly revealed actual parts. Therefore, traditional fusion methods frequently produce significant visual defects in such facades, such as repeated window panes, misaligned grids, jagged edges, and flickering seams. This complex mismatch problem, caused by the deep coupling of periodic units, dual-view parallax, directional reflection, and self-occlusion rotation, severely reduces the accuracy and consistency of panoramic facade vision, becoming a technical bottleneck restricting high-quality fusion of low-altitude multi-measurement images. Summary of the Invention

[0005] This invention provides a method for low-altitude digital acquisition and data fusion of UAVs for multi-measurement integration, which solves the technical problems mentioned in the background art.

[0006] This invention provides a method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) for multi-measurement integration, including: Extract the main elevation direction of the reference image and the image to be stitched and perform coordinate mapping to unify the reference image and the image to be stitched to the elevation coordinate system; The unit spacing is adaptively calculated within the elevation coordinate system, and multiple repeating units are divided based on the unit spacing. Extract the luminance centroids of the reference image and the image to be stitched within the repeating unit, and obtain the normal luminance offset field between the repeating units; The normal position of the image to be stitched is compensated using the normal brightness offset field to obtain the compensated image to be stitched. A stitching cost map is constructed using the normal brightness offset field, and an optimal stitching seam with edge avoidance is generated in the stitching cost map; Using the repeating unit as a unit, the compensated image to be stitched and the reference image are subjected to unit-level exposure correction and image fusion along the optimal stitching seam that is avoided by the edge to obtain the initial stitched image; The centroid residual of the initial stitched image is calculated and superimposed onto the normal brightness offset field to obtain the updated normal brightness offset field. The updated normal brightness offset field is then used to re-perform the normal position compensation on the image to be stitched, and the optimal stitching seam for edge avoidance is regenerated. The unit-level exposure correction and image fusion are then performed to output the final stitched image.

[0007] The beneficial effects of this invention are as follows: by unifying the reference image and the image to be stitched into the facade coordinate system, and adaptively dividing the repeating units to extract the normal brightness offset field, the image to be stitched is driven to complete the normal position compensation. At the same time, the optimal stitching seam is generated by combining the edge avoidance strategy and the unit-level exposure correction is performed. Finally, the residual backflow correction mechanism is introduced to complete the closed-loop update, thereby effectively eliminating the visible defects such as window pane repetition, grid misalignment and gap flicker caused by directional reflection transfer and occlusion rotation, and significantly improving the multi-image fusion accuracy and panoramic visual consistency in periodic facade scenes. Attached Figure Description

[0008] Figure 1 This is a flowchart of the UAV low-altitude digital acquisition and data fusion method for multi-measurement integration according to the present invention. Detailed Implementation

[0009] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0010] like Figure 1As shown, a method for low-altitude digital acquisition and data fusion of UAVs for multi-measurement integration includes: Extract the main elevation direction of the reference image and the image to be stitched and perform coordinate mapping to unify the reference image and the image to be stitched to the elevation coordinate system; The unit spacing is adaptively calculated within the elevation coordinate system, and multiple repeating units are divided based on the unit spacing. Extract the luminance centroids of the reference image and the image to be stitched within the repeating unit, and obtain the normal luminance offset field between the repeating units; The normal position of the image to be stitched is compensated using the normal brightness offset field to obtain the compensated image to be stitched. A stitching cost map is constructed using the normal brightness offset field, and an optimal stitching seam with edge avoidance is generated in the stitching cost map; Using the repeating unit as a unit, the compensated image to be stitched and the reference image are subjected to unit-level exposure correction and image fusion along the optimal stitching seam that is avoided by the edge to obtain the initial stitched image; The centroid residual of the initial stitched image is calculated and superimposed onto the normal brightness offset field to obtain the updated normal brightness offset field. The updated normal brightness offset field is then used to re-perform the normal position compensation on the image to be stitched, and the optimal stitching seam for edge avoidance is regenerated. The unit-level exposure correction and image fusion are then performed to output the final stitched image.

[0011] Preferably, the elevation main direction of the reference image and the image to be stitched is extracted and coordinate mapped to unify the reference image and the image to be stitched to the elevation coordinate system, including: The image power spectra of the reference image and the image to be stitched are calculated respectively, and the corresponding image smoothing scale is adaptively obtained based on the image power spectra. The calculation formula is as follows: in, Indicates the first Zhang image, when When refers to the reference image, when The time indicates the image to be stitched together; This indicates the Fourier transform operation; Indicates the first Image power spectrum of the image; A coordinate vector representing the frequency domain; Indicates the first Image smoothing scale for the images; The image structure tensors of the reference image and the image to be stitched are calculated using the image smoothing scale, respectively, using the following formula: in, Indicates the first Image structure tensor of a single image; The standard deviation represents the image smoothing scale. Gaussian smoothing kernel; Indicates the convolution operation; and They represent the first Spatial gradients of an image in the horizontal and vertical directions; By fusing the image structure tensor of the reference image and the image to be stitched, the principal direction of the facade is extracted. The principal direction of the facade includes the principal tangent and the principal normal of the facade, and the calculation formula is as follows: in, Indicates the principal tangent of the facade; Indicates the main normal of the facade; Represents the integration region of the image; This represents the area of ​​the integration region; Operators that represent the operations for finding the eigenvector corresponding to the largest eigenvalue of a matrix; This represents a rotation matrix with a rotation angle of 90 degrees. Extract the facade structure lines from the reference image and the image to be stitched, and solve the initial projection transformation matrix based on the facade structure lines. The calculation formula is as follows: in, Represents the initial projection transformation matrix; Represents the projection transformation matrix variables; This represents the total number of pairs of matched facade structural lines; The first in the reference image The homogeneous coordinate vector representation of the facade structural lines, and ; This indicates that the image to be stitched contains elements related to the first... Homogeneous coordinate vectors of feature points corresponding to each facade structural line; The initial projection transformation matrix is ​​used to perform a projection transformation on the image to be stitched, and the principal tangent and principal normal of the facade are used for coordinate transformation to unify the reference image and the image to be stitched to the facade coordinate system. The calculation formula is as follows: in, Represents a two-dimensional pixel coordinate vector; This represents the homogeneous coordinate vector corresponding to the two-dimensional pixel coordinate vector; Represents the projection operator from homogeneous coordinates to two-dimensional coordinates; This represents the image to be stitched together after projection transformation; This represents the reference image after initial mapping; This represents the normal coordinate across a periodic unit in the elevation coordinate system. This represents the tangential coordinate along the facade structure line in the facade coordinate system.

[0012] The reference image is the original low-altitude acquired image that serves as the benchmark for geometric alignment and fusion. The image to be stitched is another original low-altitude acquired image that effectively overlaps with the reference image and requires subsequent compensation and fusion processing. It can be obtained by using a UAV equipped with a visible light camera to perform oblique photography of the same target facade from adjacent low-altitude perspectives. The image notation is a serialized representation used to indicate the two original acquired images in the same way, where 1 corresponds to the reference image and 2 corresponds to the image to be stitched.

[0013] Image power spectrum is a frequency domain quantity used to describe the frequency energy distribution of an image, reflecting the relative strength of low-frequency smooth components and high-frequency structural components in the image.

[0014] The Fourier transform operation is a frequency domain analysis operation that transforms an image from the spatial domain to the frequency domain.

[0015] The frequency domain coordinate vector is a frequency variable used to calibrate the frequency domain position and to represent the energy distribution of the image power spectrum or profile spectrum at different frequency positions.

[0016] Image smoothing scale is a scale parameter used to control the smoothing intensity when calculating structure tensors. A larger value indicates a wider smoothing range, while a smaller value indicates that more local details are preserved.

[0017] The image structure tensor is a matrix-type feature quantity used to comprehensively characterize the local directionality and gradient consistency of an image.

[0018] The Gaussian smoothing kernel is a smoothing kernel function with the image smoothing scale as the width parameter. It is used to suppress local noise and reflection perturbations when calculating the structure tensor.

[0019] Spatial gradient is a derivative that describes the rate of gray-level change of an image in the horizontal and vertical directions. It is used to construct the structure tensor and characterize the principal direction.

[0020] The main orientation of a facade is a directional quantity that describes the direction of the main axis of the facade's periodic structure. The main tangential orientation of the facade indicates the direction extending along the keel or louvers, while the main normal orientation of the facade indicates the direction crossing the periodic unit.

[0021] The facade coordinate system is a normalized coordinate domain established with the facade principal normal and facade principal tangent as coordinate axes. The normal coordinate is used to represent the position across the unit direction, and the tangent coordinate is used to represent the position along the structural line direction.

[0022] The image integration region is the effective elevation calculation area used for statistical analysis of the structure tensor and principal orientation. It is preferentially selected as the common effective elevation area after coarse registration of the two images, excluding black borders, severely occluded areas, and invalid resampling areas. This ensures that the statistical results are contributed only by truly comparable regions. The area of ​​the integration region is the area of ​​this effective region, used to normalize the region's statistical results.

[0023] The eigenvector solving operator corresponding to the largest eigenvalue is a directional solving operator used to extract the dominant direction from the structure tensor.

[0024] The 90-degree rotation matrix is ​​an orthogonal transformation matrix that rotates the principal tangent of the facade into the principal normal of the facade, and is used to form another coordinate axis of the facade coordinate system.

[0025] Facade structural lines are long straight lines in a facade that have stable geometric significance, such as the edges of keel, window frame boundaries, or horizontal pressure plate boundaries.

[0026] The initial projection transformation matrix is ​​used to coarsely align the images to be stitched to the geometric framework of the reference image. Its geometric function is to perform reference alignment on the outermost planar frame of the facade, while actively retaining the motion parallax generated by the internal structure with setback depth as a dedicated normal brightness offset for subsequent processing. The projection transformation matrix variable is the matrix variable to be optimized when solving for this optimal mapping.

[0027] The total number of matching facade structure lines is the number of effective line constraints participating in the initial projection transformation solution, used to represent the size of the constraint set.

[0028] The homogeneous coordinate vector of the facade structure line is a line parameter vector used in projection geometry to express the facade structure line. Its components together determine the position and direction of the line.

[0029] The homogeneous coordinate vector of the corresponding feature point is the point constraint expression form associated with the facade structure line in the image to be stitched.

[0030] Two-dimensional pixel coordinate vectors are planar coordinate representations of pixel positions in an image, while homogeneous coordinate vectors are their extended form in projective geometry.

[0031] The projection operator is a mapping operator that restores homogeneous coordinate results to two-dimensional pixel coordinate results.

[0032] The image to be stitched after projection transformation is a coarsely normalized image obtained after the initial projection mapping of the image to be stitched. It is the input image for subsequent periodic analysis and drift compensation.

[0033] The reference image for initialization mapping is a reference image representation expressed in a unified coordinate system while maintaining the reference image's datum position.

[0034] Convolution is a computational method that locally aggregates a Gaussian smoothing kernel with a quadratic gradient term.

[0035] In detail, because periodic curtain walls and louvered facades simultaneously contain low-frequency brightness fluctuations, high-frequency structural lines, and pseudo-high-frequency components formed by reflections, a fixed smoothing scale can easily oversmooth the real structure in some images while retaining too much reflective noise in others. Therefore, the frequency domain energy distribution of the entire image should be statistically analyzed using the power spectrum first, and then the smoothing scale adapted to the image should be deduced from the frequency domain energy centroid. This way, the structural tensor can retain the main direction information while suppressing local reflections and noise interference.

[0036] In detail, because ordinary corner points and texture points on periodic facades are prone to mismatch under conditions of weak texture, repetitive texture, and specular reflection, while long straight structural lines and their corresponding points have stronger geometric stability and clearer directional constraints on building facades, it is more suitable for this scenario to establish a coarse normalization transformation by using the joint constraints of structural lines and points than to simply rely on arbitrary texture feature points. This can significantly reduce the risk of misjudging reflection differences as geometric displacement in the initial registration stage.

[0037] In detail, considering that the baseline changes of adjacent low-altitude UAV views are small and the initial relative rotation angle is limited, under the previous assumption, directly fusing the image structure tensors of the two views can obtain a robust initial estimate of the structure orientation. Furthermore, since the core mismatch mainly occurs in the cross-period unit direction rather than irregular deformation in any direction, the two-angle images are first uniformly mapped to the elevation coordinate domain composed of the normal and tangential directions. This allows the directional relationship originally coupled in the two-dimensional pixel plane to be decomposed into two independent dimensions: along the unit direction and across the unit direction. Therefore, subsequent pitch analysis, centroid drift modeling, and period compensation can be stably implemented in a unified reference system.

[0038] In detail, the selection method for stable structural lines and corresponding points is as follows: First, the gradient magnitude map is calculated in the reference image and the image to be stitched. Then, based on the main direction of the facade, long connected high-response ridges in the approximate vertical and approximate horizontal directions are extracted respectively. Line fitting is performed on each ridge and the line energy is calculated. After sorting the line energy from high to low, two long vertical lines and two long horizontal lines that can simultaneously cover the two main directions are selected as stable structural lines. Then, the line correspondence is established in the two images based on the proximity of the line positions, the consistency of the directions, and the similarity of the lengths. Then, in the image to be stitched, feature points located near the endpoints of each matched structural line and with stable gradient response are selected as corresponding points. Finally, candidate line segments and points that are too short, severely broken, close to strong reflective saturation areas, or located in invalid black border areas are eliminated.

[0039] In detail, the initial projection transformation matrix degradation process is as follows: when more than three candidate structural lines are nearly parallel, causing the constraint matrix to be ill-conditioned, the intersection points of the selected vertical and horizontal lines are first calculated; if the number of effective intersection points is sufficient, the projection transformation matrix is ​​re-estimated using the line intersection points and the original structural lines to form a hybrid constraint; if the number of intersection points is still insufficient, the endpoints of the line segments are further added to the constraint set; when resolving, least squares are used and the weights of line constraints with overly similar directions are reduced; if the condition number is still too large after re-estimation, the process is reversed and the next set of horizontal or vertical lines is selected from the candidate set until a non-degenerate result is obtained.

[0040] In detail, the discretization of normal and tangential coordinates is implemented as follows: First, establish the two-dimensional position of each output pixel in the original image. Then, perform projection operations on the two-dimensional position using the principal normal and principal tangential of the elevation to obtain the corresponding normal and tangential coordinates. Establish a regular normal and tangential grid for the output image. Record the corresponding floating-point coordinates of the original image for each grid point. Then, obtain the pixel value in a unified elevation coordinate system by sampling from the original image through inverse mapping. For grid points that exceed the boundary of the original image, mark them as invalid sampling points and remove them in the subsequent calculation of the effective area.

[0041] Preferably, the unit spacing is adaptively calculated within the elevation coordinate system, and multiple repeating units are divided based on the unit spacing, including: The image gradients of the reference image and the image to be stitched are extracted in the elevation coordinate system, and the image gradients are projected onto the principal normal of the elevation. Normal structural projection profiles of the reference image and the image to be stitched are constructed respectively, using the following calculation formula: in, Indicates the first The normal structural projection profile of the image, when When refers to the reference image, when The time indicates the image to be stitched together; Indicates the integration region; The first position in the elevation coordinate system represents the first position. The image in two-dimensional pixel coordinate vectors Image gradient at the location; Indicates the main normal of the facade; Represents the Dirac function; This represents the normal coordinate in the elevation coordinate system; Fourier transforms are performed on the normal structural projection profiles of the reference image and the image to be stitched to obtain the profile spectrum, and the centroid of the profile spectrum is calculated using the following formula: in, Indicates the first The cross-sectional spectrum of the image; This indicates the Fourier transform operation; Represents the frequency variable in the spectral domain; Indicates the first The centroid of the cross-sectional spectrum of the image; Based on the centroid of the cross-sectional spectrum, the spacing between individual image units in the reference image and the image to be stitched is calculated, and the average value of the individual image unit spacing is obtained to obtain the unit spacing. The calculation formula is as follows: in, Indicates the first The spacing between individual image cells; This represents the spacing between individual image cells in the reference image; This represents the spacing between individual image units in the image to be stitched together; Indicates the unit spacing; Pi is a constant. By fusing the normal structural projection profiles of the reference image and the image to be stitched, and using the cell spacing to maximize the cosine response, the initial phase of the period is obtained. The calculation formula is as follows: in, This represents the fused normal structural projection profile. This represents the normal structural projection profile of the reference image; This represents the normal structural projection profile of the image to be stitched together; Indicates the starting phase of the period; Indicates the phase shift variable; This represents the variable operator that maximizes the objective function; Represents the cosine function; The cell boundary is determined based on the periodic start phase and the cell spacing. The multiple repeating cells are then divided using the cell boundary. The calculation formula is as follows: in, Indicates the first The initial cell boundary of a repeating cell; Indicates the first The boundary of the terminating unit of a repeating unit. The variable representing the sequence number of the repeating unit; Indicates the first One repeating unit; This represents the tangential coordinate in the elevation coordinate system.

[0042] The normal structural projection profile is a one-dimensional structural response distribution formed by projecting the image gradient along the principal normal of the elevation and then aggregating it along the tangential direction. It is used to characterize the repetitive rhythm of periodic units in the normal direction.

[0043] The image gradient operator is a derivative operator used to calculate the direction and intensity of local gray-level changes in an image.

[0044] The Dirac function is a one-dimensional cumulative position selection function used in continuous expressions to map a two-dimensional position to a specified normal coordinate position.

[0045] The profile spectrum is the frequency expression obtained by frequency domain transformation of the normal structural projection profile, and is used to identify the dominant frequency of the periodic pitch.

[0046] The centroid of the profile spectrum is a frequency statistic used to represent the location of the main energy concentration in the profile spectrum, and is used to robustly estimate the spacing between single plot cells.

[0047] The cell spacing in a single image is an estimate of the cell period width obtained from the centroid of the spectrum of a single image. The reference image and the image to be stitched each correspond to their respective estimates of the cell spacing in a single image.

[0048] The unit spacing is a uniform periodic pitch determined by the unit spacing of two images.

[0049] Pi is a mathematical constant that connects frequency and period length.

[0050] The fused normal structural projection profile is a joint representation of the normal structural response of the reference image and the image to be stitched together.

[0051] The periodic start phase is the starting position of the boundary of a uniform periodic unit in the normal coordinate system, used to determine the boundary sequence of all repeating units.

[0052] The phase shift variable is a candidate phase variable used to search for the location of the maximum cosine response.

[0053] The variable-value operator that maximizes the objective function is the operator used to return the optimal phase position.

[0054] The cosine function is a periodic function used to construct a periodic response template and phase match it with the projected profile of the structure.

[0055] The element boundary is the starting and ending point of the repeating element in the normal coordinate system.

[0056] The repeating unit number variable is a number variable used to distinguish the positions of different periodic units.

[0057] The first repeating unit is a unit region enclosed by the boundaries of two adjacent units, and it is the basic processing unit for subsequent centroid calculation, exposure correction and fusion.

[0058] In detail, because the essential characteristics of periodic curtain walls and louvers are not isolated edges, but rather the recurring rhythm of units along the normal, directly relying on manual thresholding, manual measurement, or single edge detection to determine the pitch is easily affected by reflections, occlusions, and local defects. Therefore, the gradient should first be projected along the normal to form a one-dimensional structural profile, and then the dominant period length should be estimated using the energy centroid of the profile spectrum. This can average out local anomalies, thereby obtaining a more robust unified unit spacing for both images.

[0059] In detail, because the pitch of the repeating element only gives the element width and not the starting position of the first element boundary in the normal direction, if the origin of the periodic phase is not further calculated, the subsequent element division will be misaligned as a whole. Therefore, it is necessary to match the fused normal structure projection profile with the periodic cosine template and take the phase with the largest response as the starting phase of the period. Only in this way can the boundaries of each element be aligned with the actual structure period.

[0060] In detail, the discrete accumulation method of the normal structural projection profile is as follows: First, calculate the two-dimensional gradient of each pixel in the unified elevation coordinate image, then multiply the gradient with the principal normal of the elevation and take the absolute value as the contribution value of the pixel to the normal structural response; then project the pixel position onto the normal coordinate axis and assign it to the corresponding normal accumulation bucket according to the nearest integer position; finally, add all the contribution values ​​in the same accumulation bucket, and after explicitly subtracting its global numerical average value to remove the DC bias component brought by the forward projection of the structure, a discrete one-dimensional normal structural projection profile is formed.

[0061] In detail, the calculation method of the centroid of the profile spectrum in the discrete frequency domain is as follows: First, perform a one-dimensional fast Fourier transform on the discrete normal structural projection profile to obtain a discrete spectrum sequence; to avoid the rich high-order harmonics formed at the edges of the facade structure from causing high-frequency pulling on the energy centroid, a low-pass filter is first applied to the discrete spectrum sequence to suppress high-frequency harmonic components; then, calculate the square of the spectrum amplitude at all positive frequency positions after filtering; subsequently, multiply each positive frequency position by the corresponding spectrum energy and sum them as the numerator, and then use the sum of all positive frequency energies as the denominator; finally, divide the numerator by the denominator to obtain the spectral centroid; when the denominator is close to zero, it is directly determined that the profile does not have stable periodicity and an invalid result is returned, and the effective normal structural region is reselected in the previous step before recalculation.

[0062] In detail, the edge half-unit processing method is as follows: First, generate all theoretical unit boundaries based on the periodic start phase and unit spacing; then check whether each candidate unit has both a complete start boundary and a complete end boundary in the common effective area of ​​the two images; if a unit only retains a local fragment of less than the complete width at the edge, the unit is not counted as a complete repeating unit and does not participate in the subsequent statistics of brightness centroid, offset, and average drift field; only units with complete left and right boundaries and effective sampling pixels inside are retained as subsequent processing objects.

[0063] Preferably, extracting the luminance centroids of the reference image and the image to be stitched within the repeating unit, and obtaining the normal luminance offset field between the repeating units, includes: The average brightness of the reference image and the image to be stitched within the repeating unit is calculated using the following formula: in, Indicates the first The image in the first Within each repeating unit, along the tangential coordinates of the elevation coordinate system... The average brightness of the unit, when When refers to the reference image, when The time indicates the image to be stitched together; Indicates the unit spacing; Indicates the first The initial cell boundary of a repeating cell; Indicates the first The boundary of the terminating unit of a repeating unit. The first position in the elevation coordinate system represents the first position. The image is in normal coordinates and tangential coordinates Image pixel values ​​at that location; Based on the average brightness of the unit, the positive brightness deviation portions of the reference image and the image to be stitched within the repeating unit that are higher than the average brightness of the unit are extracted respectively, and the calculation formula is as follows: in, This indicates the positive deviation in brightness; This indicates the operator for taking positive values, that is, taking the maximum value between the value inside the parentheses and zero; Using the positive brightness deviation portion, the brightness centroids of the reference image and the image to be stitched are calculated within the repeating unit, respectively, using the following formula: in, Indicates the first The image in the first Within each repeating unit along the tangential coordinate The aforementioned centroid of brightness; The difference between the luminance centroid of the image to be stitched and the luminance centroid of the reference image is used to obtain the cell normal offset corresponding to each repeating cell. The calculation formula is as follows: in, Indicates the first The element normal offset corresponding to each repeating element; This indicates the luminance centroid of the images to be stitched together; This represents the luminance centroid of the reference image; The average value of the normal offset of the multiple repeating units is calculated to obtain the normal brightness offset field between the repeating units. The calculation formula is as follows: in, This represents the normal brightness offset field between the repeating units; This indicates the total number of the multiple repeating units.

[0064] The unit average luminance is a local average luminance baseline obtained by integrating along the normal direction within a certain repeating unit, and is used to separate the main luminance contributions that are higher than the average value.

[0065] The pixel value of a certain image in the elevation coordinate system after normalization is the pixel intensity expression after being unified to the elevation coordinate domain, which is used to compare two images under the unified coordinates of normal and tangential directions.

[0066] The positive deviation in brightness is the positive value retained after subtracting the average brightness of a unit from its pixel value, representing the main bright contribution area within the unit.

[0067] The positive value operator is an operation operator that truncates differences less than zero to zero and retains differences greater than zero. It is used to retain only the portion of brightness above the average.

[0068] The luminance centroid is the centroid of the main contribution of brightness in the normal coordinate system, used to indicate the location of the main luminance peak within a cell. The reference image and the image to be stitched each correspond to their respective luminance centroids.

[0069] The cell normal offset is the difference in the brightness centroid of the same repeating cell in two images, used to represent the normal shift of the main brightness peak within that cell.

[0070] The normal brightness offset field is a field function formed by averaging the normal offsets of multiple repeating elements along the tangential direction. It is used to describe the systematic normal drift of the entire facade at different tangential positions.

[0071] The total number of complete repeating cells is the number of effective repeating cells participating in the statistics of the normal brightness offset field, which is used to average and normalize the cell offset.

[0072] In detail, because the area that is truly affected by the dual-view reflection migration in the periodic facade is the bright main contributing area inside the unit, rather than the overall average brightness that is lowered by shadows, textures and dark areas, the centroid is calculated by first using the unit mean as the baseline and only retaining the positive deviation brightness above the mean. This allows us to extract the position that truly represents the main brightness peak inside the unit. Therefore, this centroid is more accurate in reflecting the normal shift caused by reflection migration and visible boundary rotation than directly calculating the centroid of the entire unit's grayscale.

[0073] In detail, because the brightness centroid difference of a single repeating unit may be affected by local noise, local occlusion, and individual unit differences, it is difficult to stably represent the systematic drift of the entire facade if only one unit is considered. Therefore, it is necessary to average the normal offset of all complete repeating units along the tangential direction to form a normal brightness offset field. This averaging operation is specifically used to extract the dominant global parallax drift baseline, while the small local residual nonlinear deviations caused by perspective distortion are absorbed and accommodated by the subsequent gradient domain integral optimization step. In this way, what is obtained is not an accidental local error, but a periodic unit normal brightness centroid drift field with global representativeness.

[0074] In detail, because the change in the position of the main brightness under dual-angle shooting is affected by both directional reflection migration and the self-occlusion rotation of the stage structure, the difference in the centroid of unit brightness is not random noise, but a systematic shift with a clear physical source. Therefore, only by including the lateral slip term of the brightness peak and the change term of the visible range in the explanation can we explain why the drift mainly occurs along the normal direction and why we should model and compensate for the normal brightness shift field.

[0075] In detail, the solution for when the denominator of the luminance centroid is close to zero is as follows: When performing the division operation of the luminance centroid, an extremely small positive constant is implicitly superimposed on the denominator as a regularization constraint to prevent division by zero; at the same time, it is first determined whether the sum of positive luminance deviations in the cell corresponding to a certain tangential position is less than the preset lower limit of numerical stability; if so, one to two adjacent scan lines are extended upwards and downwards along the tangential direction to form a local narrow strip region, and then the positive luminance deviation and luminance centroid are recalculated; if the sum of positive deviations is still too small after the extension, the midpoint of the cell is directly taken as the luminance centroid of the tangential position; at the same time, the position is marked as a low confidence position, and its contribution weight is reduced when averaging subsequent cells.

[0076] In detail, the threshold-free smoothing method for the normal brightness offset field is as follows: First, the normal offsets of each complete repeating element are arranged into a discrete sequence along the tangential direction; then, a variational objective consisting of a fidelity term and a first-order smoothing term is constructed; the fidelity term ensures that the smoothed result does not deviate from the average offset of the original elements, and the smoothing term constrains the rate of change of adjacent tangential positions to not be too drastic; then, the corresponding linear equation system is solved to obtain a smooth offset field that changes continuously along the tangential direction; the entire process does not rely on empirical threshold truncation, but automatically obtains the balanced result through the joint solution of the data term and the smoothing term.

[0077] In detail, the total number of complete repeating units is determined as follows: First, all theoretical repeating units are generated in the common effective area of ​​the two images; then, each unit is checked to see if it meets three conditions simultaneously: both left and right boundaries fall within the common effective area, there are effective sampled pixels inside the unit, and effective brightness statistics can be calculated for both images within the unit; a unit that meets the above three conditions is counted as one complete repeating unit; the sum of the number of units that meet all the conditions is the total number of complete repeating units.

[0078] Preferably, the normal position of the image to be stitched is compensated using the normal brightness offset field to obtain the compensated image to be stitched, including: Combining the normal coordinates of the elevation coordinate system, the periodic start phase, and the cell spacing, the periodic spatial weight of the image to be stitched within the repeating cells is calculated. The periodic spatial weight is multiplied by the normal brightness offset field to obtain the local normal displacement of the image to be stitched. This local normal displacement is then superimposed on the normal coordinates of the image to be stitched to obtain the remapped normal coordinates. The remapped normal coordinates and the tangential coordinates of the elevation coordinate system are used to resample the image to be stitched, resulting in the compensated image to be stitched. The calculation formula is as follows: in, This indicates that the compensated image to be stitched is located in the normal coordinates. and the tangential coordinates Image pixel values ​​at that location; This represents the pixel value resampling function of the image to be stitched together; Represents the normal coordinates of the elevation coordinate system; This represents the tangential coordinates of the elevation coordinate system; Represents the tangential coordinates The normal brightness offset field at the location; Represents the sine function; Pi is a constant. Indicates the starting phase of the period; Indicates the unit spacing; This represents the periodic spatial weights; This represents the remapped normal coordinates.

[0079] The compensated image to be stitched is the result of remapping the normal position of the image to be stitched according to the normal brightness offset field.

[0080] Periodic spatial weights are spatial modulation functions that vary according to the periodic position of the cell. Periodic spatial weights suppress displacement at the cell boundary and enhance displacement at the cell center.

[0081] The local normal displacement is a single-pixel normal compensation displacement determined by the normal brightness offset field and the periodic spatial weight.

[0082] The remapped normal coordinates are the new normal sampling positions obtained by superimposing the original normal coordinates with the local normal displacement, and are used to perform reverse sampling on the images to be stitched.

[0083] The sine function is a periodic function used to construct periodic spatial weights. It achieves a smooth displacement distribution with zero values ​​at the boundaries and enhanced values ​​at the center through a squared form.

[0084] In detail, since the true geometric boundaries of a periodic facade are usually determined by the keel, pressure plate, or louver edge, the positions of these boundaries need to remain visually continuous and stable. Since the drift of the main brightness peak mainly occurs inside the unit, the compensation cannot be a rigid translation of the entire unit. Instead, normal reparameterization should be adopted to change with the periodic position, so that the displacement at the unit boundary decays to zero and the displacement at the unit center gradually increases. Therefore, it can correct the misalignment of the internal brightness peak without destroying the geometric continuity of the boundaries of adjacent units.

[0085] In detail, the non-integer displacement resampling method is as follows: First, the remapped normal coordinates and corresponding tangential coordinates are calculated pixel by pixel on the output image grid; since the remapped normal coordinates are usually floating-point positions, the inverse sampling method is used to return to the image to be stitched before compensation to obtain the values; bicubic interpolation or bilinear interpolation is used during interpolation; in the preferred implementation, bicubic interpolation is used in order to reduce the jaggedness and blur caused by subpixel displacement while maintaining edge continuity.

[0086] In detail, the out-of-bounds resampling process is as follows: when the reverse sampling position falls outside the image boundary, the pixel is not directly set to 0, nor is it kept as a null value. Instead, it is filled by copying the nearest valid boundary or by mirroring the edge. In the preferred implementation, the nearest valid boundary copying method is used to avoid forming black edges at the image edge and affecting the calculation of the subsequent stitching cost map.

[0087] Preferably, a stitching cost map is constructed using the normal brightness offset field, and an optimal stitching seam with edge avoidance is generated in the stitching cost map, including: Calculate the absolute pixel difference and the gradient norm 2 difference between the reference image and the compensated image to be stitched, and add the absolute pixel difference and the gradient norm 2 difference to obtain the image residual map. The calculation formula is as follows: in, This represents the normal coordinates of the image residual map in the elevation coordinate system. and tangential coordinates The value at; The reference image is represented in the normal coordinates. and the tangential coordinates Image pixel values ​​at that location; This indicates that the compensated image to be stitched is located in the normal coordinates. and the tangential coordinates Image pixel values ​​at that location; This represents the absolute value operator; Represents the image gradient operator; Represents the vector 2-norm operator; Based on the absolute value of the normal brightness offset field, the cell spacing, and the periodic spatial weight, a weight adjustment factor is calculated for each pixel. Then, the weight adjustment factor is used to perform a weighted product operation on the image residual map to obtain the stitching cost map. The calculation formula is as follows: in, The stitching cost map is represented in the normal coordinates. and the tangential coordinates The value at; Represents the tangential coordinates The normal brightness offset field at the location; This represents the absolute value of the normal brightness offset field; Indicates the unit spacing; Indicates the starting phase of the period; Pi is a constant. This represents the periodic spatial weights; This represents the weight adjustment factor; The splicing seam curve is extracted within the overlapping area of ​​the splicing cost map. A directional penalty constraint is applied to the unit vector of the tangential direction of the splicing seam curve using the principal tangential direction of the facade. Based on this directional penalty constraint, the splicing cost map is integrally optimized along the splicing seam curve. The curve that minimizes the global integral cost is identified as the optimal splicing seam for edge avoidance. The calculation formula is as follows: in, This indicates the optimal seam for edge avoidance; This represents the curve of the splice seam; Indicates the curve path variable; This represents the curve integral operator along the splice seam curve; Indicates the location of the splice seam curve on the path. The cost of splicing at the location; The unit vector representing the tangential direction of the splice seam curve; Indicates the principal tangent of the facade; This represents the vector dot product operator; This indicates the directional penalty constraint; This represents the variable-value operator that minimizes the objective functional.

[0088] An image residual map is a pixel-level inconsistency measure formed between a reference image and a compensated image to be stitched together under the combined effects of brightness difference and gradient difference.

[0089] The pixel absolute difference is the absolute difference in brightness between two images at the same location, and is used to measure inconsistency in direct radiation.

[0090] The gradient 2 norm difference is the difference between the gradient vectors of two images at the same location, used to measure the inconsistency between edge and texture structure. When calculating and adding it to the absolute difference of pixels, the gradient 2 norm difference is implicitly assigned a unit transformation length weight equal to the side length of a single pixel, so that the two terms are unified to the luminance dimension before being summed.

[0091] The absolute value operator is an operator that converts scalar differences into non-negative magnitudes.

[0092] The vector 2-norm operator is a length operation operator used to calculate the overall magnitude of the vector difference.

[0093] The absolute value of the normal brightness offset field is the magnitude of the drift intensity of the normal brightness offset field at a certain tangential position, which is used to adjust the degree of splicing cost penalty.

[0094] The weight adjustment factor is a cost enhancement factor composed of drift intensity, cell spacing, and periodic spatial weight, used to increase the splicing cost in high-drift regions.

[0095] The stitching cost map is a path search cost field formed by adjusting the weights of the image residual map. It is used to guide the stitching seams to avoid high-risk areas.

[0096] The seam curve is a candidate path curve that traverses the overlapping region and separates the contributing regions of the two images.

[0097] The optimal seam for edge avoidance is the optimal path obtained under the condition of minimizing the total cost. The optimal seam for edge avoidance should try to avoid areas with high residuals, high drift and unfavorable directions.

[0098] Curve path variables are path location variables used to parametrically represent the splice seam curve.

[0099] The curve integral operator along the splice seam curve is a path integral operation used to accumulate the total cost of the entire splice seam path.

[0100] The unit vector of the tangential direction of the curve is a unit directional quantity that describes the direction of extension of the seam at the current position.

[0101] The vector dot product operator is an operation used to measure the angular relationship between two directional vectors.

[0102] Directional penalty constraints are additional costs imposed based on the angle between the splice joint direction and the principal tangent of the facade, used to prevent the splice joint from crossing the main structural direction.

[0103] The variable-valued operator that minimizes the objective functional is the optimal-valued operator used to return the path with the minimum total cost.

[0104] In detail, because high drift areas are often the locations with the strongest reflection, the most obvious occlusion rotation, and the most prone to incorrect stitching, if the stitching seam cost is determined only by the brightness difference and gradient difference, it may still pass through the center of the unit with the highest visual risk. Therefore, it is necessary to directly write the normal brightness offset field intensity into the cost map so that the larger the drift area, the higher the cost. Thus, the optimal path will actively avoid these high-risk areas.

[0105] In detail, since the main structure of a periodic facade usually extends continuously along the main tangent of the facade, and transverse crossing of these main structures is more likely to cause window pane breakage, grid misalignment and boundary flickering, additional penalties should be imposed on paths that deviate from the main tangent of the facade in the splice optimization. Therefore, the optimal splice will tend to extend along the structural direction, thereby reducing damage to significant structural boundaries.

[0106] In detail, the discrete solution for the splicing seam is as follows: First, treat each effective pixel in the overlapping area as a graph node, and then establish the connection between nodes according to the 8-neighborhood; the edge weight of each edge is obtained by multiplying the average splicing cost of the two adjacent nodes by the corresponding step size; then perform the shortest path search between the starting boundary and the ending boundary; if the overlapping area mainly extends in the vertical direction, then use dynamic programming to accumulate the minimum cost row by row; if the shape of the overlapping area is more complex, then use graph search to solve the global minimum cost path; finally, return the discrete splicing seam with the minimum total cost.

[0107] In detail, the method for setting the start and end boundary conditions of the stitching seam is as follows: First, determine whether the stitching seam should run from the upper boundary to the lower boundary or from the left boundary to the right boundary based on the long axis direction of the overlapping area; then, regard all valid pixels on the corresponding two boundaries as candidate start or candidate end points; subsequently, calculate the minimum total path cost under each set of start and end point combinations; finally, select the set of boundary points with the minimum total cost as the final start and end points of the stitching seam, instead of manually specifying fixed pixel positions.

[0108] Preferably, taking the repeating unit as a unit, the compensated image to be stitched and the reference image are subjected to unit-level exposure correction and image fusion along the optimal stitching seam to obtain an initial stitched image, including: The mean and standard deviation of the unit images in the repeating units of the reference image and the compensated image to be stitched are calculated respectively, using the following formula: in, Indicates the first The image in the first The mean value of the unit image within each repeating unit, when When refers to the reference image, when The time indicates the compensated image to be stitched together; This represents the standard deviation of the corresponding unit image; Indicates the first The integration region of repeating units; This represents the area of ​​the integration region; Indicates the first The pixel values ​​of the image; Represents the normal coordinates; Indicates tangential coordinates; The unit-level linear correction coefficient is obtained by using the mean and standard deviation of the unit images. This coefficient is then used to correct the compensated images to be stitched together, resulting in an exposure-corrected image. The calculation formula is as follows: in, and Together they constitute the unit-level linear correction coefficients; The unit image standard deviation of the reference image; This represents the standard deviation of the unit images in the compensated image to be stitched together; This represents the mean value of the unit image of the reference image; This represents the mean value of the unit images in the compensated image to be stitched together; This refers to the exposure-corrected image; This represents the compensated image to be stitched together; Calculate the distance from each pixel to the regions on both sides of the optimal stitching seam for edge avoidance, obtain the reference side distance and the distance to be stitched side, and use the reference side distance and the distance to be stitched side distance to calculate the distance field fusion weight. The calculation formula is as follows: in, and Represents the corresponding image pixel. The distance field fusion weights; Indicates the distance to the reference side; Indicates the distance of the side to be spliced; The initial stitched image is obtained by integrating the image gradients of the reference image and the exposure-corrected image using the distance field fusion weights. The calculation formula is as follows: in, This refers to the initial stitched image; Represents the integral variable of the fused image; Indicates the global integration region; Represents the image gradient operator; This refers to the reference image; This refers to the exposure-corrected image; The operator for squaring the L2 norm of a vector; This represents the image variable operator that minimizes the objective functional.

[0109] The cell image mean is the average brightness statistic of an image within a certain repeating cell, used to describe the overall brightness level of that cell.

[0110] The standard deviation of a cell image is a statistical measure of the brightness dispersion within a specific repeating cell of an image, used to describe the contrast level of that cell.

[0111] The area of ​​the repeating unit integration region is the area of ​​the region corresponding to a certain repeating unit, which is used to normalize the brightness statistics within the unit.

[0112] The unit-level linear correction coefficient is a linear parameter that corrects the compensated image to be stitched to the statistical level of the reference image at the unit level. One coefficient is used for contrast scaling, and the other coefficient is used for brightness shifting. When performing specific point-by-point correction calculations on image pixels, spatial bilinear smooth interpolation transition is required using the correction coefficients of adjacent repeating units to eliminate possible brightness jumps at the boundaries of adjacent units.

[0113] An exposure-corrected image is a brightness-uniformed image obtained by performing unit-level linear radiometric correction on the compensated images to be stitched together.

[0114] The reference side distance and the side distance to be stitched are distance metrics from a pixel to the boundaries of the regions on both sides of the stitching seam, used to construct fusion weights for a continuous transition.

[0115] Distance field fusion weights are weighting coefficients automatically calculated based on the distance from pixels to both sides of the stitching seam, used to control the fusion contribution ratio of two images at different locations.

[0116] The initial stitched image is the stitched result after the first completion of compensation, stitch seam selection, exposure correction and gradient domain fusion. It serves as the input image for subsequent residual backflow correction.

[0117] The fused image integral variable is the fused image variable to be solved in gradient domain optimization.

[0118] The vector 2-norm square operator is a square length operator used to measure the overall energy of the gradient difference.

[0119] In detail, because the reflection state and brightness distribution of a periodic facade are not completely consistent in different units, performing global exposure compensation only once for the entire image often results in insufficient correction in some units and overcorrection in others. Therefore, the repeating unit should be used as the smallest unit of radiation uniformity, and the mean and standard deviation of each unit should be matched separately, so as to better reflect the true brightness and contrast state of the local unit.

[0120] In detail, since the image contributions on both sides of the stitching seam should change continuously with the distance from the stitching seam rather than switch abruptly, it is necessary to first construct smooth weights based on the distance from the pixels to the regions on both sides of the stitching seam, and then bring the weights into the gradient domain for fusion. This can maintain a natural transition near the stitching seam and preserve the local texture and edge continuity of the images on both sides.

[0121] In detail, the definition of the linear correction coefficient when the standard deviation of the image unit to be stitched after compensation is zero is as follows: First, determine whether all pixels in the unit are approximately always bright or always dark; if the standard deviation is zero, set the contrast scaling factor to 1 and only perform brightness shift correction; the brightness shift is the difference between the mean of the reference image unit and the mean of the image unit to be stitched after compensation; this can avoid division by zero error and at the same time keep the original structure of the unit from being abnormally stretched.

[0122] In detail, the discrete solution for the reference side distance and the side distance to be stitched is as follows: First, the overlapping area is divided into the reference image contribution area and the side distance to be stitched using the optimal stitching seam; then, Euclidean distance transformation is performed on the two areas respectively; for any pixel, the shortest Euclidean distance to the boundary of the reference side area is recorded as the reference side distance, and the shortest Euclidean distance to the boundary of the side area to be stitched is recorded as the side distance to be stitched; finally, normalized fusion weights are constructed using these two distances.

[0123] In detail, the configuration of the gradient domain fusion discrete solver is as follows: Considering that the target gradient field after the distance field weight mixing usually exhibits non-conservative field characteristics, the fusion target is first written as a discrete Poisson equation, and then projected and solved to obtain the scalar image solution with the smallest difference from the mixed non-conservative field to avoid smearing defects; then, a five-point difference linear equation system is established in the entire global integration region; the boundary conditions adopt the fixed value of the outer boundary of the reference image or the overall brightness preservation constraint; the conjugate gradient method or the discrete sine transform method is used for solving; in the preferred implementation, the conjugate gradient method is used, and the maximum number of iterations and the convergence residual threshold are set to ensure stable convergence on larger images.

[0124] Preferably, the centroid residual of the initial stitched image is calculated to update the normal brightness offset field. The updated normal brightness offset field is then used to re-perform the normal position compensation on the image to be stitched, and the optimal stitching seam is regenerated. Unit-level exposure correction and image fusion are then performed to output the final stitched image, including: Calculate the average brightness of the compensated cells in the repeating cells of the image to be stitched. Based on the average brightness of the compensated cells, extract the positive brightness deviation portion and recalculate the centroid of the compensated brightness of the image to be stitched. The calculation formula is as follows: in, This indicates that the compensated image to be stitched is at the [number]th [year]. The average brightness of the compensated unit within each repeating unit; Indicates the unit spacing; Indicates the first The initial cell boundary of a repeating cell; Indicates the first The boundary of the terminating unit of a repeating unit. This indicates that the compensated image to be stitched is in the normal coordinate system. and tangential coordinates Image pixel values ​​at that location; This indicates that the compensated image to be stitched is at the [number]th [year]. The compensated luminance centroid within each repeating unit; This indicates the operator for taking positive values; The difference between the compensated luminance centroid and the luminance centroid of the reference image is calculated, and the average value is taken for the multiple repeating units to obtain the centroid residual. The centroid residual is then superimposed on the normal luminance offset field to obtain the updated normal luminance offset field. The calculation formula is as follows: in, This represents the centroid residual; This represents the total number of the plurality of repeating units; This represents the luminance centroid of the reference image; This represents the normal brightness offset field; This represents the updated normal brightness offset field; The updated normal brightness offset field is used to perform normal position compensation, optimal stitching seam generation for edge avoidance, and unit-level exposure correction and image fusion on the image to be stitched once to obtain the final stitched image. The calculation formula is as follows: in, This refers to the final stitched image; This indicates the fusion and stitching processing operator, which utilizes the updated normal brightness offset field. The operations of normal position compensation, optimal stitching seam generation for edge avoidance, and unit-level exposure correction and image fusion are performed sequentially. This refers to the reference image; This refers to the image to be stitched together.

[0125] The average brightness of the compensated cell is the average brightness baseline obtained by recalculating the image to be stitched along the normal direction in a certain repeating cell, and is used for secondary centroid measurement.

[0126] The compensated brightness centroid is the position of the normal centroid of the main contributing region of the image to be stitched, which is recalculated within a certain repeating cell after compensation.

[0127] Centroid residual is the average normal difference between the compensated luminance centroid and the luminance centroid of the reference image, used to characterize the systematic error remaining after the first compensation.

[0128] The updated normal brightness offset field is a corrected offset field obtained by superimposing the centroid residual on the original normal brightness offset field, and is used to perform a fixed-loop recompensation once.

[0129] The final stitched image is the final output image obtained after completing a fixed residual reflux correction.

[0130] The fusion and stitching processing operator is a composite operator used to perform a complete set of processing steps, including normal position compensation, optimal stitching seam generation, and unit-level exposure correction and fusion.

[0131] In detail, because a small amount of system centroid deviation may still remain after the first normal position compensation, stitching seam generation and image fusion, it is necessary to remeasure the brightness centroid of the compensated unit and calculate the residual after the first fusion. Then, the residual is fed back into the original offset field to form the updated offset field. Therefore, a fixed closed-loop correction is adopted instead of relying on empirical thresholds to determine whether to continue iterating. This preserves the closed-loop correction capability and avoids unstable stopping conditions.

[0132] In detail, the residual backflow execution sequence is as follows: First, the average brightness of the compensated unit and the brightness centroid of the compensated unit are recalculated using the image to be stitched after the first compensation as input; then, the difference between the brightness centroid of the compensated unit and the brightness centroid of the reference image is calculated unit by unit and averaged along the tangential direction to obtain the centroid residual; then, the centroid residual is superimposed on the original normal brightness offset field to form the updated normal brightness offset field; finally, the normal position compensation, optimal stitching seam generation, unit-level exposure correction, and image fusion are redone only with the updated offset field.

[0133] In detail, the method of reflowing only once is as follows: first, the residual system error after the first fusion is regarded as a closed-loop correction amount, and then the normal brightness offset field is directly updated with this correction amount; after the update is completed, the loop iteration is no longer continued, but a complete refusion is immediately performed and the result is output; this can avoid introducing additional stopping thresholds and unstable cumulative errors from multiple iterations.

[0134] In detail, the final output object is determined as follows: First, the image obtained from the first fusion is saved as an intermediate result for calculating the centroid residual; then, the subsequent compensation and fusion process is redone with the updated normal brightness offset field as the fix; the result obtained after the redo is defined as the final stitched image; when outputting externally, only the final stitched image is output, and the intermediate result obtained from the first fusion is not output.

[0135] It should be noted that the interval and threshold sizes are set for ease of comparison. The size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameter and the quantized value. Furthermore, the above formulas are all dimensionless calculations, and the formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0136] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) for multi-measurement integration, characterized in that, include: Extract the main elevation direction of the reference image and the image to be stitched and perform coordinate mapping to unify the reference image and the image to be stitched to the elevation coordinate system; The unit spacing is adaptively calculated within the elevation coordinate system, and multiple repeating units are divided based on the unit spacing. Extract the luminance centroids of the reference image and the image to be stitched within the repeating unit, and obtain the normal luminance offset field between the repeating units; The normal position of the image to be stitched is compensated using the normal brightness offset field to obtain the compensated image to be stitched. A stitching cost map is constructed using the normal brightness offset field, and an optimal stitching seam with edge avoidance is generated in the stitching cost map; Using the repeating unit as a unit, the compensated image to be stitched and the reference image are subjected to unit-level exposure correction and image fusion along the optimal stitching seam that is avoided by the edge to obtain the initial stitched image; The centroid residual of the initial stitched image is calculated and superimposed onto the normal brightness offset field to obtain the updated normal brightness offset field. The updated normal brightness offset field is then used to re-perform the normal position compensation on the image to be stitched, and the optimal stitching seam for edge avoidance is regenerated. The unit-level exposure correction and image fusion are then performed to output the final stitched image.

2. The method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) oriented towards multi-measurement integration as described in claim 1, characterized in that, Extracting the main elevation directions of the reference image and the image to be stitched and performing coordinate mapping to unify the reference image and the image to be stitched to the elevation coordinate system includes: The image power spectra of the reference image and the image to be stitched are calculated respectively, and the corresponding image smoothing scale is adaptively obtained based on the image power spectra; The image structure tensors of the reference image and the image to be stitched are calculated using the image smoothing scale. By fusing the image structure tensor of the reference image and the image to be stitched together, the main orientation of the facade is extracted. The main orientation of the facade includes the main tangent and the main normal of the facade, so as to construct the tangent coordinates and normal coordinates of the facade coordinate system accordingly. Extract the facade structure lines from the reference image and the image to be stitched together, and solve the initial projection transformation matrix based on the facade structure lines; The initial projection transformation matrix is ​​used to perform a projection transformation on the image to be stitched, and the principal tangent and principal normal of the facade are used to perform a coordinate transformation to unify the reference image and the image to be stitched into the facade coordinate system.

3. The method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) oriented towards multi-measurement integration as described in claim 2, characterized in that, Within the elevation coordinate system, the element spacing is adaptively calculated, and multiple repeating elements are divided based on the element spacing, including: The image gradients of the reference image and the image to be stitched are extracted in the elevation coordinate system, and the image gradients are projected onto the main normal of the elevation to construct the normal structure projection profiles of the reference image and the image to be stitched. Fourier transforms are performed on the normal structural projection profiles of the reference image and the image to be stitched to obtain the profile spectrum, and the profile spectrum centroid is calculated. The spacing between individual image units of the reference image and the image to be stitched is calculated based on the centroid of the cross-sectional spectrum, and the average value of the individual image unit spacing is obtained to obtain the unit spacing. By fusing the normal structural projection profiles of the reference image and the image to be stitched together, and using the cell spacing to maximize the cosine response, the periodic initial phase is obtained. The cell boundary is determined based on the periodic start phase and the cell spacing, and the multiple repeating cells are divided using the cell boundary.

4. The method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) oriented towards multi-measurement integration as described in claim 3, characterized in that, Extracting the luminance centroids of the reference image and the image to be stitched within the repeating units, and obtaining the normal luminance offset field between the repeating units, includes: Calculate the average brightness of the reference image and the image to be stitched within the repeating unit, respectively; Based on the average brightness of the unit, the positive brightness deviation portions of the reference image and the image to be stitched are respectively extracted from the repeating unit where the brightness is higher than the average brightness of the unit; Using the positive brightness deviation portion, the brightness centroids of the reference image and the image to be stitched are calculated within the repeating unit, respectively. The difference between the brightness centroid of the image to be stitched and the brightness centroid of the reference image is calculated to obtain the cell normal offset corresponding to each repeating cell; The average value of the normal offset of the multiple repeating units is calculated to obtain the normal brightness offset field between the repeating units.

5. The method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) oriented towards multi-measurement integration as described in claim 4, characterized in that, The normal position of the image to be stitched is compensated using the normal brightness offset field to obtain the compensated image to be stitched, including: By combining the normal coordinates of the elevation coordinate system, the periodic start phase, and the unit spacing, the periodic spatial weight of the image to be stitched is calculated within the repeating unit, wherein the periodic spatial weight is zero at the unit boundary of the repeating unit and is at its maximum value at the center of the repeating unit. The periodic spatial weights are multiplied by the normal brightness offset field to obtain the local normal displacement of the image to be stitched. The local normal displacement is superimposed on the normal coordinates of the image to be stitched to obtain the remapped normal coordinates; Using the remapped normal coordinates and the tangential coordinates of the elevation coordinate system, the image to be stitched is resampled pixel by pixel to obtain the compensated image to be stitched.

6. The method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) oriented towards multi-measurement integration as described in claim 5, characterized in that, A stitching cost map is constructed using the normal brightness offset field, and an optimal stitching seam with edge avoidance is generated in the stitching cost map, including: Calculate the absolute pixel difference and the gradient norm difference between the reference image and the compensated image to be stitched, and add the absolute pixel difference and the gradient norm difference to obtain the image residual map; Based on the absolute value of the normal brightness offset field, the cell spacing, and the periodic spatial weight, the weight adjustment factor for each pixel is calculated. The image residual map is weighted and multiplied using the weight adjustment factor to obtain the stitching cost map. In the stitching cost map, the stitching seam curve is extracted in the overlapping area of ​​the reference image and the image to be stitched, and the directional penalty constraint is applied to the unit vector of the tangential direction of the stitching seam curve using the main tangential direction of the facade. Based on the directional penalty constraint, the splicing cost map is integrally optimized along the splicing seam curve to find the curve that minimizes the global integral cost, which is then used as the optimal splicing seam for edge avoidance.

7. The method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) oriented towards multi-measurement integration as described in claim 6, characterized in that, Using the repeating unit as a unit, the compensated image to be stitched and the reference image are subjected to unit-level exposure correction and image fusion along the optimal stitching seam that avoids the edge, to obtain an initial stitched image, including: Calculate the mean and standard deviation of the unit images in the repeating unit for both the reference image and the compensated image to be stitched. The unit-level linear correction coefficient is obtained by using the mean and standard deviation of the unit image, and the compensated image to be stitched is corrected to obtain the exposure correction image. Calculate the distance from the pixel to the two sides of the optimal stitching seam where the edge is avoided, and obtain the reference side distance and the side distance to be stitched. The distance field fusion weight is calculated using the reference side distance and the side distance to be stitched. The initial stitched image is obtained by integrating the image gradients of the reference image and the exposure-corrected image using the distance field fusion weights.

8. The method for low-altitude digital acquisition and data fusion of unmanned aerial vehicles (UAVs) oriented towards multi-measurement integration as described in claim 7, characterized in that, The centroid residual of the initial stitched image is calculated and superimposed onto the normal brightness offset field to obtain an updated normal brightness offset field. The updated normal brightness offset field is then used to re-perform the normal position compensation on the image to be stitched, and the optimal stitching seam for edge avoidance is regenerated. Unit-level exposure correction and image fusion are then performed to output the final stitched image, including: Calculate the average brightness of the compensated cells in the repeating cells of the image to be stitched; Based on the average brightness of the compensated unit, the positive deviation of the compensated brightness is extracted, and the centroid of the compensated brightness of the image to be stitched is recalculated. The difference between the compensated luminance centroid and the luminance centroid of the reference image is calculated, and the average value of the multiple repeating units is obtained to obtain the centroid residual. The centroid residual is superimposed onto the normal brightness offset field to obtain the updated normal brightness offset field; The updated normal brightness offset field is used to perform normal position compensation, optimal stitching seam generation for edge avoidance, and unit-level exposure correction and image fusion on the image to be stitched once to obtain the final stitched image.

Images