Large-scale satellite image automatic registration and enhancement method based on adaptive multi-source fusion

Abstract

The application provides a large-scale satellite image automatic registration and enhancement method based on adaptive multi-source fusion, comprising: performing orthorectification and multi-resolution unified projection on a multi-source original data set obtained to obtain a coarse registration image set; calculating a local tensor structure of the coarse registration image set, performing repeated texture recognition, generating a final structure representation and a structure reliability; performing regional registration to obtain a full-image dense deformation field, then performing image transformation to be aligned and registration uncertainty estimation to obtain a high-precision registration image and structure registration uncertainty estimation; combining a pseudo-change probability of the high-precision registration image and the structure registration uncertainty estimation to obtain a change reliability weight; under the constraint of the change reliability weight, performing consistency enhancement on the high-precision registration image to output a quantitative and reliable enhanced image. The application realizes high-precision, low-false alarm and quantifiable and reliable automatic registration and enhancement of multi-source large-scale satellite images.

Description

Technical Field

[0001] This invention relates to the field of automatic satellite image registration technology, and in particular to a method for automatic registration and enhancement of large-scale satellite images based on adaptive multi-source fusion. Background Technology

[0002] Satellite imagery is remote sensing data obtained by sensors on Earth observation satellites imaging the Earth's surface. It typically contains information such as time, sensor type, imaging geometry (attitude or orbit), and geographic coordinate reference. Registration and enhancement of satellite imagery refers to aligning two or more images (from different times, different sensors, different viewpoints, and different resolutions) to the same spatial reference through geometric transformations. This ensures that the same ground feature appears in the same location (consistent pixels or latitude and longitude) in different images, and then processes the imagery to improve visual readability or algorithmic usability.

[0003] Registration and enhancement of satellite imagery enable reliable comparisons for disaster assessment, illegal construction monitoring, and agricultural changes; map and surveying; and analysis of crop growth curves and urban heat island evolution. However, existing satellite imagery registration and enhancement processes suffer from the following problems: 1) Direct registration of multi-source images using grayscale or single features is prone to failure across modalities: significant differences between optical and SAR radiometric mechanisms lead to unstable grayscale correlation and conventional feature point matching, resulting in mismatches; 2) Single-strategy registration across the entire map is not robust to weak textures, repetitive textures, and strong terrain, resulting in mismatches in repetitive texture areas, no matching in weak texture areas, and large residuals due to local non-rigidity in strong terrain areas; 3) Change detection is susceptible to false alarms caused by differences in cloud cover, shadows, snow, phenological variations, sensor differences, and residual registration errors, mistaking differences in imaging conditions or occlusion for changes in ground features; or minor registration errors may cause pseudo-changes at edges. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide an automatic registration and enhancement method for large-scale satellite images based on adaptive multi-source fusion, which achieves high-precision, low-false-alarm, quantifiable and reliable automatic registration and enhancement of large-scale satellite images from multiple sources.

[0005] To achieve the above objectives, the present invention provides the following solution: a method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion, comprising: By acquiring the original image set, prior geometric and physical data, and prior information on radiation and imaging conditions, a multi-source original data set is obtained. Based on the multi-source raw data set, orthorectification and multi-resolution unified projection are performed to obtain a physically consistent coarsely registered image set. For the coarsely registered image set, input processing, multi-scale gradient calculation, local smoothing, and feature value calculation are performed to obtain the local tensor structure. Based on the local structure tensor, repetitive texture recognition is performed, and the final structure representation and structure confidence are generated. Based on the coarsely registered image set, regional registration is performed using the final structural representation and structural reliability to obtain a dense deformation field of the whole image. Then, the image to be aligned is transformed and the registration uncertainty is estimated based on the dense deformation field of the whole image to obtain a high-precision registered image and a structural registration uncertainty estimate. Candidate masks are generated and pseudo-changes are detected for the high-precision registered image to obtain the pseudo-change probability. The pseudo-change probability and the structural registration uncertainty estimate are fused to obtain the change confidence weight. Under the constraint of the changing confidence weights, the high-precision registered image is enhanced for consistency and finally detected using the constructed enhancement target total loss function, and then the quantitatively reliable enhanced image and the high-precision registered image are output.

[0006] Optionally, the original image set, geometric and physical prior data, and prior information on radiation and imaging conditions are acquired to obtain a multi-source original data set, including: Image retrieval is performed according to the same region, the same time window, and multiple source types to obtain optical images and SAR images, resulting in a raw image set. The raw image set is then subjected to initial quality screening and download to obtain the raw image set and image metadata. The image metadata includes coverage, imaging time, resolution, cloud cover estimation, and solar elevation angle. For the original image set, the acquired rational polynomial coefficients, orbital attitude information and DEM data are written into a unified structure or configuration file to obtain geometric and physical prior data. By acquiring solar elevation angle, solar azimuth angle, sensor spectral response function, and recorded radiometric calibration and atmospheric correction states, prior information on radiation and imaging conditions can be obtained. The original image set, the image metadata, the geometric and physical prior data, and the prior information on radiation and imaging conditions are organized into a multi-source original data set.

[0007] Optionally, based on the multi-source raw data set, orthorectification and multi-resolution unified projection are performed to obtain a physically consistent coarsely registered image set, including: A unified map coordinate system is set, the pixel resolution of the output grid is defined, and the output image coverage is obtained to obtain a unified target grid; Based on the multi-source raw data set, for the rational polynomial coefficients, a forward mapping from geographic coordinates to pixel coordinates and a reverse mapping from pixel coordinates to geographic coordinates are constructed to obtain the RPC projection relationship and the back projection relationship. According to the projection relationship and the back projection relationship, pixel-by-pixel back projection is performed on the target grid to obtain the orthorectified image. The orthorectified image is resampled using bilinear interpolation, and the resampling method is recorded in the image metadata. During the resampling process, the DEM missing mask, the RPC back projection area mask falling outside the original image, and the potential error are calculated to obtain the orthorectified projection and orthorectified quality label. Based on the orthophoto projection, target resolution and boundary unification and image pyramid construction are performed, outputting consistent resolution, band combination, image size and boundary, resulting in a physically consistent coarsely registered image set.

[0008] Optionally, for the coarsely registered image set, input processing, multi-scale gradient calculation, local smoothing, and eigenvalue calculation are performed to obtain the local tensor structure, including: Select a reference image from the coarse registration image set and set the remaining images as images to be aligned. For the optical multispectral images in the images to be aligned, the visible light bands are weighted and summed to generate a brightness composite image. For the SAR images in the images to be aligned, the amplitude or backscattering intensity is converted to the logarithmic domain. Then, the brightness composite image is robustly normalized to obtain the image structure input. For the image structure input, Gaussian smoothing is used to smooth the image at different scales. Based on the image smoothing results, the derivatives in the horizontal and vertical directions are calculated to obtain the multi-scale gradient, gradient magnitude and gradient direction. Based on the derivatives in the horizontal and vertical directions, an original structure tensor matrix is ​​constructed. Each element in the original structure tensor matrix is ​​locally smoothed to obtain a smoothed structure tensor. Then, two eigenvalues ​​of the smoothed structure tensor are calculated to obtain the first eigenvalue and the second eigenvalue. By comparing the magnitudes of the first eigenvalue and the second eigenvalue, the structural strength and structural type are obtained. The first eigenvalue and the second eigenvalue are then weighted and fused to output the structural saliency. The structural strength, the structural type, and the structural saliency are then incorporated into the original structural tensor matrix to obtain the local structural tensor.

[0009] Optionally, based on the local structure tensor, repeating texture recognition is performed, and a final structure representation and structure confidence are generated, including: Based on the local structure tensor, the proportion of strong edges is counted within a fixed window, and the directional distribution within the fixed window is counted according to the multi-scale gradient direction histogram or the structure direction histogram to obtain the local structure entropy. Combining the proportion of strong edges, the structural saliency, and the local structure entropy, repeating texture recognition is performed to complete the structural reliability judgment. By fusing the multi-scale gradient, the local structure tensor, and the strong edge ratio, a final structural representation is obtained. Then, the structural reliability of the final structural representation is calculated using a defined multi-index gating method. The calculation expression for the multi-index gating is as follows: ; in, For structural credibility, For the Sigmoid function, This is the normalized value of the structural significance. This is the normalized value for the proportion of strong edges. To mitigate the risk of duplicate textures, , , These are the weighting coefficients.

[0010] Optionally, based on the coarsely registered image set, regional registration is performed using the final structural representation and structural reliability to obtain a dense deformation field across the entire image. Then, based on the dense deformation field, the image to be aligned is transformed and registration uncertainty is estimated to obtain a high-precision registered image and structural registration uncertainty estimate, including: Based on the coarsely registered image set, using the final structural representation and structural confidence, the structural confidence of the reference image and the image to be aligned is calculated at each pixel location to obtain joint confidence and structural type labels. The slope or elevation change rate is calculated using the DEM to obtain terrain risk. Then, based on the joint confidence, the structural type labels, and the terrain risk, a partition label map is generated using threshold rules or a lightweight classifier. The partition label map includes high-structure areas, repetitive texture areas, weak texture areas, strong terrain areas, and occlusion areas. For each region type in the partitioned label map, region registration is performed to obtain deformation results for high-structure regions, repetitive texture regions, weak texture regions, and strong terrain regions. The deformation results are fused into a full-image dense deformation field using weighted fusion and smoothing regularization. Based on this full-image dense deformation field, the image to be aligned is transformed into the coordinate system of the reference image, and structural differences are compared to obtain a high-precision registered image and structural spatial residuals. The structural spatial residuals and the joint confidence level are then combined to calculate and output a structural registration uncertainty estimate. The expression for the structural registration uncertainty estimate is as follows: ; in, For set uncertainty, For the Sigmoid function, This represents the normalized value of the structural space residual. The normalized value of the joint credibility. , These are the weighting coefficients.

[0011] Optionally, for each region type of the partitioned label map, region registration is performed separately to obtain deformation results for high-structure regions, repetitive texture regions, weak texture regions, and strong terrain regions, including: For the high-structure region, key points are extracted from the final structure representation, feature descriptors are constructed using the key points, and then the feature descriptors are matched and outliers are removed using random sampling consistency to obtain a sparse control point set and the deformation result of the high-structure region. For the repetitive texture region, it is divided into blocks according to a fixed-size window to obtain multiple blocks. The optical similarity of the blocks is measured by normalized cross-correlation, and the cross-modal similarity of the blocks is measured by mutual information to output the region displacement observation and obtain the deformation result of the repetitive texture region. For the weak texture region, during the diffusion process of the displacement field obtained from the high structure region or the repetitive texture region to the weak texture region, a smoothing regularization is introduced to perform neighborhood smooth propagation and constrain the diffusion range to obtain the deformation result of the weak texture region. For the strong terrain area, a grid is divided, an affine transformation is performed on the grid, an affine matrix is ​​output, and then the affine matrix is ​​estimated by a deformation function to obtain the deformation result of the strong terrain area.

[0012] Optionally, candidate mask generation and pseudo-change detection are performed on the high-precision registration image to obtain pseudo-change probabilities. The pseudo-change probabilities are then fused with the structural registration uncertainty estimate to obtain change confidence weights, including: Perform cloud detection, shadow detection, and snow detection on the high-precision registered image, and output cloud candidate masks, shadow candidate masks, and snow candidate masks; For the cloud candidate mask and the shadow candidate mask, perform geometric consistency verification of cloud and shadow, output a cloud-shadow consistency confidence map, and calculate structural difference residual and radiation residual based on each candidate mask to determine the cause of pseudo-changes caused by cross-phenological or sensor differences and calculate the probability of pseudo-changes. The pseudo-change probabilities of each candidate mask are weighted and fused to obtain a pseudo-change mask. Then, the pseudo-change mask and the structural registration uncertainty estimate are fused to obtain the change confidence weight.

[0013] Optionally, based on the spectral and radiometric contrast domain and the structural contrast domain, cloud detection, shadow detection, and snow detection are performed on the high-precision registered image, and cloud candidate masks, shadow candidate masks, and snow candidate masks are output, including: Based on the spectral and radiation contrast domain and the structural contrast domain, for the optical image in the high-precision registered image, the visible light brightness and whiteness are calculated, and the relationship between the visible light brightness and the whiteness and the corresponding threshold is determined to obtain the cloud candidate mask. Based on the high-precision registered image, a saturation index is constructed, shadow detection is performed according to the saturation index, and a shadow candidate mask is generated. Then, snow detection is performed using the normalized differential snow cover index, and a snow candidate mask is generated.

[0014] Optionally, under the constraint of the changing confidence weights, the high-precision registered image is subjected to consistency enhancement and final detection using the constructed enhanced target total loss function, and then the quantitatively reliable enhanced image and the high-precision registered image are output, including: The high-precision registered image is radiometrically aligned, and the defined contrast, spectral consistency and structural consistency terms are weighted and fused to obtain the total loss function for enhanced targets. The calculation expression for the comparison term is: ; in, For the enhanced output pixels, For the aligned input pixels, Based on the contrast enhancement operator, To change the credibility weight; The calculation expression for the spectral consistency term is as follows: ; ; in, For the input multi-band vector, The spectral angle reflects the difference in the directions of the two vectors. To enhance the output multi-band vector, <> represents the vector dot product, and ‖‖ represents the norm. To prevent division by zero of extremely small constants; The calculation expression for the structural consistency term is: ; in, Let |||1 be the gradient operator, and |||1 be the L1 norm. The high-precision registered image is denoised and dehazed according to the total loss function of the enhanced target to complete image enhancement, output quantitatively reliable enhanced image and enhancement uncertainty, and then the enhancement uncertainty and the change confidence weight are combined to obtain the final detection weight. The spectral difference calculation, structural difference calculation, and weight suppression are performed on the reliable enhanced image using the final detection weights to complete change detection, obtain a high-reliability change detection result, complete the automatic registration and enhancement of satellite imagery, and output the high-precision registered image and the quantitative reliable enhanced image.

[0015] This invention discloses the following technical effects by providing a method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion: 1. Physical consistency: Unify multi-source imagery into the same map grid and add orthophoto quality markings to reduce systematic geometric differences and sources of uncontrollable errors.

[0016] 2. Cross-modal robust registration: Establish structural characterizations that are comparable across optical and SAR modes using structural tensors and multi-scale gradients, and explicitly control the risk of repetitive textures.

[0017] 3. Adaptive partitioned deformation estimation: Select the most suitable registration strategy according to structure, topography and confidence level and merge them into a dense deformation field, taking into account both accuracy and continuity.

[0018] 4. Credibility-driven spurious change suppression: Cloud shadows, snow, phenology, sensor differences, and residual registration errors are uniformly incorporated into the spurious change probability and integrated with registration uncertainty to form a unified weight.

[0019] 5. Weighted Consistency Enhancement and High-Reliability Detection: By changing the reliability weight constraint, the enhancement target is strengthened, so that both enhancement and detection follow the reliability, reducing false alarms and improving interpretability and stability.

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the method flow provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of automatic registration of large-scale satellite images provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of large-scale satellite image enhancement provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] like Figure 1 As shown, this invention provides a method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion, including: Step 1: Obtain the original image set, geometrical-physical prior data, and prior information on radiation and imaging conditions to obtain a multi-source original data set; Step 1 includes: 1.1 Image retrieval is performed according to the same region, the same time window, and multiple source types to obtain optical images and SAR images, resulting in a raw image set. The raw image set is then subjected to initial quality screening and download to obtain the raw image set and image metadata. The image metadata includes coverage, imaging time, resolution, cloud cover estimation, and solar elevation angle.

[0026] Optical imaging: Includes multispectral and optional panchromatic imaging.

[0027] SAR imagery can be used as a stable supplementary source in cloudy areas or nighttime scenes.

[0028] Initial quality screening: For optical images, perform initial screening based on cloud cover thresholds (e.g., cloud cover < a certain threshold). If high cloud cover images must be used, mark them as "high-risk images" for subsequent cloud and shadow masking modules to focus on. For SAR images: Check whether the polarization mode and incident angle range are compatible with the target scene, and record any indications that the speckle noise may be strong.

[0029] 1.2 For the original image set, the acquired rational polynomial coefficients, orbital attitude information and DEM data are written into a unified structure or configuration file to obtain geometric and physical prior data.

[0030] 1.3 Obtain the solar elevation angle, solar azimuth angle, sensor spectral response function, and recorded radiometric calibration and atmospheric correction states to obtain prior information on radiation and imaging conditions.

[0031] 1.4 The original image set, the image metadata, the geometric and physical prior data, and the prior information on radiation and imaging conditions are organized into a multi-source original data set.

[0032] Step 2, as follows Figure 2 As shown, based on the multi-source raw data set, orthorectification and multi-resolution unified projection are performed to obtain a physically consistent coarsely registered image set; step 2 includes: 2.1 Set a unified map coordinate system, define the pixel resolution of the output grid and the coverage area of ​​the output image to obtain a unified target grid.

[0033] 2.2 Based on the multi-source raw data set, for the rational polynomial coefficients, construct a forward mapping from geographic coordinates to pixel coordinates and a reverse mapping from pixel coordinates to geographic coordinates to obtain the RPC projection relationship and the back projection relationship. According to the projection relationship and the back projection relationship, perform pixel-by-pixel back projection on the target grid to obtain the orthorectified image.

[0034] 2.3 The orthorectified image is resampled using bilinear interpolation, and the resampling method is recorded in the image metadata. During the resampling process, the DEM missing mask, the RPC back projection area mask falling outside the original image, and the potential error are calculated to obtain the orthorectified projection and orthorectified quality label.

[0035] 2.4 After orthorectification, all images are in the same coordinate system, but inconsistencies in resolution, band combination, image size, and boundaries may still exist. Based on the orthorectification, target resolution and boundaries are unified, and an image pyramid is constructed, outputting consistent resolution, band combination, image size, and boundaries, resulting in a physically consistent coarsely registered image set.

[0036] Step 3, as follows Figure 2 As shown, for the coarsely registered image set, input processing, multi-scale gradient calculation, local smoothing and feature value calculation are performed to obtain the local tensor structure. Based on the local structure tensor, repetitive texture recognition is performed, and the final structure representation and structure confidence are generated. 3.1 For the coarsely registered image set, input processing, multi-scale gradient calculation, local smoothing, and eigenvalue calculation are performed to obtain the local tensor structure, including: 3.1.1 Select a reference image from the coarse registration image set and set the remaining images as images to be aligned. For the optical multispectral images in the images to be aligned, sum the visible light bands according to the weights to generate a brightness composite image. For the SAR images in the images to be aligned, convert the amplitude or backscattering intensity into the logarithmic domain, and then perform robust normalization on the brightness composite image to obtain the image structure input.

[0037] 3.1.2 For the image structure input, Gaussian smoothing is used to smooth the image at different scales. Based on the image smoothing results, the derivatives in the horizontal and vertical directions are calculated to obtain the multi-scale gradient, gradient magnitude and gradient direction.

[0038] 3.1.3 Based on the derivatives in the horizontal and vertical directions, construct the original structure tensor matrix, perform local smoothing on each element in the original structure tensor matrix to obtain the smoothed structure tensor, and then calculate the two eigenvalues ​​of the smoothed structure tensor to obtain the first eigenvalue and the second eigenvalue.

[0039] 3.1.4 Compare the magnitude relationship between the first eigenvalue and the second eigenvalue to obtain the structural strength and structural type. Then, perform a weighted fusion of the first eigenvalue and the second eigenvalue to output the structural saliency. Finally, integrate the structural strength, the structural type, and the structural saliency into the original structural tensor matrix to obtain the local structural tensor.

[0040] 3.2 Based on the local structure tensor, perform repetitive texture recognition and generate the final structure representation and structure confidence level, including: 3.2.1 Based on the local structure tensor, the proportion of strong edges (edge ​​density) is statistically analyzed within a fixed window, and the directional distribution within the fixed window is statistically analyzed according to the multi-scale gradient direction histogram or the structure direction histogram to obtain the local structure entropy. Combining the strong edge proportion, the structure saliency, and the local structure entropy, repeating texture recognition is performed to complete the structural reliability judgment.

[0041] Low edge density: weak texture area, prone to drift.

[0042] Moderate edge density and diverse structures: reliable region.

[0043] High edge density but repetitive structure: This could be a repeating texture.

[0044] 3.2.2 By fusing the multi-scale gradient, the local structure tensor, and the strong edge ratio, the final structure representation is obtained. Then, the structural reliability of the final structure representation is calculated using the defined multi-index gating. The calculation expression for the multi-index gating is: ; in, For structural credibility, For the Sigmoid function, This is the normalized value of the structural significance. This is the normalized value for the proportion of strong edges. To mitigate the risk of duplicate textures, , , These are the weighting coefficients.

[0045] Step 4, as follows Figure 2 As shown, based on the coarsely registered image set, regional registration is performed using the final structural representation and structural reliability to obtain a dense deformation field across the entire image. Then, based on the dense deformation field, the image to be aligned is transformed and the registration uncertainty is estimated to obtain a high-precision registered image and a structural registration uncertainty estimate. Step 4 includes: 4.1 Based on the coarsely registered image set, using the final structural representation and structural confidence, the structural confidence of the reference image and the image to be aligned is calculated at each pixel location to obtain joint confidence and structural type labels. Slope or elevation change rate is calculated using the DEM to obtain terrain risk. Then, based on the joint confidence, the structural type labels, and the terrain risk, a partition label map is generated using threshold rules or a lightweight classifier. The partition label map includes: High-structure region: The joint credibility is high and the structure type is corner, intersection, or obvious edge.

[0046] Repeating texture regions: have significant structures but a high risk of repetition, such as low entropy and strong periodicity.

[0047] Weakly textured regions: have low joint confidence and low structural saliency.

[0048] Strong terrain areas: high credibility of joint efforts.

[0049] Occlusion area: A priori mask with clouds or shadows.

[0050] 4.2 For each region type in the aforementioned partitioned label map, region registration is performed separately to obtain deformation results for high-structure regions, repetitive texture regions, weak texture regions, and strong terrain regions; specifically including: 4.2.1 For the high-structure region, key points are extracted from the final structure representation, feature descriptors are constructed using the key points, and then the feature descriptors are matched and outliers are removed using random sampling consistency to obtain a sparse control point set and the deformation result of the high-structure region.

[0051] 4.2.2 For the repeated texture region, it is divided into blocks according to a fixed-size window to obtain multiple blocks. The optical similarity of the blocks is measured by normalized cross-correlation, and the cross-modal similarity of the blocks is measured by mutual information to output the region displacement observation and obtain the deformation result of the repeated texture region.

[0052] 4.2.3 For the weak texture region, during the diffusion process of the displacement field obtained from the high structure region or the repeating texture region to the weak texture region, a smoothing regularization is introduced to perform neighborhood smooth propagation and constrain the diffusion range to obtain the deformation result of the weak texture region.

[0053] 4.2.4 For the strong terrain area, a grid block is divided, an affine transformation is performed on the grid block, an affine matrix is ​​output, and then the affine matrix is ​​estimated by the deformation function to obtain the deformation result of the strong terrain area.

[0054] 4.3 The various deformation results are fused into a full-image dense deformation field using weighted fusion and smoothing regularization. Based on this full-image dense deformation field, the image to be aligned is transformed into the coordinate system of the reference image, and structural differences are compared to obtain a high-precision registered image and structural spatial residuals. Combining the structural spatial residuals and the joint confidence level, a structural registration uncertainty estimate is output. The calculation expression for the structural registration uncertainty estimate is as follows: ; in, For set uncertainty, For the Sigmoid function, This represents the normalized value of the structural space residual. The normalized value of the joint credibility. , These are the weighting coefficients.

[0055] Step 5, as follows Figure 3 As shown, candidate mask generation and pseudo-change detection are performed on the high-precision registration image to obtain the pseudo-change probability. The pseudo-change probability and the structural registration uncertainty estimate are fused to obtain the change confidence weight; step 5 includes: 5.1 Perform cloud detection, shadow detection, and snow detection on the high-precision registered image, and output cloud candidate masks, shadow candidate masks, and snow candidate masks; specifically including: 5.1.1 Based on the spectral and radiation contrast domain and the structural contrast domain, for the optical image in the high-precision registered image, calculate the visible light brightness and whiteness, determine the relationship between the visible light brightness and the whiteness and the corresponding threshold, and obtain the cloud candidate mask.

[0056] 5.1.2 Based on the high-precision registered image, a luminosity index is constructed, shadow detection is performed according to the luminosity index, a shadow candidate mask is generated, and then snow detection is performed using the normalized differential snow cover index to generate a snow candidate mask.

[0057] 5.2 For the cloud candidate mask and the shadow candidate mask, perform geometric consistency verification of cloud and shadow, output a cloud-shadow consistency confidence map, and calculate the structural difference residual and radiation residual based on each candidate mask to determine the cause of pseudo-changes caused by cross-phenological or sensor differences and calculate the probability of pseudo-changes.

[0058] 5.3 The pseudo-change probabilities of each candidate mask are weighted and fused to obtain a pseudo-change mask. Then, the pseudo-change mask and the structural registration uncertainty estimate are fused to obtain the change confidence weight.

[0059] Step 6, as follows Figure 3 As shown, under the constraint of the changing confidence weights, the constructed enhancement target total loss function is used to perform consistency enhancement and final detection on the high-precision registered image, and then the quantitatively reliable enhanced image and the high-precision registered image are output. Step 6 includes: 6.1 Perform radiometric alignment on the high-precision registered image, and then perform weighted fusion of the defined contrast term, spectral consistency term, and structural consistency term to obtain the total loss function for enhanced targets; The calculation expression for the comparison term is: ; in, For the enhanced output pixels, For the aligned input pixels, Based on the contrast enhancement operator, To change the credibility weight.

[0060] The calculation expression for the spectral consistency term is as follows: ; ; in, For the input multi-band vector, The spectral angle reflects the difference in the directions of the two vectors. To enhance the output multi-band vector, <> represents the vector dot product, and ‖‖ represents the norm. To prevent division by zero of extremely small constants.

[0061] The calculation expression for the structural consistency term is: ; in, Let be the gradient operator, and |||1 be the L1 norm.

[0062] 6.2 Denoise and dehaze the high-precision registered image according to the total loss function of the enhanced target to complete image enhancement, output quantitatively reliable enhanced image and enhancement uncertainty, and then combine the enhancement uncertainty and the change confidence weight to obtain the final detection weight.

[0063] 6.3 The spectral difference calculation, structural difference calculation and weight suppression are performed on the reliable enhanced image through the final detection weight to complete the change detection, obtain the high reliability change detection result, complete the automatic registration and enhancement of satellite image, and output the high-precision registered image and the quantitative reliable enhanced image.

[0064] Therefore, this invention provides an automatic registration and enhancement method for large-scale satellite images based on adaptive multi-source fusion, achieving high-precision, low-false-alarm, quantifiable and reliable automatic registration and enhancement of large-scale satellite images from multiple sources.

[0065] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0066] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion, characterized in that, include: By acquiring the original image set, prior geometric and physical data, and prior information on radiation and imaging conditions, a multi-source original data set is obtained. Based on the multi-source raw data set, orthorectification and multi-resolution unified projection are performed to obtain a physically consistent coarsely registered image set. For the coarsely registered image set, input processing, multi-scale gradient calculation, local smoothing, and feature value calculation are performed to obtain the local tensor structure. Based on the local structure tensor, repetitive texture recognition is performed, and the final structure representation and structure confidence are generated. Based on the coarsely registered image set, regional registration is performed using the final structural representation and structural reliability to obtain a dense deformation field of the whole image. Then, the image to be aligned is transformed and the registration uncertainty is estimated based on the dense deformation field of the whole image to obtain a high-precision registered image and a structural registration uncertainty estimate. Candidate masks are generated and pseudo-changes are detected for the high-precision registered image to obtain the pseudo-change probability. The pseudo-change probability and the structural registration uncertainty estimate are fused to obtain the change confidence weight. Under the constraint of the changing confidence weights, the high-precision registered image is enhanced for consistency and finally detected using the constructed enhancement target total loss function, and then the quantitatively reliable enhanced image and the high-precision registered image are output.

2. The method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion according to claim 1, characterized in that, By acquiring the original image set, prior geometric and physical data, and prior information on radiation and imaging conditions, a multi-source original data set is obtained, including: Image retrieval is performed according to the same region, the same time window, and multiple source types to obtain optical images and SAR images, resulting in a raw image set. The raw image set is then subjected to initial quality screening and download to obtain the raw image set and image metadata. The image metadata includes coverage, imaging time, resolution, cloud cover estimation, and solar elevation angle. For the original image set, the acquired rational polynomial coefficients, orbital attitude information and DEM data are written into a unified structure or configuration file to obtain geometric and physical prior data. By acquiring solar elevation angle, solar azimuth angle, sensor spectral response function, and recorded radiometric calibration and atmospheric correction states, prior information on radiation and imaging conditions can be obtained. The original image set, the image metadata, the geometric and physical prior data, and the prior information on radiation and imaging conditions are organized into a multi-source original data set.

3. The method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion according to claim 2, characterized in that, Based on the aforementioned multi-source raw data set, orthorectification and multi-resolution unified projection are performed to obtain a physically consistent coarsely registered image set, including: A unified map coordinate system is set, the pixel resolution of the output grid is defined, and the output image coverage is obtained to obtain a unified target grid; Based on the multi-source raw data set, for the rational polynomial coefficients, a forward mapping from geographic coordinates to pixel coordinates and a reverse mapping from pixel coordinates to geographic coordinates are constructed to obtain the RPC projection relationship and the back projection relationship. According to the projection relationship and the back projection relationship, pixel-by-pixel back projection is performed on the target grid to obtain the orthorectified image. The orthorectified image is resampled using bilinear interpolation, and the resampling method is recorded in the image metadata. During the resampling process, the DEM missing mask, the RPC back projection area mask falling outside the original image, and the potential error are calculated to obtain the orthorectified projection and orthorectified quality label. Based on the orthophoto projection, target resolution and boundary unification and image pyramid construction are performed, outputting consistent resolution, band combination, image size and boundary, resulting in a physically consistent coarsely registered image set.

4. The method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion according to claim 3, characterized in that, For the coarsely registered image set, input processing, multi-scale gradient calculation, local smoothing, and eigenvalue calculation are performed to obtain the local tensor structure, including: Select a reference image from the coarse registration image set and set the remaining images as images to be aligned. For the optical multispectral images in the images to be aligned, the visible light bands are weighted and summed to generate a brightness composite image. For the SAR images in the images to be aligned, the amplitude or backscattering intensity is converted to the logarithmic domain. Then, the brightness composite image is robustly normalized to obtain the image structure input. For the image structure input, Gaussian smoothing is used to smooth the image at different scales. Based on the image smoothing results, the derivatives in the horizontal and vertical directions are calculated to obtain the multi-scale gradient, gradient magnitude and gradient direction. Based on the derivatives in the horizontal and vertical directions, an original structure tensor matrix is ​​constructed. Each element in the original structure tensor matrix is ​​locally smoothed to obtain a smoothed structure tensor. Then, two eigenvalues ​​of the smoothed structure tensor are calculated to obtain the first eigenvalue and the second eigenvalue. By comparing the magnitudes of the first eigenvalue and the second eigenvalue, the structural strength and structural type are obtained. The first eigenvalue and the second eigenvalue are then weighted and fused to output the structural saliency. The structural strength, the structural type, and the structural saliency are then incorporated into the original structural tensor matrix to obtain the local structural tensor.

5. The method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion according to claim 4, characterized in that, Based on the local structure tensor, repeating texture recognition is performed, and a final structure representation and structure confidence are generated, including: Based on the local structure tensor, the proportion of strong edges is counted within a fixed window, and the directional distribution within the fixed window is counted according to the multi-scale gradient direction histogram or the structure direction histogram to obtain the local structure entropy. Combining the proportion of strong edges, the structural saliency, and the local structure entropy, repeating texture recognition is performed to complete the structural reliability judgment. By fusing the multi-scale gradient, the local structure tensor, and the strong edge ratio, a final structural representation is obtained. Then, the structural reliability of the final structural representation is calculated using a defined multi-index gating method. The calculation expression for the multi-index gating is as follows: ； in, For structural credibility, For the Sigmoid function, This is the normalized value of the structural significance. This is the normalized value for the proportion of strong edges. To mitigate the risk of duplicate textures, , , These are the weighting coefficients.

6. The method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion according to claim 5, characterized in that, Based on the coarsely registered image set, regional registration is performed using the final structural representation and structural reliability to obtain a dense deformation field across the entire image. Then, based on this dense deformation field, the images to be aligned are transformed and registration uncertainty is estimated to obtain a high-precision registered image and structural registration uncertainty estimation, including: Based on the coarsely registered image set, using the final structural representation and structural confidence, the structural confidence of the reference image and the image to be aligned is calculated at each pixel location to obtain joint confidence and structural type labels. The slope or elevation change rate is calculated using the DEM to obtain terrain risk. Then, based on the joint confidence, the structural type labels, and the terrain risk, a partition label map is generated using threshold rules or a lightweight classifier. The partition label map includes high-structure areas, repetitive texture areas, weak texture areas, strong terrain areas, and occlusion areas. For each region type in the partitioned label map, region registration is performed to obtain deformation results for high-structure regions, repetitive texture regions, weak texture regions, and strong terrain regions. The deformation results are fused into a full-image dense deformation field using weighted fusion and smoothing regularization. Based on this full-image dense deformation field, the image to be aligned is transformed into the coordinate system of the reference image, and structural differences are compared to obtain a high-precision registered image and structural spatial residuals. The structural spatial residuals and the joint confidence level are then combined to calculate and output a structural registration uncertainty estimate. The expression for the structural registration uncertainty estimate is as follows: ； in, For set uncertainty, For the Sigmoid function, This represents the normalized value of the structural space residual. The normalized value of the joint credibility. , These are the weighting coefficients.

7. The method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion according to claim 6, characterized in that, For each region type in the partitioned label map, region registration is performed to obtain deformation results for high-structure regions, repetitive texture regions, weak texture regions, and strong terrain regions, including: For the high-structure region, key points are extracted from the final structure representation, feature descriptors are constructed using the key points, and then the feature descriptors are matched and outliers are removed using random sampling consistency to obtain a sparse control point set and the deformation result of the high-structure region. For the repetitive texture region, it is divided into blocks according to a fixed-size window to obtain multiple blocks. The optical similarity of the blocks is measured by normalized cross-correlation, and the cross-modal similarity of the blocks is measured by mutual information to output the region displacement observation and obtain the deformation result of the repetitive texture region. For the weak texture region, during the diffusion process of the displacement field obtained from the high structure region or the repetitive texture region to the weak texture region, a smoothing regularization is introduced to perform neighborhood smooth propagation and constrain the diffusion range to obtain the deformation result of the weak texture region. For the strong terrain area, a grid is divided, an affine transformation is performed on the grid, an affine matrix is ​​output, and then the affine matrix is ​​estimated by a deformation function to obtain the deformation result of the strong terrain area.

8. The method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion according to claim 7, characterized in that, Candidate masks are generated and pseudo-changes are detected for the high-precision registered image to obtain pseudo-change probabilities. These pseudo-change probabilities are then fused with the structural registration uncertainty estimate to obtain change confidence weights, including: Perform cloud detection, shadow detection, and snow detection on the high-precision registered image, and output cloud candidate masks, shadow candidate masks, and snow candidate masks; For the cloud candidate mask and the shadow candidate mask, perform geometric consistency verification of cloud and shadow, output a cloud-shadow consistency confidence map, and calculate structural difference residual and radiation residual based on each candidate mask to determine the cause of pseudo-changes caused by cross-phenological or sensor differences and calculate the probability of pseudo-changes. The pseudo-change probabilities of each candidate mask are weighted and fused to obtain a pseudo-change mask. Then, the pseudo-change mask and the structural registration uncertainty estimate are fused to obtain the change confidence weight.

9. The method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion according to claim 8, characterized in that, Based on the spectral and radiometric contrast domains and the structural contrast domains, cloud detection, shadow detection, and snow detection are performed on the high-precision registered image, outputting cloud candidate masks, shadow candidate masks, and snow candidate masks, including: Based on the spectral and radiation contrast domain and the structural contrast domain, for the optical image in the high-precision registered image, the visible light brightness and whiteness are calculated, and the relationship between the visible light brightness and the whiteness and the corresponding threshold is determined to obtain the cloud candidate mask. Based on the high-precision registered image, a saturation index is constructed, shadow detection is performed according to the saturation index, and a shadow candidate mask is generated. Then, snow detection is performed using the normalized differential snow cover index, and a snow candidate mask is generated.

10. The method for automatic registration and enhancement of large-scale satellite imagery based on adaptive multi-source fusion according to claim 9, characterized in that, Under the constraint of the changing confidence weights, the high-precision registered image is subjected to consistency enhancement and final detection using the constructed enhanced target total loss function, and then a quantitatively reliable enhanced image and the high-precision registered image are output, including: The high-precision registered image is radiometrically aligned, and the defined contrast, spectral consistency and structural consistency terms are weighted and fused to obtain the total loss function for enhanced targets. The calculation expression for the comparison term is: ； in, For the enhanced output pixels, For the aligned input pixels, Based on the contrast enhancement operator, To change the credibility weight; The calculation expression for the spectral consistency term is as follows: ； ； in, For the input multi-band vector, The spectral angle reflects the difference in the directions of the two vectors. To enhance the output multi-band vector, <> represents the vector dot product, and ‖‖ represents the norm. To prevent division by zero of extremely small constants; The calculation expression for the structural consistency term is: ； in, Let |||1 be the gradient operator, and |||1 be the L1 norm. The high-precision registered image is denoised and dehazed according to the total loss function of the enhanced target to complete image enhancement, output quantitatively reliable enhanced image and enhancement uncertainty, and then the enhancement uncertainty and the change confidence weight are combined to obtain the final detection weight. The spectral difference calculation, structural difference calculation, and weight suppression are performed on the reliable enhanced image using the final detection weights to complete change detection, obtain a high-reliability change detection result, complete the automatic registration and enhancement of satellite imagery, and output the high-precision registered image and the quantitative reliable enhanced image.

Images