Multi-source satellite data image automatic registration method and system

Abstract

The present invention relates to the field of satellite remote sensing application support technology, and in particular to a multi-source satellite data image automatic registration method and system. For multispectral satellite data images, a single-band reference image and a single-band image to be corrected are obtained through optimal band selection, and data preprocessing operations are performed on the reference image and the image to be corrected; the geometric displacement of the input reference image and the image to be corrected is detected; and the correction and verification of the detected image geometric displacement are performed. The present invention estimates sub-pixel displacement in the frequency domain based on a phase correlation method. By combining various verification and quality estimation indicators, a dense grid of spatial displacement vectors is created and automatically filtered. The method is capable of automatically detecting and correcting sub-pixel misalignments between various remote sensing data sets, is independent of spatial or spectral features, and is robust to high cloud cover and spectral and temporal land cover dynamics, thereby achieving automated, efficient, and accurate registration of remote sensing big data.

Description

Technical Field

[0001] The present invention relates to the field of satellite remote sensing application support technology, and in particular to a multi-source satellite data image automatic registration method and system. Background Art

[0002] Remote sensing image geospatial registration is a necessary prerequisite for processing remote sensing data. It is the matching and superposition of two or more images acquired at different times, different sensors (imaging equipment) or different conditions (weather, illumination, camera position and angle, etc.). Among them, image registration is the calibration of one image (reference image) against another image in the same area so that the pixels with the same name in the two images are aligned. Registration errors between or within sensors will have a negative impact on any subsequent image analysis, especially when processing multi-sensor or multi-temporal data. Existing multi-source satellite remote sensing image registration mostly uses manual measurement of the same point name, which is time-consuming and labor-intensive and cannot meet the needs of remote sensing big data and automated processing. Therefore, there is an urgent need for an efficient and accurate automatic registration method to achieve precise registration of multi-source satellite remote sensing images. Summary of the Invention

[0003] To this end, the present invention provides a method and system for automatic registration of multi-source satellite data images, which can automatically detect and correct sub-pixel misalignments between various remote sensing datasets, is independent of spatial or spectral characteristics, and is robust to high cloud cover and spectral and temporal land cover dynamics.

[0004] According to the design scheme provided by the present invention, a method for automatic registration of multi-source satellite data images is provided, comprising:

[0005] For multispectral satellite data images, a single-band reference image and a single-band image to be corrected are obtained through optimal band selection. Data preprocessing operations are then performed on the reference image and the image to be corrected. The data preprocessing operations include image overlap area calculation, pixel coordinate dense grid equalization processing, and matching window adaptive adjustment.

[0006] The geometric displacement of the input reference image and the image to be corrected is detected. In local registration, each point of the dense grid in the reference image and the image to be corrected is processed in a sliding window manner to realize the geometric displacement detection. In global registration, the geometric displacement detection is realized by processing a single coordinate position.

[0007] The geometric displacement of the detected image is corrected and verified. During local registration, the geometric displacement is verified and corrected by distorting the target image in the image to be calibrated. During global registration, the geometric displacement is verified and corrected by the X / Y displacement vector of a single pixel point.

[0008] As the multi-source satellite data image automatic registration method of the present invention, further, before obtaining the single-band reference benchmark image and the single-band image to be corrected through optimal band selection, it also includes: setting the dense grid connection point distance according to the resolution of the reference benchmark image.

[0009] As the multi-source satellite data image automatic registration method of the present invention, further, obtaining a single-band reference benchmark image and a single-band image to be corrected by optimal band selection includes:

[0010] First, for multispectral images, the wavelength position of the target band is considered. If the corresponding image data is available, the optimal band of the image is selected based on the band with the minimum spectral distance. Otherwise, the first band is used as the optimal band of the image.

[0011] Then, single-band image data is obtained based on the best image band.

[0012] As the automatic registration method of multi-source satellite data images of the present invention, further, the image overlapping area calculation includes:

[0013] First, the bad data masks are separated by combining the preset masks to fuse the data-free masks in the image;

[0014] Next, raster-to-vector conversion is applied to the no-data mask to calculate the covering polygons of the reference fiducial image and the image to be corrected;

[0015] Then, the image space overlapping area of ​​the reference benchmark image and the image to be corrected is obtained by identifying the common range of the covering polygon.

[0016] As the automatic registration method for multi-source satellite data images of the present invention, further, the matching window is adaptively adjusted, including: for local registration, the matching window position is determined by using dense grid coordinates; for global registration, the matching window position is determined by the center of the covering polygon.

[0017] The method for automatic registration of multi-source satellite data images of the present invention further includes detecting the geometric displacement between the input reference image and the image to be corrected, and further includes: first, obtaining the geometric displacement between the input reference image and the image in the image to be corrected; and then, using a preset complementary verification rule to verify the matching spatial displacement between the images.

[0018] The automatic registration method for multi-source satellite data images of the present invention further includes obtaining a geometric displacement between an input reference image and an image to be corrected, comprising:

[0019] First, the input reference image and the image within the matching window of the image to be corrected are converted into the frequency domain using fast Fourier transform. Phase correlation is performed on the images in the frequency domain to generate a cross-power spectrum. The images are then converted into the spatial domain using inverse fast Fourier transform. The maximum peak position is obtained in the spatial domain using the normalized form of the cross-power spectrum.

[0020] Next, an integer offset is obtained based on the distance between the maximum peak position and the center position of the cross-power spectrum. The matching window coordinates are adjusted based on the integer offset, and subset images of the reference image and the image to be corrected are constructed at the new matching window position. The two subset images are converted back to the frequency domain and a new cross-power spectrum is created. The sub-pixel displacement is obtained based on the maximum peak position of the new cross-power spectrum in the spatial domain and the direct displacement in the X and Y directions.

[0021] Then, the sub-pixel displacement is added to the target image and the total displacement in pixel units is converted to mapping units.

[0022] As a multi-source satellite data image automatic registration method of the present invention, a preset complementary verification rule is further used to verify the spatial displacement of the matching between images, including: first, correcting the integer displacement within the corresponding matching window by moving the image, and checking the validity of the integer displacement; then, using a threshold check to eliminate the displacement that does not meet the preset threshold; then, in the local displacement correction, the maximum peak of the cross power spectrum is used to evaluate the reliability of the outlier connection point, the average structural similarity index is used to evaluate the similarity of the target image content in the matching window before and after the shift correction, and a random sample consistency algorithm is used to identify dense grid coordinate outliers.

[0023] As the multi-source satellite data image automatic registration method of the present invention, further, the detected image geometric displacement is corrected and verified, and further includes: determining whether the input image provides unequal mapping projections, and if so, using displacement mapping projection when the image is distorted, aligning the dense grid of the image to be corrected with the dense grid of the reference benchmark image or changing the spatial resolution.

[0024] Furthermore, the present invention also provides a multi-source satellite data image automatic registration system, comprising: a data preprocessing module, a displacement detection module and a displacement correction module, wherein:

[0025] The data preprocessing module is used to obtain a single-band reference image and a single-band image to be corrected by selecting the optimal band for multispectral satellite data images, and perform data preprocessing operations on the reference image and the image to be corrected. The data preprocessing operations include image overlap area calculation, pixel coordinate dense grid equalization processing, and matching window adaptive adjustment.

[0026] The displacement detection module is used to detect the geometric displacement of the input reference image and the image to be corrected. In local registration, the geometric displacement detection is realized by processing each point of the dense grid in the reference image and the image to be corrected in a sliding window manner. In global registration, the geometric displacement detection is realized by processing a single coordinate position.

[0027] The displacement correction module is used to correct and verify the geometric displacement of the detected image. During local registration, the geometric displacement is verified and corrected by distorting the target image in the image to be calibrated. During global registration, the geometric displacement is verified and corrected by the X / Y displacement vector of a single pixel point.

[0028] Beneficial effects of the present invention:

[0029] This method uses phase correlation to estimate sub-pixel displacement in the frequency domain. By incorporating various validation and quality estimation metrics, it creates and automatically filters a dense grid of spatial displacement vectors. This method supports masking, such as clouds and cloud shadows, to exclude spatial displacement detection in these areas, enabling automated, efficient, and accurate registration of remote sensing big data. Further testing on over 9,000 satellite images acquired from various sensors demonstrated a root mean square error (RMS) of better than 0.3 pixels. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic diagram of the automatic registration process of multi-source satellite data images in the embodiment;

[0031] Figure 2 This is a schematic diagram of the principle of the automatic registration algorithm for multi-source satellite data images in the embodiment;

[0032] Figure 3 Schematic diagram of subset images within the spatial domain and frequency domain matching windows and corresponding cross-power spectra in an embodiment;

[0033] Figure 4 Schematic diagram of the integer displacement verification process in the sub-pixel X / Y displacement calculation process in the embodiment;

[0034] Figure 5 Schematic diagram of calculation efficiency statistics in the embodiment. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention is further described in detail below with reference to the accompanying drawings and technical solutions.

[0036] For this purpose, see the embodiment of the present invention. Figure 1 As shown, a method for automatic registration of multi-source satellite data images is provided, comprising:

[0037] S101. For a multispectral satellite data image, a single-band reference image and a single-band image to be corrected are obtained through optimal band selection, and data preprocessing operations are performed on the reference image and the image to be corrected, wherein the data preprocessing operations include image overlap area calculation, pixel coordinate dense grid equalization processing, and matching window adaptive adjustment;

[0038] S102, detecting the geometric displacement of the input reference image and the image to be corrected, wherein the geometric displacement detection is achieved by processing each point of the dense grid in the reference image and the image to be corrected in a sliding window manner during local registration, and by processing a single coordinate position during global registration;

[0039] S103, correcting and verifying the geometric displacement of the detected image, wherein the geometric displacement is verified and corrected by distorting the target image in the image to be calibrated during local registration, and the geometric displacement is verified and corrected by the X / Y displacement vector of a single pixel during global registration.

[0040] Phase correlation can only be used for two monochrome (single-band) input images with exactly the same pixel size, representing roughly the same (or slightly shifted) geographic location on the Earth's surface, and ideally also with similar pixel intensity values. This is usually not possible when processing multi-time and multi-sensor image data. In this embodiment, the Fourier shift theorem is used to perform sub-pixel displacement estimation in the frequency domain based on the phase correlation method. By combining various verification and quality estimation indicators, a dense grid of spatial displacement vectors can be created and automatically filtered, which can determine the precise X / Y offset of the image at a given geographic location, thereby achieving efficient and accurate automatic calibration of remote sensing data.

[0041] Local registration methods apply phase correlation to a (default) regular grid of coordinate points in a sliding window fashion and estimate the X / Y translation for each point within the overlapping region of the input image. The resulting grid of shift vectors is validated based on a set of independent quality metrics. Points that pass all initialization validations are used as tie points to fit an image affine transformation model, which is used to correct local geometric misregistration of the images. In contrast, global registration methods compute the displacement at only one coordinate location in a small subset of the image (from this point on: the matching window) and then shift the target image according to the single displacement vector. This process is computationally inexpensive and therefore very fast. Furthermore, it allows for selectively disabling image warping to correct sub-pixel X / Y shifts without any resampling (simply by adjusting geocoding metadata). This preserves the original pixel values ​​but does not match the coordinate grid of the reference image. However, global registration assumes that the overall misregistration between the reference and target images can be described by a constant (global) translation in the X and Y directions. This can be useful if the displacement pattern can be confirmed a priori or if modifying the original pixel values ​​through resampling is a concern and must be strictly avoided. In the context of the need for generality and operational applicability of multi-sensor and multi-temporal image data, a focus can be placed on local registration methods.

[0042] As a preferred embodiment, further, before obtaining the single-band reference benchmark image and the single-band image to be corrected through optimal band selection, it also includes: setting the dense grid connection point distance according to the resolution of the reference benchmark image.

[0043] Before preprocessing, the resolution of the reference image and the benchmark image (in the case of local registration) needs to be provided to define the distance between the connection points within the generated dense grid. The appropriate grid resolution depends on the complexity of the expected misregistration pattern. Smaller point distances can correct more complex distortions, but will increase the computational load. The input image can have valid geocoding information corresponding to any geographic projection or UTM map projection (equal geographic data). In addition, various common raster image formats are supported. A mask of non-opaque cloud areas is provided in the input image to reduce the probability of errors in registration and speed up processing. Figure 2 As shown, a no-data mask is automatically generated to represent the areas filled with no-data values. The no-data values ​​are read from the input image metadata or automatically derived from a 3×3 window at each image corner. The generated and user-provided masks are combined to separate the (1-bit) bad data masks for the reference and target images.

[0044] Automatic Projection Transformation: To address the inconsistencies in georeferences between multi-source remote sensing images, which can severely impact geometric registration, we can automatically obtain the georeferenced coordinate system of the image to be processed before geometric registration and use a projection transformation algorithm to unify it into a common reference coordinate system. This facilitates further processing using phase correlation-based methods.

[0045] Furthermore, in the embodiment of this case, a single-band reference image and a single-band image to be corrected are obtained by selecting the optimal band, which can be specifically designed to include the following content:

[0046] First, for multispectral images, the wavelength position of the target band is considered. If the corresponding image data is available, the optimal band of the image is selected based on the band with the minimum spectral distance. Otherwise, the first band is used as the optimal band of the image.

[0047] Then, single-band image data is obtained based on the best image band.

[0048] When registering multispectral or hyperspectral input images, automatically determining the specific spectral band (wavelength location) to use as the registration input image is key to ensuring that the intensity information in the reference and target images is as similar as possible. While phase correlation methods focus only on the actual image content, they are not sensitive to variations in intensity differences (regardless of their origin). Therefore, in this embodiment, the wavelength location of the target band is considered, and if corresponding image metadata is available, the optimal band of the reference image is automatically selected by determining the band with the smallest spectral distance. Otherwise, the first band is selected.

[0049] If the registration target is a multi-band data cube, you can either apply the processing band by band in a loop or apply the registration result of a single band to the remaining bands. The former is computationally more expensive, but the registration accuracy of the target image band and the corresponding reference band is higher.

[0050] Furthermore, the calculation of the image overlapping area can be specifically designed to include:

[0051] First, the bad data masks are separated by combining the preset masks to fuse the data-free masks in the image;

[0052] Next, raster-to-vector conversion is applied to the no-data mask to calculate the covering polygons of the reference fiducial image and the image to be corrected;

[0053] Then, the image space overlapping area of ​​the reference benchmark image and the image to be corrected is obtained by identifying the common range of the covering polygon.

[0054] Computing the exact spatial overlap region of the two input images, excluding any no-data regions, is a prerequisite for automatically determining the coordinate position of each matching window used to calculate the spatial displacement. Furthermore, it effectively reduces the computational effort required when performing local registration based on a large number of tie points, since all points in the dense grid where the input images have no-data values ​​can be excluded. Therefore, in this embodiment, the covering polygons of the reference and target images can be calculated by applying a raster-to-vector conversion to the previously generated no-data mask, and then identifying their common range to be used as the overlapping polygon.

[0055] Remote sensing images often use non-uniform pixel coordinate grids, especially in multi-sensor situations, where the spacing and grid origins are often different and the images may provide different map projections. Therefore, before applying phase correlation to detect geometric displacements, one image must be resampled to the exact pixel spacing, origin and map projection of the other image (based on their respective geocoding information and projections). Since having the same number of rows and columns is one of the prerequisites for applying phase correlation, equalization of the pixel grid is essential. In this embodiment, images with higher spatial resolution can be downsampled to lower resolution images. Upsampling low-resolution images will not improve the registration accuracy, but will reduce processing speed. By default, a cubic resampling technique can be used to avoid changes in the position of ground objects in the image.

[0056] In the case of local registration, this processing step is performed before applying the phase correlation to the dense grid, while in the global case it is embedded in the displacement calculation. Its purpose is to avoid resampling only the subset of the image that is actually used (within the matching window) in the global case, while preventing repeated resampling of the same locations due to overlapping adjacent matching windows in the local registration.

[0057] In the adaptive adjustment of the matching window, for local registration, the matching window position is determined by using dense grid coordinates, and for global registration, the matching window position is determined by the center of the covering polygon.

[0058] In the case of local registration, the matching window position is determined by a previously created dense coordinate grid. For the global method, the coordinate position defaults to the center of the overlapping polygon, thus avoiding the influence of the respective bad data mask image area. The size of the matching window can be set to 256×256 pixels by default and does not change within the image overlap area. This is crucial to maintain the comparability of adjacent displacement vector results, as the calculated displacement accuracy will vary with the size of the matching window. However, at the edge of the overlapping area, it is necessary to ensure that the matching window does not protrude into the data-free area of ​​the reference image or the target image. Otherwise, the edge between the image data and the data-free area may affect the geometric displacement detection results through phase correlation due to its frequency response in the Fourier domain.

[0059] As a preferred embodiment, further, detecting the geometric displacement of the input reference benchmark image and the image to be corrected also includes: first, obtaining the geometric displacement between the input reference benchmark image and the image in the image to be corrected; then, using a preset complementary verification rule to verify the matching spatial displacement between the images.

[0060] The process of obtaining the geometric displacement between the input reference image and the image to be corrected includes:

[0061] First, the input reference image and the image within the matching window of the image to be corrected are converted into the frequency domain using fast Fourier transform. Phase correlation is performed on the images in the frequency domain to generate a cross-power spectrum. The images are then converted into the spatial domain using inverse fast Fourier transform. The maximum peak position is obtained in the spatial domain using the normalized form of the cross-power spectrum.

[0062] Next, an integer offset is obtained based on the distance between the maximum peak position and the center position of the cross-power spectrum. The matching window coordinates are adjusted based on the integer offset, and subset images of the reference image and the image to be corrected are constructed at the new matching window position. The two subset images are converted back to the frequency domain and a new cross-power spectrum is created. The sub-pixel displacement is obtained based on the maximum peak position of the new cross-power spectrum in the spatial domain and the direct displacement in the X and Y directions.

[0063] Then, the sub-pixel displacement is added to the target image and the total displacement in pixel units is converted to mapping units.

[0064] The preset complementary verification rules are used to verify the spatial displacement of image matching, including: first, correcting the integer displacement within the corresponding matching window by moving the image, and checking the validity of the integer displacement; then, using a threshold check to eliminate the displacement that does not meet the preset threshold; then, in the local displacement correction, the maximum peak of the cross power spectrum is used to evaluate the reliability of the outlier connection point, the average structural similarity index is used to evaluate the similarity of the target image content in the matching window before and after shift correction, and the random sample consistency algorithm is used to identify dense grid coordinate outliers.

[0065] The geometric displacement between the input images is calculated in a matching window. The two sub-images can be converted into the frequency domain using FFTW (Fast Fourier Transform). The two images obtained in the frequency domain are phase correlated to generate their cross power spectrum, which is then converted back to the spatial domain using inverse FFTW. The normalized form of the cross power spectrum in the spatial domain has a clear peak at the registration point of the input image, such as Figure 3 As shown, it can be used to quantify the image displacement. The integer offset can be obtained from the distance between the maximum peak position and the center position of the spectrum.

[0066] Based on the calculated integer displacement, the subset image of the target dataset is temporarily moved by adjusting the angular coordinates of the matching window and reconstructing the subset image for the target dataset at the new position. To obtain the sub-pixel displacement, the two subset images (reference and integer-corrected target) are again converted to the frequency domain and a new cross-power spectrum is created. The sub-pixel displacement can be estimated according to Equation 1 and Equation 2, where v(0,0) represents the cross-power spectrum at the peak position, and v(1,0) and v(0,1) represent the direct displacement in the X and Y directions.

[0067]

[0068]

[0069] The calculated sub-pixel displacement is added and the total displacement in pixel units is converted to map units. In the case of the local displacement correction method, a grid of junction points is generated by combining the total map displacement and the absolute map coordinate offset provided by the geocoding information of the input image.

[0070] In spatial displacement verification, the reliability of X / Y displacement is related to the similarity of input image pixel values ​​caused by different sensor noise. In this embodiment, five complementary verification techniques can be used to define and quantify the potential impact:

[0071] First, the calculated integer shift is checked for validity, as Figure 4As shown. It is assumed that correcting the integer shift within the corresponding matching window by moving the target subset image must result in zero shift. If the subsequent calculation still produces a non-zero X / Y shift, the match is considered to be a false detection and discarded. In this case, the newly calculated integer shift is applied and the shift calculation is repeated to evaluate whether the remaining integer shift is now zero until the maximum number of iterations is reached (5 by default). If the shift remains, the match of the corresponding geographic location fails. Otherwise, a valid match will be found (according to the integer shift) and the sub-pixel shift will eventually be derived. The calculation process is as follows Figure 4 shown.

[0072] After successfully calculating the sub-pixel X / Y displacement, a threshold check (the second validation method) can be used to eliminate unrealistically large displacements. For example, prior knowledge can be incorporated by simply rejecting all vectors exceeding a certain length (by default, 5 reference image pixels).

[0073] It is still possible that some of the registered points are not matched correctly, for example, due to repeated patterns in one of the input images that introduce false detections. To address this issue, three other validation techniques are combined to effectively filter outlier tie points in the presence of local displacement correction. Two of these validation metrics are computed separately for each tie point of the dense grid after computing the sub-pixel displacements. The first (i.e., the third validation method in the complementarity) analyzes the 3D shape of the cross power spectrum, aiming to quantify the sharpness of the peaks, e.g. Figure 3 As shown, it is a measure to evaluate the reliability of each connection point. It returns a reliability percentage R calculated using the following formula.

[0074]

[0075]

[0076]

[0077]

[0078] Because v(r,c) represents the value of the cross-power spectrum at a specific location, and N is the number of pixels in the subset, the average power within a 3×3 window centered at the peak of the spectrum is related to the average power of the remaining spectrum plus three times its standard deviation. This affects the reliability percentage of each tie point and allows points below a threshold to be excluded from the subsequent calculation of geometric transformation parameters. Empirical evidence suggests that a threshold of 30% is recommended, below which tie points are rejected.

[0079] A fourth validation method is also performed for each point in the dense grid, evaluating the similarity of the subset image content within the matching window before and after shift correction. This is based on the assumption that a successful registration correction will improve the similarity between the reference and target images. The mean structural similarity index (MSSIM) was chosen as a metric for image similarity, as it is highly sensitive to edge image shifts. MSSIM returns a value between 0 and 1, where 0 indicates a mismatch and 1 indicates a perfect match. To compute the MSSIM before and after shift correction, the subset target image is first globally offset using the phase-correlated input data according to the calculated sub-pixel offsets of the corresponding positions, and then the MSSIM value is calculated. All points in the grid that are densely connected points reduced by MSSIM are marked and excluded from estimating the transformation parameters.

[0080] The fifth verification method combines all calculated displacement vectors simultaneously. This is crucial to avoid artifacts in the registration results caused by outliers in the dense grid of point data. To this end, the Random Sample Consensus (RANSAC) algorithm is chosen, assuming that misregistrations can be roughly corrected using an affine transformation in remote sensing satellite data. RANSAC is used to automatically estimate the parameters of the assumed affine transformation between the reference and target images, thereby identifying outliers in the previously calculated displacement vector grid. Parameter estimation is performed automatically by RANSAC in an iterative process, but the input parameters need to be set in advance. The most important one (based on experience) is the threshold used to separate inliers from outliers. In this case, the optimal threshold is highly dependent on the heterogeneity of the input samples and, therefore, on the variance and reliability of the calculated displacement vectors. This variance, in turn, is linked to the input images themselves. This means that, for example, extensive cloud cover can strongly influence the calculated tie points by often leading to repetitive image patterns, resulting in a higher proportion of false detections. Furthermore, RANSAC suffers from a high proportion of outliers in the input samples. To overcome this issue and avoid incorrect estimation of the affine parameters, several strategies are designed in this verification step.

[0081] First, the bad data mask can be considered for the input image, effectively excluding these critical image regions from the matching process. Consequently, RANSAC reduces the variance of outliers within the input data, significantly reducing the probability of internal outliers dominating the input samples. To further improve the robustness of outlier identification, RANSAC has been implemented as the final layer of validation. Therefore, the input samples to RANSAC are pre-filtered; tie points with low reliability values ​​or decreasing MSSIM scores are no longer passed to RANSAC. Finally, an iterative method is used to automatically optimize the RANSAC threshold described above to separate interpolants from outliers. RANSAC is run multiple times, observing the ratio of interpolants to outliers in the results, to determine a threshold that marks the number of true outliers in the input data. The first iteration begins with an initial guess for the threshold, which is then iteratively increased or decreased based on whether the proportion of flagged outliers is too low or too high. Testing has shown that an outlier ratio of 10% works well for most remote sensing data. Therefore, the overall registration is terminated when 10% ± 2% of the input tie points are flagged as outliers.

[0082] Furthermore, the correction and verification of the detected image geometric displacement also includes: determining whether the input image provides unequal mapping projections. If so, using displacement mapping projection when the image is distorted, aligning the dense grid of the image to be corrected with the dense grid of the reference benchmark image or changing the spatial resolution.

[0083] During displacement correction, local distortions are inherent, and geometric registration is handled based on tie points that meet the aforementioned validation criteria and quality metrics. In the case of global matching, a single tie point determines the X / Y transformation applied to the target dataset. If the input images provide unequal projections, the calculated displacement projection is used during image warping. The warping itself is based on a cubic resampling technique that aligns the pixel grid of the target image with that of the reference image or changes the target spatial resolution. Resampling is performed only once throughout the registration workflow to avoid unintended degradation of geometric and spectral image quality. Global registration allows us to avoid resampling entirely, leaving the pixel values ​​of the target image unchanged. In this case, only the geocoded image metadata with misaligned pixel grids is adjusted.

[0084] Furthermore, based on the above method, an embodiment of the present invention also provides a multi-source satellite data image automatic registration system, comprising: a data preprocessing module, a displacement detection module and a displacement correction module, wherein:

[0085] The data preprocessing module is used to obtain a single-band reference image and a single-band image to be corrected by selecting the optimal band for multispectral satellite data images, and perform data preprocessing operations on the reference image and the image to be corrected. The data preprocessing operations include image overlap area calculation, pixel coordinate dense grid equalization processing, and matching window adaptive adjustment.

[0086] The displacement detection module is used to detect the geometric displacement of the input reference image and the image to be corrected. In local registration, the geometric displacement detection is realized by processing each point of the dense grid in the reference image and the image to be corrected in a sliding window manner. In global registration, the geometric displacement detection is realized by processing a single coordinate position.

[0087] The displacement correction module is used to correct and verify the geometric displacement of the detected image. During local registration, the geometric displacement is verified and corrected by distorting the target image in the image to be calibrated. During global registration, the geometric displacement is verified and corrected by the X / Y displacement vector of a single pixel point.

[0088] To verify the effectiveness of this solution, the following is a further explanation based on experimental data:

[0089] 1. The root mean square (RMSE) errors of the detected displacements before and after correction are compared to quantitatively evaluate the overall performance of the proposed algorithm.

[0090] Registration experiments with Sentinel-2 / Landsat-8 test images showed that after correction, the RMSE of spatial displacement decreased from 34.63 m to 4.45 m (from 2.31 pixels to 0.3 pixels in the low-resolution reference image). Experiments with RapidEye-5 / Landsat-8 images showed that the RMSE of spatial displacement decreased from an initial RMSE of 13.69 m to 4.55 m, corresponding to 0.15 pixels in the low-resolution 30m reference image after image distortion, with the spatial displacement distribution pointing in opposite directions. Experiments with TerraSAR-X imagery showed no significant changes in the direction of the displacement or the absolute length of the displacement vector. After displacement correction, the overall RMSE was 0.89 m (0.32 TerraSAR-X pixels), compared to an initial RMSE of 11.54 m (4.2 TerraSAR-X pixels).

[0091] Furthermore, the corrected images still exhibit directional variations in the residual displacement, which is particularly evident in TerraSAR-X imagery. During the image matching process, the respective X / Y displacements relative to the spatial resolution clearly do not exceed sub-pixel accuracy. Experiments based on over 40 TerraSAR-X datasets show that they do not depend on a regular distribution of tie points within a dense grid. Furthermore, a correlation with the number of tie points and the matching window size cannot be determined. The reason is that the initial misregistration pattern can be modeled by a more or less affine transformation, which preserves straight lines and planes but only allows for scaling, translation, and rotation. Therefore, residual directional shifts are to be expected, as this assumption is relied upon when calculating the transformation parameters for misdetection and image distortion. Assuming a more complex transformation model, or one with higher polynomials, could reduce these effects.

[0092] 2. Computational efficiency analysis: The algorithm of this case solution is run on a server with a 32-core CPU (2.3GHz) and 256GB of working memory to verify the computational efficiency of the algorithm of this case solution.

[0093] The benchmark uses tie point grid resolutions between 1000 and 100 pixels of the target image, creating from 70 to 8073 tie points. The time measurement is divided into: (1) initialization of the registration workflow, including disk access, image footprint detection, and coordinate grid equalization; (2) tie point grid calculation and filtering; and (3) image warping and displacement correction.

[0094] Initialization is performed in a single CPU process, while the rest of the process uses all available CPUs. Registration with up to nearly 2000 tie points can be performed in a total processing time of 1 minute. Figure 5 As shown in the figure, further time savings can be achieved by using fewer TPs, which may be sufficient for simple registration patterns. For example, when only 100 connection points are used, the processing time is reduced to approximately 30 seconds, while the workflow initialization rate exceeds 50%. There is a time overhead for multiprocessing to perform additional multiprocessing-specific background tasks (Python object serialization process), which shows that in a single pass, less time is required per CPU core to process the same number of TPs.

[0095] The above data demonstrates that our approach can safely handle high heterogeneity between input images and provide registration results in the sub-pixel domain for both optical and radar satellite imagery. Registration accuracy is highly dependent on the spatial resolution of the input images, the complexity of the registration pattern, the similarity of the input images in terms of surface cover dynamics, the signal-to-noise ratio of the input images, and the selected user-specified parameters. This integration of displacement detection and assessment into a unified framework allows deployment across various platforms, facilitating subsequent monitoring or parameter mapping data analysis applications.

[0096] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0097] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0098] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0099] Those skilled in the art will appreciate that all or part of the steps in the above method can be performed by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk. Alternatively, all or part of the steps in the above embodiment can be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or software functional modules. The present invention is not limited to any specific combination of hardware and software.

[0100] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for automatic registration of multi-source satellite data images, characterized in that: Include: For multispectral satellite data images, a single-band reference image and a single-band image to be corrected are obtained through optimal band selection. Data preprocessing operations are then performed on the reference image and the image to be corrected. The data preprocessing operations include image overlap area calculation, pixel coordinate dense grid equalization processing, and matching window adaptive adjustment. The geometric displacement of the input reference image and the image to be corrected is detected, wherein the geometric displacement detection is realized by processing each point of the dense grid in the reference image and the image to be corrected in a sliding window manner during local registration, and the geometric displacement detection is realized by processing a single coordinate position during global registration; and the geometric displacement detection of the input reference image and the image to be corrected is also performed, which further includes: using fast Fourier transform to convert the input reference image and the image in the matching window in the image to be corrected into the frequency domain, performing phase correlation on the image in the frequency domain and generating a cross power spectrum; and converting the image to the spatial domain through inverse fast Fourier transform, and normalizing the spatial domain through the cross power spectrum. to obtain the maximum peak position; obtain an integer offset based on the distance between the maximum peak position and the center position of the cross-power spectrum, adjust the matching window coordinates based on the integer offset and construct a subset image of the reference benchmark image and the image to be corrected at the new matching window position; convert the two subset images into the frequency domain again and create a new cross-power spectrum, and obtain the sub-pixel displacement based on the maximum peak position of the new cross-power spectrum in the spatial domain and the direct displacement in the X and Y directions; add the sub-pixel displacement to the target image, and convert the total displacement in pixel units into mapping units to obtain the geometric displacement between the input reference benchmark image and the image in the image to be corrected; then, use the preset complementary verification rule to verify the spatial displacement matching between the images; The geometric displacement of the detected image is corrected and verified. During local registration, the geometric displacement is verified and corrected by distorting the target image in the image to be calibrated. During global registration, the geometric displacement is verified and corrected by the X / Y displacement vector of a single pixel point.

2. The method for automatic registration of multi-source satellite data images according to claim 1, characterized in that: Before obtaining a single-band reference benchmark image and a single-band image to be corrected through optimal band selection, it also includes: setting a dense grid connection point distance according to the resolution of the reference benchmark image.

3. The method for automatic registration of multi-source satellite data images according to claim 1, characterized in that: Obtain a single-band reference image and a single-band image to be corrected through optimal band selection, including: First, for multispectral images, the wavelength position of the target band is considered. If the corresponding image data is available, the optimal band of the image is selected based on the band with the minimum spectral distance. Otherwise, the first band is used as the optimal band of the image. Then, single-band image data is obtained based on the best image band.

4. The method for automatic registration of multi-source satellite data images according to claim 1, characterized in that: Image overlap area calculation, including: First, the bad data masks are separated by combining the preset masks to fuse the data-free masks in the image; Next, raster-to-vector conversion is applied to the no-data mask to calculate the covering polygons of the reference fiducial image and the image to be corrected; Then, the image space overlapping area of ​​the reference benchmark image and the image to be corrected is obtained by identifying the common range of the covering polygon.

5. The method for automatic registration of multi-source satellite data images according to claim 4, characterized in that: Adaptive adjustment of the matching window includes: for local registration, the matching window position is determined by using dense grid coordinates, and for global registration, the matching window position is determined by the center of the covering polygon.

6. The method for automatic registration of multi-source satellite data images according to claim 1, characterized in that: The preset complementary verification rules are used to verify the spatial displacement of image matching, including: first, correcting the integer displacement within the corresponding matching window by moving the image, and checking the validity of the integer displacement; then, using a threshold check to eliminate the displacement that does not meet the preset threshold; then, in the local displacement correction, the maximum peak of the cross power spectrum is used to evaluate the reliability of the outlier connection point, the average structural similarity index is used to evaluate the similarity of the target image content in the matching window before and after shift correction, and the random sample consistency algorithm is used to identify dense grid coordinate outliers.

7. The method for automatic registration of multi-source satellite data images according to claim 1, characterized in that: Correction verification of the detected image geometric displacement also includes: determining whether the input image provides unequal mapping projections. If so, using displacement mapping projections when the image is distorted, aligning the dense grid of the image to be corrected with the dense grid of the reference benchmark image, or changing the spatial resolution.

8. A multi-source satellite data image automatic registration system, characterized in that: The method according to claim 1 is implemented, comprising: a data preprocessing module, a displacement detection module and a displacement correction module, wherein: The data preprocessing module is used to obtain a single-band reference image and a single-band image to be corrected by selecting the optimal band for multispectral satellite data images, and perform data preprocessing operations on the reference image and the image to be corrected. The data preprocessing operations include image overlap area calculation, pixel coordinate dense grid equalization processing, and matching window adaptive adjustment. The displacement detection module is used to detect the geometric displacement of the input reference image and the image to be corrected. In local registration, the geometric displacement detection is realized by processing each point of the dense grid in the reference image and the image to be corrected in a sliding window manner. In global registration, the geometric displacement detection is realized by processing a single coordinate position. The displacement correction module is used to correct and verify the geometric displacement of the detected image. During local registration, the geometric displacement is verified and corrected by distorting the target image in the image to be calibrated. During global registration, the geometric displacement is verified and corrected by the X / Y displacement vector of a single pixel point.

Images