Remote sensing image geometric fine correction method, system, equipment, medium and product

Abstract

The invention provides a remote sensing image geometric fine correction method, system and device, a medium and a product, and relates to the technical field of remote sensing image processing.The method comprises the steps that a to-be-corrected remote sensing image and an initial reference image corresponding to the to-be-corrected remote sensing image are obtained, and the initial reference image is the same as the to-be-corrected remote sensing image in geographic position information and corresponds to the to-be-corrected remote sensing image; the reference images have the same modality; performing time-phase alignment processing on the initial reference image to obtain a reference image after time-phase alignment processing; performing image scale alignment processing on the reference image after the time alignment processing to obtain a target reference image; obtaining initial control point pairs, and screening all the initial control point pairs according to the pixel distance between the control points in all the initial control point pairs to obtain a target control point pair; and performing geometric fine correction processing on the to-be-corrected remote sensing image based on the geometric correction model constructed by the target control point pair to obtain a corrected remote sensing image. The correction precision of the remote sensing image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular, to a geometric fine correction method, system, device, medium and product for remote sensing images. Background Art

[0002] When a remote sensing satellite images, it is affected by various factors, including the satellite's motion state, imaging attitude, the curvature of the earth, sensor errors, and system-level geometric correction errors. The superposition of these factors will cause geometric distortions such as squeezing, stretching, twisting and offset in the image. Since these distortions are caused by complex and diverse reasons and are difficult to express in a regular way, they cannot be deterministically identified and eliminated.

[0003] The geometric registration and fine correction of remote sensing images is an important link in the image processing process and has a direct impact on the accurate extraction of the position information of ground object elements in the image. Existing geometric fine correction methods based on image matching require a high-quality reference image that is as similar as possible to the input image in terms of imaging modality, imaging time, and spatial scale. However, in practical applications, such an ideal reference image is difficult to obtain. Generally, only a corresponding part can be cropped from the existing base map as the reference image, without further requiring consistency in all aspects, resulting in low correction accuracy of remote sensing images.

[0004] Therefore, there is an urgent need for a geometric fine correction method, system, device, medium and product for remote sensing images to solve the above problems. Summary of the Invention

[0005] In view of the problems existing in the prior art, the present invention provides a geometric fine correction method, system, device, medium and product for remote sensing images.

[0006] The present invention provides a geometric fine correction method for remote sensing images, including: Obtaining a remote sensing image to be corrected and an initial reference image corresponding to the remote sensing image to be corrected, wherein the initial reference image is a reference image with the same geographical location information and the same modality as the remote sensing image to be corrected; Performing time alignment processing on the initial reference image to obtain a reference image after time alignment processing; Performing image scale alignment processing on the reference image after time alignment processing to obtain a target reference image; Obtaining an initial control point pair between the remote sensing image to be corrected and the target reference image, and screening all the initial control point pairs according to the pixel distance between the control points in all the initial control point pairs to obtain a target control point pair; Based on the geometric correction model constructed using the target control points, perform geometric fine correction on the remotely sensed image to be corrected to obtain the corrected remotely sensed image.

[0007] According to a method for geometric fine correction of remotely sensed images provided by the present invention, before obtaining the remotely sensed image to be corrected and the initial reference image corresponding to the remotely sensed image to be corrected, the method further includes: Obtain a historical reference image, where the geographical location information between the historical reference image and the remotely sensed image to be corrected is the same; Determine whether the modal types between the historical reference image and the remotely sensed image to be corrected are the same. If the modal types are the same, determine the historical reference image as the initial reference image; If the modal types are different, determine a target image modal transfer model from multiple image modal transfer models according to the modal type of the remotely sensed image to be corrected, and perform modal transfer processing on the historical reference image based on the target image modal transfer model to obtain the initial reference image, where the multiple image modal transfer models are obtained by training a deep neural network model according to sample remotely sensed images and the corresponding sample reference images; the modal types between the sample remotely sensed images and the sample reference images are different.

[0008] According to a method for geometric fine correction of remotely sensed images provided by the present invention, the deep neural network model is constructed by ControlNet and U-Net.

[0009] According to a method for geometric fine correction of remotely sensed images provided by the present invention, the step of performing temporal alignment on the initial reference image to obtain the temporally aligned reference image includes: Based on the optimal transport algorithm, perform temporal alignment on the pixel spectral values in the initial reference image and the pixel spectral values in the remotely sensed image to be corrected to obtain the temporally aligned reference image.

[0010] According to a method for geometric fine correction of remotely sensed images provided by the present invention, the step of performing image scale alignment on the temporally aligned reference image to obtain the target reference image includes: According to the image scale information of the remotely sensed image to be corrected, obtain the spatial resolution of the remotely sensed image to be corrected; Based on the spatial resolution of the remotely sensed image to be corrected, resample the temporally aligned reference image to obtain the target reference image with the same spatial resolution as the remotely sensed image to be corrected.

[0011] A geometric precise correction method for remote sensing images provided by the present invention, the method for obtaining initial control point pairs between the to-be-corrected remote sensing image and the target reference image, and screening all the initial control point pairs according to the pixel distances between the control points in all the initial control point pairs to obtain target control point pairs, includes: Based on the image sizes of the to-be-corrected remote sensing image and the target reference image, group all the initial control point pairs to obtain multiple initial control point pair groups; In the order from high to low similarity values of the initial control point pairs, calculate the pixel distances between the current-order initial control point pair in the initial control point pair group and other initial control point pairs in turn, and delete the other initial control point pairs with pixel distances less than the first distance threshold from the initial control point pair group until the pixel distances between all the initial control point pairs in the initial control point pair group are greater than or equal to the first distance threshold, to obtain a preliminarily screened initial control point pair group; Construct a corresponding homography matrix according to the initial control point pairs in the preliminarily screened initial control point pair group; Based on the homography matrix, convert the target control point pixel coordinates corresponding to the preliminarily screened initial control point pair group in the target reference image to obtain undetermined control point pixel coordinates, and obtain the reference control point pixel coordinates corresponding to the target control point pixel coordinates, where the undetermined control point pixel coordinates are the pixel coordinates corresponding in the to-be-corrected remote sensing image after the control point pixel coordinates in the target reference image are converted by the homography matrix; the reference control point pixel coordinates represent the control point pixel coordinates corresponding to the target control point pixel coordinates in the to-be-corrected remote sensing image in an initial control point pair; Calculate the coordinate distances between the reference control point pixel coordinates and the undetermined control point pixel coordinates, and delete the initial control point pairs with coordinate distances greater than the second distance threshold in the preliminarily screened initial control point pair group until the coordinate distances corresponding to all the initial control point pairs in the preliminarily screened initial control point pair group are less than or equal to the second distance threshold, to obtain the target control point pairs.

[0012] The present invention also provides a geometric precise correction system for remote sensing images, including: A remote sensing data acquisition module, configured to acquire a to-be-corrected remote sensing image and an initial reference image corresponding to the to-be-corrected remote sensing image, where the initial reference image is a reference image with the same geographical location information and the same modality as the to-be-corrected remote sensing image; A first alignment processing module, configured to perform time alignment processing on the initial reference image to obtain a time-aligned reference image; The second alignment processing module is configured to perform image scale alignment processing on the reference image after the time-phase alignment processing to obtain a target reference image; The control point screening module is configured to obtain an initial control point pair between the to-be-corrected remote sensing image and the target reference image, and screen all the initial control point pairs according to the pixel distance between the control points in all the initial control point pairs to obtain a target control point pair; The image correction module is configured to perform geometric fine correction processing on the to-be-corrected remote sensing image based on the geometric correction model constructed based on the target control point pair to obtain a corrected remote sensing image.

[0013] The present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the geometric fine correction method for remote sensing images as described in any one of the above is implemented.

[0014] The present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the geometric fine correction method for remote sensing images as described in any one of the above is implemented.

[0015] The present invention further provides a computer program product, including a computer program. When the computer program is executed by a processor, the geometric fine correction method for remote sensing images as described in any one of the above is implemented.

[0016] The geometric fine correction method, system, device, medium, and product for remote sensing images provided by the present invention obtain a to-be-corrected remote sensing image and its corresponding reference image, and then perform time-phase alignment and scale alignment processing on the reference image, obtain and screen a target control point pair, and thus construct a geometric correction model based on the target control point pair to implement fine correction of the remote sensing image, improving the correction accuracy of the remote sensing image. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic flowchart of the geometric fine correction method for remote sensing images provided by the present invention; Figure 2 It is a schematic diagram of the image modality migration process provided by the present invention; Figure 3Schematic diagram of the overall process of image modality transfer provided by the present invention; Figure 4 Effect diagram of temporal alignment of remote sensing images based on the optimal transport algorithm provided by the present invention; Figure 5 Effect diagram of image scale alignment based on resampling provided by the present invention; Figure 6 Schematic diagram of the structure of the geometric precise correction system for remote sensing images provided by the present invention; Figure 7 Schematic diagram of the structure of the electronic device provided by the present invention. Detailed implementation manners

[0019] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0020] The existence of geometric distortion will reduce the application value and application ability of remote sensing images. For example, in applications such as disaster monitoring, geometric errors will affect the accuracy of the assessment of the disaster area and the degree of disaster, and reduce the reliability of auxiliary decision-making; while in applications such as change detection, pattern recognition, and image fusion, geometric errors will cause remote sensing images with different sensors, different imaging times, and different spectral ranges to be unable to be accurately superimposed together, and unable to accurately extract the spatial position information or change information of ground objects.

[0021] Geometric precise correction of remote sensing images is to correct and eliminate these geometric distortions by using known ground control points or mathematical models. Considering the difficulty and regional limitations of obtaining ground control points, the method of matching the image to be corrected with a high-precision reference image to achieve geometric precise correction without control points or with few control points has wide application value.

[0022] The accuracy of existing matching and calibration methods depends to a large extent on geometric reference images. Based on the quality of the reference images themselves, it also involves the consistency between the reference images and the images to be calibrated in terms of modality, temporal phase, and spatial scale. However, in practical applications, it is difficult to obtain reference images that are ideal in all aspects. Generally, only the corresponding parts are intercepted from existing base maps as reference images, and no further requirement for consistency in all aspects is imposed. Currently, customizing and generating the required reference images using image simulation technology is an ideal technical approach, but it is difficult in terms of methods. Especially, the mutual migration and conversion between different modality images such as visible light, infrared, hyperspectral, synthetic aperture radar (SAR for short), and light detection and ranging (LiDAR for short) have long been a very difficult task, and the accuracy and reliability of the conversion are relatively low.

[0023] In view of the problems existing in the above-mentioned prior art, the present invention provides a geometric precise calibration method for remote sensing images based on simulated reference images. By means of image simulation and migration technology, a reference image that is more consistent with the input image to be calibrated in multiple dimensions such as modality, time, and space is generated, thereby improving the geometric calibration accuracy of remote sensing images.

[0024] Figure 1 It is a schematic flowchart of the geometric precise calibration method for remote sensing images provided by the present invention. As Figure 1 shown, the present invention provides a geometric precise calibration method for remote sensing images, including: Step 101, obtaining the remote sensing image to be calibrated and the initial reference image corresponding to the remote sensing image to be calibrated, where the initial reference image is a reference image with the same geographical location information as the remote sensing image to be calibrated and the same modality.

[0025] In the present invention, first, it is determined whether the initial reference image is consistent with the modality of the remote sensing image to be calibrated. If they are consistent, subsequent processes such as temporal alignment and image scale alignment can be directly carried out. If they are inconsistent, corresponding modality conversion processing needs to be performed on the reference image. For example, if the image to be calibrated is a synthetic aperture radar image and the obtained reference image is a visible light image, modality conversion is performed through an image modality migration network constructed based on a diffusion model. In the present invention, the modality types of remote sensing images mainly include visible light, infrared, hyperspectral, synthetic aperture radar, and lidar, etc.

[0026] In the present invention, the initial reference image is an existing image that corresponds to the remote sensing image to be corrected and is considered to be relatively accurate or standard. Moreover, the initial reference image and the remote sensing image to be corrected cover the same geographical area to ensure that the two can be effectively compared and corrected. It should be noted that in the present invention, the initial reference image usually comes from historical data, which has already undergone certain processing and verification, provides temporal continuity, can be used to monitor and analyze changes in geographical areas, and reduces the cost and time of acquiring new data.

[0027] Step 102: Perform temporal alignment processing on the initial reference image to obtain the reference image after temporal alignment processing.

[0028] Since the spectral performance of the ground objects in the remote sensing image to be corrected is highly correlated with its temporal states such as the year, season, and time period when the image was taken. For example, the vegetation in winter and summer is very different, and the temporal states between the obtained initial reference image and the remote sensing image to be corrected are not completely consistent. Therefore, in the present invention, based on the optimal transport algorithm, temporal alignment is achieved through the optimal transport mapping between the image spectral distributions. Optimal transport is an algorithm that migrates between two probability distributions at the minimum cost. By counting the spectral values of all pixels in each of the two images as two distributions, the mapping relationship between the spectral values can be calculated through the optimal transport algorithm, thereby achieving pixel-by-pixel temporal migration.

[0029] Step 103: Perform image scale alignment processing on the reference image after temporal alignment processing to obtain the target reference image.

[0030] In the present invention, the reference image after temporal alignment processing in the above embodiment is transformed so that the obtained target reference image has the same spatial resolution as the remote sensing image to be corrected, thereby eliminating the influence of different scales on geometric feature extraction. In the present invention, the image scale alignment processing process can be achieved by means of image bilinear interpolation resampling.

[0031] Step 104: Obtain the initial control point pairs between the remote sensing image to be corrected and the target reference image, and screen all the initial control point pairs according to the pixel distances between the control points in all the initial control point pairs to obtain the target control point pairs.

[0032] Since the target reference image and the remote sensing image to be corrected already have a high degree of similarity and consistency. Therefore, in the present invention, a large number of dense control point pairs can be obtained through feature extraction and feature matching methods. In order to obtain more accurate control point pairs, the present invention uses a method that combines the spatial distance, similarity, and homography matrix of the control points to screen the optimal control point pairs that meet the geometric correction requirements.

[0033] Step 105: Based on the geometric correction model constructed using the target control point pairs, perform geometric fine correction on the remotely sensed image to be corrected to obtain the corrected remotely sensed image.

[0034] In the present invention, by using the selected control point pairs, a geometric correction model is constructed, and thus, through this geometric correction model, geometric fine correction is performed on the remotely sensed image to be corrected, and finally, the corrected result image, i.e., the corrected remotely sensed image, is output. In the present invention, the geometric correction model can be constructed based on methods such as projective transformation, first-order polynomial, second-order polynomial, third-order polynomial, and triangulation network.

[0035] The geometric fine correction method for remotely sensed images provided by the present invention obtains the remotely sensed image to be corrected and its corresponding reference image, then performs temporal alignment and scale alignment processing on the reference image, obtains and selects the target control point pairs, and thus constructs a geometric correction model based on the target control point pairs to achieve fine correction of the remotely sensed image and improve the correction accuracy of the remotely sensed image.

[0036] Based on the above embodiments, before obtaining the remotely sensed image to be corrected and the initial reference image corresponding to the remotely sensed image to be corrected, the method further includes: Obtain a historical reference image, where the geographical location information between the historical reference image and the remotely sensed image to be corrected is the same; Determine whether the modal types between the historical reference image and the remotely sensed image to be corrected are the same. If the modal types are the same, determine the historical reference image as the initial reference image; If the modal types are different, determine a target image modal transfer model from multiple image modal transfer models according to the modal type of the remotely sensed image to be corrected, and based on the target image modal transfer model, perform modal transfer processing on the historical reference image to obtain the initial reference image, where the multiple image modal transfer models are obtained by training a deep neural network model according to a sample remotely sensed image and a sample reference image corresponding to the sample remotely sensed image; the modal types between the sample remotely sensed image and the sample reference image are different.

[0037] In the present invention, first, a historical reference image with the same geographical location information as the remotely sensed image to be corrected is obtained, indicating that the two images cover the same area. Moreover, the historical reference image is usually an image that has been corrected, has high quality, and can be used as a reference.

[0038] Furthermore, determine whether the modal types of this historical reference image and the remote sensing image to be corrected are the same. The modal type refers to the way or technology of image acquisition, such as visible light images, infrared images, and radar images, etc. Different modal types may have different spectral characteristics, resolutions, and imaging conditions.

[0039] If the modal types of the historical reference image and the remote sensing image to be corrected are the same, then the historical reference image can be directly used as the initial reference image.

[0040] If the modal types of the historical reference image and the remote sensing image to be corrected are different, then modal transfer processing is required. Specifically, according to the modal type of the remote sensing image to be corrected, select a suitable model from multiple pre-trained image modal transfer models. These image modal transfer models are trained based on sample remote sensing images and corresponding sample reference images, and can learn how to convert an image of one modality into an image of another modality. Then, use the selected target image modal transfer model to process the historical reference image and convert it into an image of the same modality as the remote sensing image to be corrected, thereby obtaining the initial reference image, which can be used for subsequent image correction processing.

[0041] In the present invention, multiple image modal transfer models are obtained by training a deep neural network model, where the deep neural network model is constructed by ControlNet and U-Net. In the present invention, modal transfer is implemented based on a diffusion model. Taking the conversion from synthetic aperture radar images to visible light images as an example, the conversion between other modalities also uses the same method, and the network model between each two modalities is trained separately. Figure 2 For the schematic diagram of the image modal transfer process provided by the present invention, the process of converting a reference image with the modality of synthetic aperture radar images into a reference image with the visible light image modality can be referred to Figure 2 as shown, where , , , and are the corresponding images (i.e., intermediate images) formed at different time steps during the image modal transfer process of the input synthetic aperture radar images.

[0042] Figure 3 For the schematic diagram of the overall process of the image modal transfer provided by the present invention, it can be referred to Figure 3 as shown. During the training stage of the model, the input samples are SAR images and visible light images, where SAR images are input as conditional information ControlNet . During the training process, ControlNet extracts the multi-level features of SAR images , these multi-level features (e.g., edges, colors, contours, brightness, and textures, etc.) serve as key conditional features to guide the generation of visible light images.

[0043] In the present invention, the visible light image is the target image, and noise is gradually added during the diffusion process to generate a noise image at a specific time step t . This process is the degradation of the original image to completely random noise, providing input for the subsequent inverse diffusion process. Subsequently, the noise-added visible light image is input into U-Net , combined with SAR the conditional information of the image and the time step t features, to gradually remove the noise, thereby restoring the predicted image, i.e., the restored image .

[0044] In the present invention, U-Net the structure consists of an encoder and a decoder. Among them, the encoder is responsible for extracting high-dimensional features, while the decoder, through the skip connection mechanism, combines low-level and high-level features to make the generation process more accurate and delicate. The extracted SAR conditional features of the image are embedded into the network through a multi-layer fusion method to ensure that the model can comprehensively utilize multi-modal data for image restoration and generation.

[0045] In each iteration, by comparing U-Net the predicted noise and the real noise , the objective function is optimized, thereby gradually improving the performance of the image modality transfer model in the inverse diffusion stage. Finally, the visible light image generated by the image modality transfer model, i.e., the restored image is as close as possible to the original visible light image , achieving high-quality restoration from the noise image to the clear image.

[0046] After the model training is completed, it needs to go through the noise prediction stage and the inverse diffusion stage. Among them, noise prediction only uses ControlNet and U-Net the decoder part of SAR . Similar to the training process, ControlNet the image is first input into U-Net to extract multi-level features as conditional information. Subsequently, the conditional features are input into the decoder of

[0047] ​During the inverse diffusion process, an image with high noise (i.e., a noisy image ) is gradually generated into a visible light image . Each inverse diffusion time step is executed to generate the corresponding intermediate image , , until a clear image is restored . The total number of time steps in the entire inverse diffusion process is n steps ( n-steps ). Each step of this process depends on the output of the noise prediction module SAR . The

[0048] image provides key conditional information to help the model more accurately restore the details in the visible light image. Based on the optimal transport algorithm, the pixel spectral values in the initial reference image are temporally aligned with the pixel spectral values in the remote sensing image to be corrected, and the reference image after temporal alignment processing is obtained.

[0049] In the present invention, the optimal transport algorithm (Optimal Transport, abbreviated as OT) is used to solve the problem of the minimum cost required to convert one distribution into another. In the field of image processing, it can be used to compare, match, or align features in two images, such as pixel values and spectral characteristics. The core of the optimal transport algorithm lies in finding an optimal "transport plan", that is, how to map the pixels (or features) in one image to the corresponding pixels (or features) in another image in the most efficient way, so that a certain cost function (such as distance, difference, etc.) is minimized.

[0050] In the present invention, through the optimal transport algorithm, the multi-band probability distributions of the initial reference image and the remote sensing image to be corrected are transformed at the minimum cost to achieve the purpose of temporal alignment. Assume represents the source image (i.e., the initial reference image), represents the probability distribution of the pixel values of the target image (i.e., the remote sensing image to be corrected) that the source image is to be aligned to, all possible values in include different values, all possible values in include different values. is the probability vector of each value in , that is, the rd value in the vector represents in the probability of occurrence; is the probability vector of each value in, that is, the vector the th value represents in the probability of occurrence. All the above values can be directly calculated from the image.

[0051] Furthermore, let the matrix represent and the distance matrix between each element of, each element in represents and the Euclidean distance between, then the optimal transport problem can be defined as: ; ; ; where is and the migration loss between, which is the object to be minimized; represents and the joint probability distribution between; and represent vectors where all elements are 1 and the lengths are and respectively; represents the optimal transport plan that can minimize the migration loss, which is one of all possible joint probability distributions. Among them, represents migrated from to

[0052] Furthermore, using algorithms such as Sinkhorn, the above optimal transport problem can be quickly solved. After obtaining a transport plan , the pixel spectral value mapping relationship in the process of migrating the source image to the target image is , that is, it can be calculated that each value in corresponds to the value after being migrated to the Figure 4The figure shows the effect of temporal alignment of remote sensing images based on the optimal transport algorithm provided by the present invention. By inputting an image (i.e., the remote sensing image to be corrected), the existing reference image (such as the reference image representing spring and summer) is temporally aligned with the remote sensing image to be corrected (such as the image representing autumn and winter), and the temporally aligned reference image obtained can be referred to Figure 4 as shown

[0053] Since the number of pixels in remote sensing images is very large, a whole image generally contains tens of millions or hundreds of millions of pixels, and there are also a very large number of possible pixel values in the case of multiple bands. Optionally, before calculating the optimal transport between images, the present invention first performs pixel sampling. The sampling quantity is determined according to the image size, and the maximum number of pixel samples for each image does not exceed 10,000. Then, the optimal transport mapping relationship is calculated between the two sets of sampled data (i.e., the initial reference image and the remote sensing image to be corrected).

[0054] Table 1 shows the mapping relationship of pixel spectral values between a group of 300 samples, which can be referred to as shown in Table 1 Table 1 Example of the mapping representation of pixel spectral values in image bands generated by the optimal transport algorithm

[0055] In Table 1, according to the mapping relationship of these 300 samples, all pixels in the whole reference image are migrated. The specific method is that for any pixel, first find the pixel in the samples (i.e., the pixel values corresponding to the serial numbers in Table 1) that is closest to its pixel value. For example, for a pixel with a pixel value of 203 in the reference image, the pixel value in the samples that is closest to it is 200. The difference between the two is retained, and then the closest pixel value is migrated. The pixel value corresponding to 200 is 191 plus the difference between the two (203 - 200 = 3). Finally, the pixel with a pixel value of 203 in the reference image is mapped to 191 + 3 = 196 in the target image. Further, in the above manner, all pixels in the initial reference image are migrated to the remote sensing image to be corrected, and then the temporal alignment from the reference image to the remote sensing image to be corrected is completed

[0056] On the basis of the above embodiments, performing image scale alignment processing on the reference image after the temporal alignment processing to obtain a target reference image includes acquiring the spatial resolution of the remote sensing image to be corrected according to the image scale information of the remote sensing image to be corrected resampling the reference image after the temporal alignment processing based on the spatial resolution of the remote sensing image to be corrected to obtain the target reference image with the same spatial resolution as the remote sensing image to be corrected

[0057] In the present invention, the image scale alignment process between the reference image after time-phase alignment and the remote sensing image to be corrected can be achieved through resampling calculation. The present invention comprehensively considers accuracy and efficiency and selects bilinear interpolation to resample the image. Since the reference image in the geometric matching and precise correction scenario usually has a high spatial resolution, after operations such as modality transfer and time-phase alignment in the above embodiments, the spatial resolution of the reference image is resampled to be the same as that of the remote sensing image to be corrected. Figure 5 It is a schematic diagram of the effect of image scale alignment based on resampling provided by the present invention. The reference image after scale alignment (i.e., the target reference image) can be referred to Figure 5 as shown.

[0058] Based on the above embodiments, the obtaining of the initial control point pairs between the remote sensing image to be corrected and the target reference image, and the screening of all the initial control point pairs according to the pixel distances between the control points in all the initial control point pairs to obtain the target control point pairs includes: Grouping all the initial control point pairs based on the image sizes of the remote sensing image to be corrected and the target reference image to obtain a plurality of initial control point pair groups; Calculating the pixel distances between the current-order initial control point pair in the current initial control point pair group and other initial control point pairs in turn according to the similarity values of the initial control point pairs from high to low, and deleting the other initial control point pairs with pixel distances less than the first distance threshold from the initial control point pair group until the pixel distances between all the initial control point pairs in the initial control point pair group are greater than or equal to the first distance threshold, to obtain the initially screened initial control point pair group; Constructing a corresponding homography matrix according to the initial control point pairs in the initially screened initial control point pair group; Based on the homography matrix, converting the target control point pixel coordinates corresponding to the initially screened initial control point pair group in the target reference image to obtain the pending control point pixel coordinates, and obtaining the reference control point pixel coordinates corresponding to the target control point pixel coordinates, where the pending control point pixel coordinates are the pixel coordinates corresponding to the control point pixel coordinates in the target reference image after being converted by the homography matrix in the remote sensing image to be corrected; the reference control point pixel coordinates represent the control point pixel coordinates corresponding to the target control point pixel coordinates in the remote sensing image to be corrected in an initial control point pair; Calculate the coordinate distance between the reference control point pixel coordinates and the to-be-determined control point pixel coordinates, and delete the initial control point pairs in the initially screened initial control point pair groups where the coordinate distance is greater than the second distance threshold until the coordinate distances corresponding to the initial control point pairs in all initially screened initial control point pair groups are less than or equal to the second distance threshold, to obtain the target control point pairs.

[0059] After the processing of the above embodiments, the similarity between the reference image (i.e., the target reference image) and the remote sensing image to be corrected is relatively high, which makes the feature extraction and matching method generate a large number of dense control point pairs, and there are a small number of control point pairs with low accuracy, thus affecting the difficulty and accuracy of constructing the geometric correction model. Therefore, after completing the feature extraction and matching to obtain a plurality of initial control point pairs, the present invention uses a method combining the spatial distance, similarity, and homography matrix of the control points to screen the optimal control point pairs that meet the geometric correction requirements. The specific steps are as follows: Step 201, automatic feature extraction and matching. The present invention uses existing feature extraction and feature matching algorithms to complete the feature matching between the remote sensing image to be corrected and the target reference image, generating a large number of control point pairs. Preferably, if the size of the remote sensing image to be corrected is greater than 10000×10000, image block automatic matching is performed according to 10000×10000, that is, the initial control point pairs obtained by matching are grouped based on each image block. For each pair of successfully matched control points, the result value of the feature matching (such as the cosine distance) is used as the similarity of the control points.

[0060] Step 202, control point grouping. According to the pixel positions of the control points on the remote sensing image to be corrected, the control points are divided into several groups according to the image block areas, that is, multiple initial control point pair groups.

[0061] Step 203, removing control points that do not meet the distance requirements within the group. Specifically, first, all the control points in each initially screened initial control point pair group are sorted from high to low according to the similarity. It should be noted that since the removal process is for a pair of control points, in the present invention, the pixel distance between any control point on one side of the reference image or the remote sensing image to be corrected and other control points within its own group can be used for judgment. Further, taking one of the initially screened initial control point pair groups as an example for illustration, based on the similarity sorting result, the control point with the highest similarity in this initially screened initial control point pair group is sequentially used as the seed control point (that is, according to the similarity value from high to low, the initial control point pair with the highest similarity in the initially screened initial control point pair group is selected), and the pixel distance between the seed control point and each other control point within the group is calculated. If the pixel distance is less than the first distance threshold T, then delete other control points; then, use the control point with the second highest similarity as the seed control point, continue to calculate the pixel distance between the seed control point and each of the remaining control points in the group, and delete the remaining control points in the group whose pixel distance is less than the first distance threshold T through continuous iteration and deletion in step 203 until the distances between all control points in the group are greater than the first distance threshold T , then the screening of control points in the group is completed, where the first distance threshold in the present invention T can be determined according to historical experience data.

[0062] Step 204, delete the control points in the group whose accuracy does not meet the requirements. Specifically, use all the control points retained in the group (that is, the initial control points after preliminary screening obtained by completing the screening process of control points in the group in the above embodiments) to solve the homography matrix; then, for each pair of control points in the group of initially screened initial control points, use the homography matrix to convert the pixel coordinates of the control points on the reference image in the group to the pixel coordinates of the control points on the remotely sensed image to be corrected (that is, obtain the undetermined control point pixel coordinates), and then calculate the coordinate distance between the control point coordinates (that is, the reference control point pixel coordinates) and the undetermined control point pixel coordinates on the remotely sensed image to be corrected. If the coordinate distance is greater than the second distance threshold S , then it is considered that this pair of control points is a control point whose accuracy does not meet the requirements, and delete this pair of control points; otherwise, retain this pair of control points until the coordinate distances corresponding to all the initial control point pairs in the group of initially screened initial control points are less than or equal to the second distance threshold, and obtain the target control point pairs. It should be noted that the second distance threshold in the present invention S can be determined according to historical experience data.

[0063] The geometric precise rectification system for remotely sensed images provided by the present invention will be described below. The geometric precise rectification system for remotely sensed images described below can be mutually referred to with the geometric precise rectification method for remotely sensed images described above.

[0064] Figure 6 is a schematic structural diagram of the geometric precise rectification system for remotely sensed images provided by the present invention, as Figure 6As shown in the figure, the present invention provides a geometric precise rectification system for remote sensing images, including a remote sensing data acquisition module 601, a first alignment processing module 602, a second alignment processing module 603, a control point screening module 604, and an image rectification module 605. Among them, the remote sensing data acquisition module 601 is used to obtain the remote sensing image to be rectified and the initial reference image corresponding to the remote sensing image to be rectified. The initial reference image is a reference image with the same geographical location information and the same modality as the remote sensing image to be rectified. The first alignment processing module 602 is used to perform temporal alignment processing on the initial reference image to obtain the reference image after temporal alignment processing. The second alignment processing module 603 is used to perform image scale alignment processing on the reference image after temporal alignment processing to obtain the target reference image. The control point screening module 604 is used to obtain the initial control point pairs between the remote sensing image to be rectified and the target reference image, and screen all the initial control point pairs according to the pixel distances between the control points in all the initial control point pairs to obtain the target control point pairs. The image rectification module 605 is used to perform geometric precise rectification processing on the remote sensing image to be rectified based on the geometric rectification model constructed by the target control point pairs to obtain the rectified remote sensing image.

[0065] The geometric precise rectification system for remote sensing images provided by the present invention obtains the remote sensing image to be rectified and its corresponding reference image, then performs temporal alignment and scale alignment processing on the reference image, obtains and screens the target control point pairs, and thus constructs a geometric rectification model based on the target control point pairs to achieve precise rectification of the remote sensing image and improve the rectification accuracy of the remote sensing image.

[0066] The system provided by the embodiments of the present invention is used to execute the above method embodiments. For the specific process and detailed content, please refer to the above embodiments and will not be elaborated here.

[0067] Figure 7 It is a schematic structural diagram of the electronic device provided by the present invention, as Figure 7As shown in the figure, the electronic device may include: a processor 701, a communications interface 702, a memory 703, and a communication bus 704. Among them, the processor 701, the communications interface 702, and the memory 703 complete communication with each other through the communication bus 704. The processor 701 may call the logical instructions in the memory 703 to execute the geometric precise correction method for remote sensing images. The method includes: obtaining the to-be-corrected remote sensing image and the initial reference image corresponding to the to-be-corrected remote sensing image, where the initial reference image is a reference image with the same geographical location information and the same modality as the to-be-corrected remote sensing image; performing time alignment processing on the initial reference image to obtain the reference image after time alignment processing; performing image scale alignment processing on the reference image after time alignment processing to obtain the target reference image; obtaining the initial control point pairs between the to-be-corrected remote sensing image and the target reference image, and screening all the initial control point pairs according to the pixel distances between the control points in all the initial control point pairs to obtain the target control point pairs; performing geometric precise correction processing on the to-be-corrected remote sensing image based on the geometric correction model constructed based on the target control point pairs to obtain the corrected remote sensing image.

[0068] In addition, when the logical instructions in the above-mentioned memory 703 can be implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0069] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the remote sensing image geometric precise correction method provided by each of the above methods. The method includes: obtaining a to-be-corrected remote sensing image and an initial reference image corresponding to the to-be-corrected remote sensing image, where the initial reference image is a reference image with the same geographical location information and the same modality as the to-be-corrected remote sensing image; performing time alignment processing on the initial reference image to obtain a time-aligned reference image; performing image scale alignment processing on the time-aligned reference image to obtain a target reference image; obtaining an initial control point pair between the to-be-corrected remote sensing image and the target reference image, and screening all the initial control point pairs according to the pixel distance between the control points in all the initial control point pairs to obtain a target control point pair; performing geometric precise correction processing on the to-be-corrected remote sensing image based on the geometric correction model constructed from the target control point pair to obtain a corrected remote sensing image.

[0070] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the remote sensing image geometric precise correction method provided by each of the above embodiments. The method includes: obtaining a to-be-corrected remote sensing image and an initial reference image corresponding to the to-be-corrected remote sensing image, where the initial reference image is a reference image with the same geographical location information and the same modality as the to-be-corrected remote sensing image; performing time alignment processing on the initial reference image to obtain a time-aligned reference image; performing image scale alignment processing on the time-aligned reference image to obtain a target reference image; obtaining an initial control point pair between the to-be-corrected remote sensing image and the target reference image, and screening all the initial control point pairs according to the pixel distance between the control points in all the initial control point pairs to obtain a target control point pair; performing geometric precise correction processing on the to-be-corrected remote sensing image based on the geometric correction model constructed from the target control point pair to obtain a corrected remote sensing image.

[0071] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0072] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing image geometric precision correction method, characterized in that: include: Acquire a remote sensing image to be corrected and an initial reference image corresponding to the remote sensing image to be corrected, wherein the initial reference image is a reference image having the same geographical location information and the same modality as the remote sensing image to be corrected; Performing a time-phase alignment process on the initial reference image to obtain a reference image after the time-phase alignment process; Performing image scale alignment processing on the reference image after the time phase alignment processing to obtain a target reference image; Acquire initial control point pairs between the remote sensing image to be corrected and the target reference image, and screen all the initial control point pairs according to the pixel distances between the control points in all the initial control point pairs to obtain target control point pairs; Based on the geometric correction model constructed by the target control point pair, the remote sensing image to be corrected is subjected to geometric precision correction processing to obtain a corrected remote sensing image.

2. The remote sensing image geometric precision correction method according to claim 1, characterized in that: Before acquiring the remote sensing image to be corrected and the initial reference image corresponding to the remote sensing image to be corrected, the method further includes: Acquire a historical reference image, wherein the geographical location information between the historical reference image and the remote sensing image to be corrected is the same; Determining whether the modality types of the historical reference image and the remote sensing image to be corrected are the same, and if the modality types are the same, determining the historical reference image as the initial reference image; If the modality types are not the same, a target image modality migration model is determined from multiple image modality migration models according to the modality type of the remote sensing image to be corrected, and based on the target image modality migration model, modality migration processing is performed on the historical reference image to obtain the initial reference image, wherein the multiple image modality migration models are obtained by training a deep neural network model based on sample remote sensing images and sample reference images corresponding to the sample remote sensing images; and the modality types between the sample remote sensing image and the sample reference image are different.

3. The remote sensing image geometric precision correction method according to claim 2, characterized in that: The deep neural network model is constructed by ControlNet and U-Net.

4. The remote sensing image geometric precision correction method according to claim 1, characterized in that: The performing time-phase alignment processing on the initial reference image to obtain the reference image after the time-phase alignment processing includes: Based on the optimal transport algorithm, the pixel spectrum values ​​in the initial reference image are aligned in time with the pixel spectrum values ​​in the remote sensing image to be corrected to obtain the reference image after the time alignment process.

5. The remote sensing image geometric precision correction method according to claim 1 or 4, characterized in that: The step of performing image scale alignment processing on the reference image after the time-phase alignment processing to obtain a target reference image includes: Acquiring the spatial resolution of the remote sensing image to be corrected according to the image scale information of the remote sensing image to be corrected; Based on the spatial resolution of the remote sensing image to be corrected, the reference image after the time alignment process is resampled to obtain the target reference image with the same spatial resolution as the remote sensing image to be corrected.

6. The remote sensing image geometric precision correction method according to claim 1, characterized in that: The step of obtaining initial control point pairs between the remote sensing image to be corrected and the target reference image, and screening all the initial control point pairs according to the pixel distances between the control points in all the initial control point pairs to obtain the target control point pairs includes: Based on the image sizes of the remote sensing image to be corrected and the target reference image, grouping all the initial control point pairs to obtain a plurality of initial control point pair groups; Calculating the pixel distances between the initial control point pair in the current order and other initial control point pairs in the initial control point pair group in descending order of the similarity values ​​of the initial control point pairs, and deleting the other initial control point pairs whose pixel distances are less than a first distance threshold from the initial control point pair group, until the pixel distances between all the initial control point pairs in the initial control point pair group are greater than or equal to the first distance threshold, thereby obtaining a preliminarily screened initial control point pair group; According to the initial control point pairs in the initial control point pair group after the preliminary screening, construct a corresponding homography matrix; Based on the homography matrix, the initial control point pairs after the preliminary screening are grouped into the target control point pixel coordinates corresponding to the target reference image to obtain the undetermined control point pixel coordinates, and the reference control point pixel coordinates corresponding to the target control point pixel coordinates are obtained, wherein the undetermined control point pixel coordinates are the control point pixel coordinates in the target reference image after the control point pixel coordinates are transformed by the homography matrix, and the pixel coordinates of the reference control point correspond to the pixel coordinates in the remote sensing image to be corrected; the reference control point pixel coordinates represent the control point pixel coordinates corresponding to the target control point pixel coordinates in one of the initial control point pairs in the remote sensing image to be corrected; The coordinate distance between the pixel coordinates of the reference control point and the pixel coordinates of the undetermined control point is calculated, and the initial control point pairs whose coordinate distances are greater than a second distance threshold in the initial control point pair groups after the preliminary screening are deleted, until the coordinate distances corresponding to the initial control point pairs in all the initial control point pair groups after the preliminary screening are less than or equal to the second distance threshold, thereby obtaining the target control point pairs.

7. A remote sensing image geometric precision correction system, characterized in that: include: A remote sensing data acquisition module, used to obtain a remote sensing image to be corrected and an initial reference image corresponding to the remote sensing image to be corrected, wherein the initial reference image is a reference image having the same geographical location information and the same modality as the remote sensing image to be corrected; A first alignment processing module is used to perform a time-phase alignment process on the initial reference image to obtain a reference image after the time-phase alignment process; A second alignment processing module is used to perform image scale alignment processing on the reference image after the time phase alignment processing to obtain a target reference image; A control point screening module, used to obtain initial control point pairs between the remote sensing image to be corrected and the target reference image, and screen all the initial control point pairs according to the pixel distances between the control points in all the initial control point pairs to obtain target control point pairs; The image correction module is used to perform geometric precision correction processing on the remote sensing image to be corrected based on the geometric correction model constructed by the target control point pair to obtain a corrected remote sensing image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the remote sensing image geometric precision correction method as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the remote sensing image geometric precision correction method as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the remote sensing image geometric precision correction method as claimed in any one of claims 1 to 6 is implemented.

Images