A method and device for detecting misregistration changes in remote sensing images

Abstract

The present invention discloses a method and device for detecting misaligned changes in remote sensing images. The method comprises the following steps: extracting feature points from a first remote sensing image and a second remote sensing image respectively to obtain a first feature point set and a second feature point set; matching corresponding feature points in the first feature point set and the second feature point set to obtain a plurality of matching point pairs; calculating a homography matrix based on the plurality of matching point pairs, and performing inverse transformation on the first remote sensing image and the second remote sensing image based on the homography matrix; performing change detection on the inversely transformed first remote sensing image and the second remote sensing image to obtain a changed image; the method and device for detecting misaligned changes in remote sensing images are described. The method comprises the following steps: screening feature points in a pre-remote sensing image and a post-remote sensing image, matching the feature points to obtain matching point pairs, confirming the spatial relationship of the images through the matching point pairs, and finally transforming the images according to the spatial relationship to realize the detection of the remote sensing images. The method can provide one-stop processing for the detection of misaligned remote sensing images, thereby improving the detection accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image change detection, and in particular relates to a remote sensing image misregistration change detection method and device. Background Art

[0002] Change detection in remote sensing images has long been a prominent area of ​​research in remote sensing, with widespread applications in urban planning, surface monitoring, and disaster assessment. Accurately and efficiently identifying changes in buildings and objects in disaster-stricken areas using remote sensing images is particularly valuable.

[0003] A drawback of existing remote sensing image change detection methods is that they typically target pre-registered data. However, in reality, multi-temporal data, both before and after, is acquired by sensors of different modalities and captured from different angles, making them difficult to consider as registered pairs. Furthermore, they are less robust to misregistration data, and some change detection methods may even fail. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and device for detecting unregistered changes in remote sensing images, which integrates change detection and registration of remote sensing images, thereby improving detection accuracy.

[0005] The present invention adopts the following technical solution: a method for detecting misregistration changes in remote sensing images, comprising the following steps:

[0006] Extracting feature points from the first remote sensing image and the second remote sensing image respectively to obtain a first feature point set and a second feature point set;

[0007] Matching corresponding feature points in the first feature point set and the second feature point set to obtain a plurality of matching point pairs;

[0008] Calculating a homography matrix according to a plurality of matching point pairs, and performing an inverse transformation on the first remote sensing image and the second remote sensing image based on the homography matrix;

[0009] Change detection is performed based on the inversely transformed first remote sensing image and the second remote sensing image to obtain a change image.

[0010] Furthermore, extracting feature points of the first remote sensing image / the second remote sensing image includes:

[0011] performing homography adaptation transformation on the first remote sensing image / the second remote sensing image;

[0012] Using Magic_Point detector to extract feature points in the first remote sensing image / the second remote sensing image of homography adaptation transformation;

[0013] Perform inverse transformation on feature points;

[0014] The feature points whose confidence after inverse transformation is greater than or equal to the confidence threshold are selected as the final feature points.

[0015] Furthermore, extracting feature points of the first remote sensing image / the second remote sensing image further includes:

[0016] The coordinates of the feature points of the first remote sensing image / the second remote sensing image and the pixel information of the blocks around the feature points are extracted.

[0017] Furthermore, after obtaining a number of matching point pairs, the following steps are also included:

[0018] Select the correct matching point pair from several matching point pairs.

[0019] Furthermore, selecting a correct matching point pair from among the plurality of matching point pairs includes:

[0020] The mean shift clustering method is used to select the correct matching point pair from several matching point pairs.

[0021] Furthermore, performing change detection based on the inversely transformed first remote sensing image and the second remote sensing image includes:

[0022] Extracting features of the transformed first remote sensing image and the transformed second remote sensing image at different scales respectively to obtain a first feature vector set of the transformed first remote sensing image and a second feature vector set of the transformed second remote sensing image;

[0023] The first feature vector set and the second feature vector set are respectively enhanced by a channel attention mechanism to obtain first channel enhanced features and second channel enhanced features; the first feature vector set and the second feature vector set are enhanced by a spatial attention mechanism to obtain first spatial enhanced features and second spatial enhanced features;

[0024] A difference map is generated according to the first channel enhancement feature, the second channel enhancement feature, the first space enhancement feature, and the second space enhancement feature, and a threshold segmentation is performed on the difference map to obtain a change map.

[0025] Furthermore, generating a difference map according to the first channel enhancement feature, the second channel enhancement feature, the first space enhancement feature, and the second space enhancement feature includes:

[0026]

[0027] Among them, Dist represents the difference map, represents the first channel enhancement feature, represents the second channel enhancement feature, represents the first spatial enhancement feature, Represents the second spatial enhancement feature.

[0028] Furthermore, the loss function used for change detection based on the inversely transformed first remote sensing image and the second remote sensing image is:

[0029] L(A,B)=L BCL (A,B)+λL Dice (A,B),

[0030] Among them, L(A,B) represents the loss function, L BCL (A, B) represents BCL loss, L Dice (A, B) represents the Dice loss, and λ is the adjustment factor of the deep supervision module.

[0031] Another technical solution of the present invention: a misregistration change detection device for remote sensing images, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.

[0032] Another technical solution of the present invention: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0033] The beneficial effects of the present invention are as follows: the present invention screens feature points in pre-remote sensing images and post-remote sensing images, matches the feature points to obtain matching point pairs, confirms the spatial relationship of the images through the matching point pairs, and finally transforms the images according to the spatial relationship to realize the detection of remote sensing images. It can provide one-stop processing for the detection of unregistered remote sensing images, thereby improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of a method for detecting misregistration changes in remote sensing images according to an embodiment of the present invention;

[0035] Figure 2 Schematic diagram of the process of processing unregistered data for change detection in an embodiment of the present invention;

[0036] Figure 3 Schematic diagram of self-supervisory construction of label data by the Superpoint network in an embodiment of the present invention;

[0037] Figure 4 Schematic diagram of a homography adaptation method in a Superpoint network according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of image change detection after affine transformation in an embodiment of the present invention;

[0039] Figure 6This is a comparison chart of the effects of different change detection methods after affine transformation in the embodiment of the present invention;

[0040] Figure 7 This is a comparison chart of the detection results of images in different data sets using different methods after projection transformation in the embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] Originally defined as "change detection," the goal was to leverage multi-temporal data from the same geographic area. This broad definition encompasses misaligned data. To simplify the change detection process and improve efficiency, most existing change detection methods target pre-registered datasets. While high change detection accuracy can be achieved within the framework of registered data, most methods fail to incorporate this crucial preprocessing step.

[0043] In reality, multi-temporal data before and after the event are acquired by sensors of different modalities, and cannot be considered as registered paired data due to different shooting angles. Currently, only a few methods include unregistered data in the task of change detection, and the results obtained need to be optimized. Some methods use traditional manual registration methods combined with classic change detection methods, which can effectively deal with rigid affine transformations, but are less effective when dealing with perspective transformations caused by differences in shooting angles. Other methods use generative models to obtain aligned registration data, but have difficulty handling unregistered data with large differences in lighting and angles, and are not suitable for slightly more complex situations. Another method designs an end-to-end joint estimation network to couple the registration and change detection tasks into one model, but this also brings problems such as model complexity and poor interpretability.

[0044] The present invention proposes a change detection method applicable to unregistered remote sensing data, which integrates change detection and registration of remote sensing images to improve the accuracy of change detection.

[0045] In the method of the present invention, the data format is unified before the registration operation; Superpoint features are constructed to effectively extract and utilize the rich textures and features of remote sensing images for registration; an adaptive feature matching strategy is designed to obtain reliable matching results; and finally, the matched data is input into a change detection network to obtain better change detection effects.

[0046] The present invention aims to propose a change detection method based on feature matching that is applicable to unregistered remote sensing data. Figure 1As shown in the figure, the whole process is divided into two parts: registration and change detection. Before change detection, the unregistered data needs to be registered. The unregistered data processing process for change detection designed by the present invention is as follows: Figure 2 shown.

[0047] First, the image registration stage is carried out. The overall process is as follows Figure 2 As shown in the figure, a pair of unregistered before-after images A and B are input, and features are extracted through the Superpoint network. Then, a more reliable matching point pair is obtained based on the Lightglue feature matching strategy and adaptive screening rules. final (A, B), and then calculate the homography matrix H based on the information of the matching point pair, and then perform inverse transformation A on the before and after images A and B at the physical coordinate level based on the homography matrix H cd =wrap(H,A),B cd = wrap(H,B), get the data sample for change detection {A cd , B cd , label cd}, where A cd represents the transformed prior image, B cd Represents the transformed image, label cd The actual change image represents the before-image and the after-image, which is only used for the evaluation of the subsequent network training and test results.

[0048] Finally, the aligned data is input into the change detection network for change detection to obtain the final change detection results.

[0049] Remote sensing images typically have rich textures and features, and feature point detection and matching methods can effectively extract and utilize these features for image registration. During the image registration stage, feature points are extracted from the first remote sensing image (i.e., the before-image) A and the second remote sensing image (i.e., the after-image) B, respectively, to obtain a first feature point set and a second feature point set. The first feature point set is the set of all feature points in the first remote sensing image, and the second feature point set is the set of all feature points in the second remote sensing image. These feature points can effectively describe key objects in remote sensing images and are an effective means of improving change detection accuracy.

[0050] In the embodiment of the present invention, the Superpoint network is used as a joint detection and description model. It takes the entire image input as the description, so that the two tasks of key point detection and description are integrated into one model for learning. Superpoint adopts a self-supervised approach, in which the label information (i.e., whether the pixel is a feature point label) comes from the self-labeling of its basic detector model. The specific generation steps are as follows: Figure 3As shown in Figure 2, Magic_point represents the basic detector and Homo adaptation represents the homography transformation.

[0051] In an embodiment of the present invention, extracting feature points of the first remote sensing image / the second remote sensing image includes: performing a homography adaptive transformation on the first remote sensing image / the second remote sensing image; extracting feature points in the homography adaptively transformed first remote sensing image / the second remote sensing image using a Magic_Point detector; performing an inverse transformation on the feature points; and selecting feature points whose confidence levels after the inverse transformation are greater than or equal to a confidence threshold as final feature points.

[0052] After obtaining the final feature points, these feature points can also be used to optimize the Magic_Point detector to make the accuracy of the Magic_Point detector higher and higher.

[0053] The specific process of homography transformation is as follows Figure 4 As shown in the figure, we first label points of interest from synthetic data containing significant features and train a basic detector Magic_Point. Then, we use the homography adaptation method Homo adaptation to introduce actual remote sensing scenes, randomly project the target data in the image to be tested, and feed it into the detector. Finally, we perform an inverse transformation on all data (including feature points) and take points with high confidence as points of interest to update and optimize the basic detector, thus extending the points of interest from synthetic data to real scenes.

[0054] The Superpoint network extracts possible points of interest and descriptors on the first remote sensing image and the second remote sensing image respectively. It can be applied to both homogeneous data and heterogeneous data. Its network structure consists of two parts: the point of interest extraction part and the feature descriptor extraction part. The two parts work together to extract Superpoint features. Superpoint features are composed of interest points Inter(x) and descriptors Desc(x), where x is the input image annotated with Magic_point. Superpoint features are expressed mathematically as shown in formula (1), that is, input image x and score score is obtained. sp 、Coordinate pos sp , descriptor desc sp Superpoint feature composed of three parts sp (x).

[0055]

[0056] Among them, the coordinates are used to describe the location information of the interest point, and the probability of the interest point represents the confidence of the interest point. The corresponding interest point can be selected as a feature point according to the confidence threshold. spThe pixel information of the image blocks surrounding the point of interest is described, such as the texture, shape, and color information of the surrounding image blocks. In other words, extracting the feature points of the first remote sensing image / the second remote sensing image further includes extracting the coordinates of the feature points of the first remote sensing image / the second remote sensing image and the pixel information of the image blocks surrounding the feature points.

[0057] After extracting feature points, the corresponding feature points in the first and second feature point sets need to be matched to obtain a number of matching point pairs. Because feature points effectively describe the key objects to be detected in the remote sensing image, these feature points represent the key objects in the remote sensing image, i.e., the objects to be detected. Therefore, before change detection, the objects to be detected in the first and second remote sensing images need to be mapped, and these matching point pairs represent this mapping relationship.

[0058] After obtaining matching point pairs, a large number of matching point pairs will exist in the first remote sensing image and the second remote sensing image. Among these matching point pairs, there will be some correct matching point pairs and some mismatched outliers (i.e., incorrectly matched matching point pairs). Therefore, it is necessary to screen the large number of matching point pairs to eliminate the mismatched outliers and retain the correct matching point pairs for subsequent steps. In other words, after obtaining a number of matching point pairs, the process also includes selecting the correct matching point pairs from the number of matching point pairs.

[0059] In this embodiment of the present invention, an adaptive feature matching strategy is constructed to obtain reliable matching results. Specifically, this solution combines the Lightglue matcher (a deep matcher based on a deep neural network used to match local features in an image) with an adaptive screening rule to retain as many matching point pairs as possible with the highest matching confidence to predict the changed image.

[0060] This solution uses the Lightglue matching strategy, and the input contains the coordinates pos sp and descriptor desc sp , output coordinate matching description match pos and confidence score matching description match conf After obtaining the preliminary key point matching pairs, this scheme plans to further refine the matching pairs and filter out some outlier matching points.

[0061] Confidence score thresholds are typically used to remove incorrect matching point pairs from a set of matching point pairs. However, in the present invention, an adaptive feature removal rule is employed to balance detection efficiency and accuracy. Specifically, selecting the correct matching point pair from a set of matching point pairs includes selecting the correct matching point pair from a set of matching point pairs using a mean shift clustering method.

[0062] Assume that after Lightglue matching, the correct matching point pairs account for the largest number of classes in all data, and the distance deviation between them in physical position is small, and the difference with the mismatched outliers is large.

[0063] In a specific embodiment, the coordinates of the matching point pair are matched pos (A) match pos (B) is used as input, and the physical coordinate distance between the two is used as the deviation metric dist = || match pos (A, B)||, use clustering method to find the largest cluster and record it as a more reliable matching point pair match final (A, B).

[0064] Next, the homography matrix is ​​calculated based on a number of matching point pairs, and the first remote sensing image and the second remote sensing image are inversely transformed based on the homography matrix. final After (A, B), calculate the homography matrix H, and perform inverse transformation on the first remote sensing image and the second remote sensing image at the physical coordinate level to obtain A cd =wrap(H,A) and B cd =wrap(H,B), thus obtaining the data sample (A cd , B cd , label cd That is, H is calculated based on the set of all feature points on one image and the set of all feature points on another image.

[0065] After the above processing, the registration of the first remote sensing image and the second remote sensing image is completed. Then, change detection needs to be performed based on the inversely transformed first remote sensing image and the second remote sensing image to obtain a change image.

[0066] In the change detection stage, the embodiment of the present invention adopts the DSAM network, namely DSAMNet. In this step, the remote sensing image (A cd , B cd , label cd}(In the actual detection stage, only A cd and B cd , does not include label cd ), A cd and B cd Simultaneously feed the shared weighted twin network to extract features in, A represents the scale of the kth layer cd Features, is the k-th scale B cd Features, Feat A and FeatB That is, the features of the transformed first remote sensing image and the transformed second remote sensing image at different scales are extracted respectively to obtain a first feature vector set of the transformed first remote sensing image and a second feature vector set of the transformed second remote sensing image.

[0067] Then, the first feature vector set and the second feature vector set are respectively enhanced through the channel attention mechanism to obtain the first channel enhanced features and the second channel enhanced features; the first feature vector set and the second feature vector set are enhanced through the spatial attention mechanism to obtain the first space enhanced features and the second space enhanced features.

[0068] Finally, a difference map is generated according to the first channel enhancement feature, the second channel enhancement feature, the first space enhancement feature and the second space enhancement feature, and the difference map is subjected to threshold segmentation to obtain a change map.

[0069] As a specific implementation method, the first feature vector set and the second feature vector set are respectively sent to the measurement module containing channel attention and spatial attention to obtain the feature after attention enhancement. and in, Representative Feat A The first channel enhanced features after channel attention enhancement, Representative Feat B The second channel enhanced features after channel attention enhancement, Representative Feat A The first spatial enhancement feature after spatial attention enhancement, Representative Feat B The second spatial enhancement feature after spatial attention enhancement is composed of and Calculate the preliminary difference map Finally, the network performs threshold segmentation on the difference map to obtain the final change map.

[0070] During the network training phase, the BCL loss is calculated as follows:

[0071]

[0072] Among them, y i,j Indicates label cd The value at the midpoint (i, j), d i,jIndicates the value at point (i, j) in the difference graph, M is the size of the difference graph, and m is the distance threshold, which indicates the tolerance for negative samples. In the BCL loss, an m boundary parameter (i.e., distance threshold) is set initially, which determines the maximum distance between different categories. If d i,j Exceeding (i.e. greater than or equal to) the boundary threshold makes the contribution of this category to the loss 0. If d i,j If the value is smaller than the boundary threshold, it will be taken into consideration.

[0073] Then, the features of each scale are Send it to the deep supervision module and calculate the Dice loss:

[0074]

[0075] Among them, N represents the amount of training data, a is the length of the vector after flattening the feature vector, Represents the value at point (i, j) in the change graph, b represents the ordinal number of the training data, and c represents the number of digits after the feature vector is flattened.

[0076] Finally, the loss function of the DSAM network is obtained by fusing the BCL loss and the Dice loss, namely:

[0077] L(A,B)=L BCL (A,B)+λL Dice (A,B).

[0078] Among them, λ is the adjustment factor of the deep supervision module.

[0079] In summary, the network combines channel attention, spatial attention, and deep supervision modules at various scales, and can be used for change detection of both homogeneous and heterogeneous data.

[0080] The present invention constructs Superpoint feature description, uses Lightglue feature matching strategy, and combines adaptive feature screening rules to obtain data samples for change detection, and inputs them into the DSAMNet change detection network for change detection. It provides a better change detection method for unregistered remote sensing images, which is suitable for change detection technology methods of unregistered remote sensing data.

[0081] The Superpoint feature description, Lightglue feature matching strategy and adaptive feature screening rules are combined for data alignment, and a change detection network that combines channel attention, spatial attention and deep supervision modules at various scales is used for change detection, which greatly improves the accuracy of the change detection network.

[0082] The method of the present invention uses Superpoint features, Lightglue matching strategy and adaptive screening strategy to jointly construct the registration module, and selects DSAMNet as the change detection module. The purpose is to design a fast and concise change detection method with explicit intermediate results that are easy to explain for unregistered data.

[0083] Superpoint features serve as a joint detection and description model, using the entire image input as a description. This allows keypoint detection and description tasks to be integrated into a single learning model, enabling rapid response to scenes with significant changes in lighting and perspective. Lightglue, combined with pruned graph networks for matching, effectively handles high-noise and weak-texture areas in remote sensing images. The combination of these two approaches enables rapid and high-accuracy registration.

[0084] Compared to traditional methods like SIFT and ORB, the registration module used in this method is adaptable to a wide range of scenarios and is highly robust to non-rigid transformations such as large viewpoint changes, achieving higher registration accuracy at a lower computational cost. Compared to other deep learning registration methods like DISK, LoFTR, and XFeat, Superpoint+Lightglue boasts a relatively simple and lightweight design and mature hardware deployment, making it more flexible and adaptable in processing large-scale remote sensing imagery.

[0085] Existing public change detection datasets are all pre-registered, excluding unregistered data and formal manual annotations specifically for unregistered data. However, in practice, pre- and post-processing data often consist of unregistered images from sensors of different modalities and captured at different angles. These images exhibit not only rigid geometric differences but also differences in resolution and camera angle. Therefore, a unified data format is required before registration can be implemented. Furthermore, the amount of heterogeneous remote sensing data is relatively small, with only one pair of data per scene, making it difficult to support the requirements of registration and change detection module training.

[0086] Therefore, this method is based on accurately labeled change detection datasets (LEVIR, Shuguang, Sardinia), and artificially constructs a synthetic dataset including affine, projective transformation and scale change. The construction of the unregistered dataset is divided into two steps: unifying the data format and applying random affine perspective transformation. First, the data format needs to be unified. Before registering and detecting changes in remote sensing data, it is necessary to ensure that the data format is unified, that is, at the same size and resolution, and showing the same geographical area, so that the selected features are reliable. After regularizing the data to a unified format, artificial transformations including random projective affine transformation, scale and illumination are applied to simulate the complex situation where the actual shooting angle, resolution and illumination have large changes in external conditions.

[0087] Kappa is used to measure the difference between the experimental result graph (i.e. the above-mentioned change graph) and the reference graph (i.e. label cd )The coefficient of consistency. The larger the Kappa coefficient, the more accurate the change detection result.

[0088] The red area in the experimental result graph indicates the area where changes have occurred in the reference image but no changes have been detected in the experimental result graph, that is, the missed detection area; the green area in the experimental result graph indicates the area where changes have not occurred in the reference image but no changes have been detected in the experimental result graph, that is, the false detection area.

[0089] In one embodiment, the LEVIR dataset is used for change detection, such as Figure 5 As shown in the figure, the first column is three different images Image_A of the data set, the second column is three homogeneous images Image_B corresponding to image Image_A with different affine transformations added, the third column is image Wrap_A after image Image_A is aligned with the method of the present invention, the fourth column is the change true image Label obtained by performing the same affine transformation as image Image_B on the original change true image, the fourth column is the change map Change map after change detection of images Image_B and Wrap_A, and the fifth column is a schematic diagram of the comparison results of the change true image Label and the change map Change map.

[0090] Specifically, the comparison results of the three pictures are shown in Table 1 below:

[0091] Table 1

[0092]

[0093] According to Table 1, the method of the present invention can better realize the change detection of the before-and-after images.

[0094] like Figure 6 As shown in FIG, this is a comparison chart of the detection results of different methods in this verification example. Figure 6 (a) is an original image Image_A in the ShuguangVillage dataset. Figure 6 (b) is the heterogeneous image Image_B with affine transformation applied, Figure 6 (c) is the image Sp_register after the original image Image_A is registered. Figure 6 (d) is the real image label of the change, Figure 6 (e) is a change detection diagram obtained by using the detection method of the present invention, Figure 6 (f) is the change detection map obtained by using the Ge_registed detection method. Figure 6 (g) is the change detection map obtained by using SIFT+IRG-McS detection method. Figure 6 (h) is the change detection map obtained by using the ORB+IRG-McS detection method. Figure 6 (i) is the change detection map obtained by using SIFT+SRF detection method, Figure 6 (j) is the change detection diagram obtained using the ORB+SRF detection method; the Kappa values ​​of each detection method are detailed in Table 2.

[0095] Table 2

[0096]

[0097] As shown in Table 2, the Kappa value of the change detection graph obtained by the detection method of the present invention is the highest, which is much higher than that of the traditional detection method. That is, the detection accuracy of the method of the present invention is higher than that of other traditional detection methods.

[0098] like Figure 7 As shown, the comparison of the image detection accuracy of the method of the present invention and the LoFTR+DSAM method for the Shuguang Village and Sardinia datasets.

[0099] Figure 7 (a) is an original image Image_A in the Shuguang Village dataset. Figure 7 (h) is an original image Image_A in the Sardinia dataset; Figure 7 (b) and Figure 7 (i) is the heterogeneous image Wrap_B which is a projection transformation applied to the original image; Figure 7 (c) and Figure 7 (j) is the image Sp_register after the original image Image_A is registered using the method of the present invention; Figure 7 (d) and Figure 7 (k) is the image LoFTR after the original image Image_A is registered using the LoFTR method; Figure 7 (e) for and Figure 7 (l) is the two change detection true image labels; Figure 7 (f) and Figure 7 (m) is a change detection graph obtained by the detection method of the present invention; Figure 7 (g) and Figure 7 (n) is the change detection map obtained using the LoFTR+DSAM detection method.

[0100] The detection results of different detection methods for images of different data sets are shown in Table 3 below.

[0101] Table 3

[0102]

[0103] According to Table 3, the Kappa value of the change detection graph obtained by the method of the present invention is higher than the Kappa value of the change detection graph obtained by the LoFTR+DSAM detection method, which proves that the detection accuracy of the method of the present invention is better than that of the LoFTR+DSAM detection method.

[0104] The present invention also discloses a remote sensing image misregistration change detection device, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. The processor implements the above method when executing the computer program.

[0105] The present invention further discloses an embodiment that provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments are implemented.

[0106] The present invention also provides a computer program product, which, when executed on a data storage device, enables the data storage device to implement the steps in the above-mentioned method embodiments.

[0107] If the integrated unit module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying the computer program code to a storage device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0108] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

Claims

1. A method for detecting misregistration changes in remote sensing images, characterized in that: The following steps are involved: Extracting feature points from the first remote sensing image and the second remote sensing image respectively to obtain a first feature point set and a second feature point set; Matching corresponding feature points in the first feature point set and the second feature point set to obtain a plurality of matching point pairs; Calculating a homography matrix according to the plurality of matching point pairs, and performing an inverse transformation on the first remote sensing image and the second remote sensing image based on the homography matrix; Performing change detection based on the inversely transformed first remote sensing image and the second remote sensing image to obtain a change image; Performing change detection based on the inversely transformed first remote sensing image and the second remote sensing image includes: Extracting features of the transformed first remote sensing image and the transformed second remote sensing image at different scales respectively to obtain a first feature vector set of the transformed first remote sensing image and a second feature vector set of the transformed second remote sensing image; Performing feature enhancement on the first feature vector set and the second feature vector set respectively through a channel attention mechanism to obtain first channel-enhanced features and second channel-enhanced features; performing feature enhancement on the first feature vector set and the second feature vector set through a spatial attention mechanism to obtain first spatial-enhanced features and second spatial-enhanced features; A difference map is generated according to the first channel enhancement feature, the second channel enhancement feature, the first space enhancement feature, and the second space enhancement feature, and a threshold segmentation is performed on the difference map to obtain a change map.

2. The method for detecting misregistration changes in remote sensing images according to claim 1, wherein: Extracting feature points of the first remote sensing image / the second remote sensing image includes: performing homography adaptation transformation on the first remote sensing image / the second remote sensing image; Using Magic_Point detector to extract feature points in the first remote sensing image / the second remote sensing image of homography adaptation transformation; Performing an inverse transformation on the feature points; The feature point whose confidence level after inverse transformation is greater than or equal to the confidence level threshold is selected as the final feature point.

3. The method for detecting misregistration changes in remote sensing images according to claim 2, wherein: Extracting feature points of the first remote sensing image / the second remote sensing image further includes: The coordinates of the feature points of the first remote sensing image / the second remote sensing image and the pixel information of the blocks around the feature points are extracted.

4. The method for detecting misregistration changes in remote sensing images according to claim 1, wherein: After obtaining several matching point pairs, it also includes: Select the correct matching point pair from several matching point pairs.

5. The method for detecting misregistration changes in remote sensing images according to claim 4, wherein: Selecting the correct matching point pair from among several matching point pairs includes: The mean shift clustering method is used to select the correct matching point pair from several matching point pairs.

6. The method for detecting misregistration changes in remote sensing images according to claim 5, wherein: Generating a difference map according to the first channel enhancement feature, the second channel enhancement feature, the first space enhancement feature, and the second space enhancement feature includes: ， Among them, Dist represents the difference map, represents the first channel enhancement feature, represents the second channel enhancement feature, represents the first spatial enhancement feature, Represents the second spatial enhancement feature.

7. The method for detecting misregistration changes in remote sensing images according to claim 5, wherein: The loss function used for change detection based on the inverse transformed first remote sensing image and the second remote sensing image is: ， in, represents the loss function, represents BCL loss, represents the Dice loss, λ is the regulation factor of the deep supervision module.

8. A device for detecting misregistration changes in remote sensing images, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images