A method and system for heterologous remote sensing image change detection

Abstract

The present invention relates to the technical field of heterogeneous remote sensing image change detection, and in particular to a heterogeneous remote sensing image change detection method and system. The method comprises the following steps: obtaining a target heterogeneous remote sensing image; performing stationary wavelet decomposition on the target heterogeneous remote sensing image, and processing the decomposed target heterogeneous remote sensing image using an image translation network to obtain a translated image; respectively obtaining a wavelet domain difference image and a spatial domain difference image between the translated image and the target heterogeneous remote sensing image; synthesizing a fused difference image between the translated image and the target heterogeneous remote sensing image based on the wavelet domain difference image and the spatial domain difference image; and dividing pixel points in the fused difference image using threshold segmentation to obtain a change detection result of the target heterogeneous remote sensing image. The present invention can generate a high-quality difference image, thereby helping to accurately identify changed areas in the image and improving the accuracy of change detection.

Description

Technical Field

[0001] The present invention relates to the technical field of heterogeneous remote sensing image change detection, and in particular to a heterogeneous remote sensing image change detection method and system. Background Art

[0002] With the rapid development of remote sensing technology, monitoring ground dynamics using data from diverse observation platforms has become an important approach for studying the evolution of the Earth's environment. Change detection, a process for identifying changes by comparing a set of remote sensing images acquired over the same geographic area but at different times, has been widely applied in many fields, such as deforestation, urban planning, and disaster assessment. Traditional change detection methods primarily target homologous images—images acquired from the same sensor, such as very high-resolution (VHR) optical imagery, synthetic aperture radar (SAR), and hyperspectral imagery. However, due to limitations in imaging conditions and satellite revisit cycles, homologous images cannot be obtained in a timely manner. Consequently, the use of remote sensing images from different sensors, known as heterogeneous images, for change detection has attracted significant attention.

[0003] Since heterogeneous images usually have different statistical characteristics, it is difficult to directly compare them to generate difference images like homologous images. To address the above difficulties, an intuitive solution is to convert heterogeneous images into homologous images with similar statistical characteristics, so that they are comparable and the changes are highlighted. In recent years, deep learning technology has developed rapidly and has been successfully applied to the field of remote sensing image change detection. Currently, most deep learning-based methods are based on adversarial learning or style transfer methods to eliminate modal differences between heterogeneous images. For example, there are methods in the existing technology that use affinity matrix difference, namely X-Net and Adversarial Cyclic Encoder Network (ACENet), which aim to combine cycle consistency and adversarial training to translate images. There is also an unsupervised change detection method based on autoencoders (CAAE) to force the spatial alignment of heterogeneous image features and reduce the impact of pixel changes on image translation; there is also a method based on dual-image translation and dual-contrast learning in the existing technology for accurately identifying surface changes; an end-to-end multi-domain constrained translation network is designed, which uses an enhanced generative adversarial network (GAN) to ensure the realism of the translated image and integrates contrastive learning to retain the content information of the source domain image; although the above methods have achieved good change detection results, they either need to be modified before guiding the translation process, or require multiple rounds of GAN training and parameter optimization of style transfer learning, resulting in the inability to capture subtle changes in complex scenes. Summary of the Invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a heterogeneous remote sensing image change detection method, which obtains a target heterogeneous remote sensing image; performs stationary wavelet decomposition on the target heterogeneous remote sensing image, and uses an image translation network to process the decomposed target heterogeneous remote sensing image to obtain a translated image; respectively obtains a wavelet domain difference image and a spatial domain difference image between the translated image and the target heterogeneous remote sensing image; synthesizes a fused difference image between the translated image and the target heterogeneous remote sensing image based on the wavelet domain difference image and the spatial domain difference image; and uses threshold segmentation to divide pixels in the fused difference image to obtain a change detection result of the target heterogeneous remote sensing image. The present invention can generate a high-quality difference image, thereby helping to accurately identify the changed area in the image and improving the accuracy of change detection.

[0005] The present invention adopts the following technical solution, a heterogeneous remote sensing image change detection method, comprising:

[0006] Acquire a target heterogeneous remote sensing image; the target heterogeneous remote sensing image includes a first X-domain remote sensing image and a first Y-domain remote sensing image;

[0007] Perform stationary wavelet decomposition on the target heterogeneous remote sensing image, and use the image translation network to process the decomposed target heterogeneous remote sensing image to obtain a translated image;

[0008] The translated images include: a second Y-domain translated image corresponding to the first X-domain remote sensing image; a second X-domain translated image corresponding to the first Y-domain remote sensing image;

[0009] respectively obtaining a wavelet domain difference image and a spatial domain difference image between the translated image and the target heterogeneous remote sensing image;

[0010] synthesizing a fused difference image between the translated image and the target heterogeneous remote sensing image based on the wavelet domain difference image and the spatial domain difference image;

[0011] The threshold segmentation algorithm is used to divide the pixels in the fused difference image to obtain the change detection results of the target heterogeneous remote sensing image.

[0012] Furthermore, the image translation network includes: a first generator and a second generator;

[0013] The first generator is used to convert the first X-domain remote sensing image into a second Y-domain translation image, which includes a first encoder and a first decoder; the first encoder and the first decoder adopt a skip connection;

[0014] The second generator is used to convert the first Y-domain remote sensing image into a second X-domain translated image, which includes a second encoder and a second decoder; the second encoder and the second decoder adopt a jump connection.

[0015] Furthermore, the image translation network also includes:

[0016] Input the second Y-domain translated image into the second generator to obtain a third X-domain reconstructed image;

[0017] The second X-domain translated image is input into the first generator to obtain a third Y-domain reconstructed image.

[0018] Furthermore, the image translation network adopts a cross-domain triple loss function, specifically:

[0019] The cross-domain triple loss function includes style loss, content loss and reconstruction loss, which is expressed as:

[0020] ;

[0021] in, represents the cross-domain triple loss function, is the style loss, For content loss, To rebuild losses;

[0022] The style loss is expressed as:

[0023] ;

[0024] in, represents the style loss, represents the high frequency component in the first X-domain remote sensing image, represents the high-frequency component in the second X-domain translated image, represents the high-frequency component in the first Y-domain remote sensing image, represents the high-frequency component in the second Y-domain translated image, represents the L1 norm;

[0025] The content loss is expressed as:

[0026] ;

[0027] in, Indicates content loss, Represents the pixel value of the unchanged area in the target heterogeneous remote sensing image, represents the low-frequency component in the first X-domain remote sensing image, represents the low-frequency component in the second X-domain translated image, represents the low-frequency component in the first Y-domain remote sensing image, represents the low-frequency component in the second Y-domain translated image, represents the L1 norm;

[0028] The reconstruction loss is expressed as:

[0029] ;

[0030] in, represents the reconstruction loss, represents the characteristic component in the first X-domain remote sensing image, represents the characteristic component in the third X domain reconstructed image, represents the characteristic component in the first Y-domain remote sensing image, represents the characteristic component in the third Y domain reconstructed image, represents the L1 norm.

[0031] Furthermore, the wavelet domain difference image is obtained as follows:

[0032] Acquire a first low-frequency component in the first X-domain remote sensing image and a second low-frequency component in the second X-domain translated image;

[0033] acquiring a first X-domain difference image according to a Euclidean distance between the first low-frequency component and the second low-frequency component;

[0034] Acquire a third low-frequency component in the first Y-domain remote sensing image and a fourth low-frequency component in the second Y-domain translated image;

[0035] acquiring a first Y-domain difference image according to a Euclidean distance between the third low-frequency component and the fourth low-frequency component;

[0036] The first X-domain difference image and the first Y-domain difference image are fused by using an adaptive weight fusion method to obtain a wavelet domain difference image.

[0037] Furthermore, the spatial domain difference image is obtained as follows:

[0038] Inputting the first X-domain remote sensing image and the second X-domain translated image into a first encoder, obtaining a first feature vector corresponding to the first X-domain remote sensing image and a second feature vector corresponding to the second X-domain translated image;

[0039] acquiring a second X-domain difference image according to the Euclidean distance between the first eigenvector and the second eigenvector;

[0040] Inputting the first Y-domain remote sensing image and the second Y-domain translated image into a second encoder, obtaining a third eigenvector corresponding to the first Y-domain remote sensing image and a fourth eigenvector corresponding to the second Y-domain translated image;

[0041] obtaining a second Y-domain difference image according to the Euclidean distance between the third eigenvector and the fourth eigenvector;

[0042] The second X-domain difference image and the second Y-domain difference image are fused using an adaptive weight fusion method to obtain a spatial domain difference image.

[0043] Furthermore, the synthesis method of the fused difference image is:

[0044] The information entropy weighted fusion method is used to fuse the wavelet domain difference image and the spatial domain difference image to obtain a fused difference image.

[0045] Based on the above-mentioned heterogeneous remote sensing image change detection method, the present invention also proposes a heterogeneous remote sensing image change detection system, which includes:

[0046] Image acquisition module, used to collect target heterogeneous remote sensing images;

[0047] The image translation module is used to perform stationary wavelet decomposition on the target heterogeneous remote sensing image and process the decomposed target heterogeneous remote sensing image using an image translation network to obtain a translated image;

[0048] A difference image extraction module is used to obtain a wavelet domain difference image and a spatial domain difference image between the translation image and the target heterogeneous remote sensing image respectively;

[0049] A difference image fusion module is used to synthesize a fused difference image between the translation image and the target heterogeneous remote sensing image based on the wavelet domain difference image and the spatial domain difference image;

[0050] The image detection module is used to divide the pixels in the fused difference image using a threshold segmentation algorithm to obtain the change detection results of the target heterogeneous remote sensing image.

[0051] The beneficial effects of the present invention are as follows: the image translation network proposed in the present invention only includes two generators with the same structure and does not have any discriminators. Compared with other network structures, it is simpler and has fewer parameters. The feature extraction capability of the network is further enhanced by adding jump connections, so as to help generate clearer difference images in the subsequent generation. At the same time, the image translation network adopts a simple and effective cross-domain triple loss function for optimization. The style loss aims to reduce the style difference between the translated image and the target image, the content loss aims to amplify the content difference in the change area between the translated image and the target image, and the reconstruction loss is used to prevent the loss of image information, so as to better highlight the changes between heterogeneous images. For the generated difference image, the present invention adopts an adaptive weight fusion strategy and an information entropy weighted fusion strategy to fuse the forward and backward difference images in the wavelet domain and the spatial domain, which can fully extract the changes in the translation process, thereby generating a higher quality difference map, which further helps to improve the performance of change detection in image change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a flow chart of a method for detecting changes in heterogeneous remote sensing images according to an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the structure of an encoder-decoder jump connection method according to an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of a translation process of an image translation network according to an embodiment of the present invention;

[0056] Figure 4 Schematic diagram comparing the detection results of different detection methods according to an embodiment of the present invention; wherein, Figure 4 (a) is the image before the event, Figure 4 (b) is the image after the event, Figure 4 (c) is the actual ground change map, Figure 4 (d) is a schematic diagram of the change detection results using the X-Net network. Figure 4 (e) is a schematic diagram of the change detection results using the ACE-Net network. Figure 4 (f) is a schematic diagram of the change detection results using the GIR-MRF method. Figure 4 (g) is a schematic diagram of the detection results using the CAAE method. Figure 4 (h) is a schematic diagram of the detection results using the AGSCC method. Figure 4 (i) is a schematic diagram of the detection results using the SDIR method, Figure 4 (j) is a schematic diagram of the detection results of the method proposed in the present invention;

[0057] Figure 5 The figure is a schematic structural diagram of a heterogeneous remote sensing image change detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0059] A flow chart of a heterogeneous remote sensing image change detection method according to an embodiment of the present invention is shown in FIG. Figure 1 As shown, including:

[0060] Acquire target heterogeneous remote sensing images;

[0061] Heterogeneous remote sensing images refer to remote sensing images with diverse data characteristics acquired through different sensors, different platforms, or at different times. These data differences are reflected in aspects such as spectral resolution, spatial resolution, temporal resolution, or imaging principles (e.g., optical, radar, infrared, etc.). In embodiments of the present invention, target heterogeneous remote sensing images include a first X-domain remote sensing image and a first Y-domain remote sensing image. Correspondingly, in embodiments of the present invention, the first X-domain remote sensing image can be considered the image before the change, and the first Y-domain remote sensing image can be considered the image after the change. The image before the change and the image after the change are in different image domains, and different image domains represent remote sensing images acquired through different methods. In embodiments of the present invention, remote sensing images can be acquired through any of a variety of methods, such as multi-sensor and satellite collaboration, drone and satellite image fusion, or a combination of visible light and thermal infrared images. When training an image translation network, the present invention accesses an existing remote sensing image library for downloading, thereby acquiring a large number of remote sensing images to construct a remote sensing image dataset, thereby saving training time.

[0062] Perform stationary wavelet decomposition on the target heterogeneous remote sensing image, and use the image translation network to process the decomposed target heterogeneous remote sensing image to obtain a translated image;

[0063] In an embodiment of the present invention, a stationary wavelet decomposition method avoids downsampling operations, making the decomposition process translation-invariant, and can better preserve the local features and detail information of the image. By performing stationary wavelet decomposition on the target heterogeneous remote sensing image, the target heterogeneous remote sensing image is decomposed into a low-frequency component and three high-frequency detail components, wherein the low-frequency component represents the approximate part of the image, and the three high-frequency components represent the horizontal, vertical and diagonal parts of the image, respectively. Therefore, the low-frequency component can represent the content information of the target heterogeneous remote sensing image, and the high-frequency component can represent the style information of the image.

[0064] In an embodiment of the present invention, an image translation network includes a first generator and a second generator, wherein the first generator and the second generator form a pseudo Siamese network, and the first generator and the second generator share the same structure;

[0065] The first generator includes a first encoder and a first decoder, which are used to convert the first X-domain remote sensing image into a second Y-domain translated image. The second generator includes a second encoder and a second decoder, which are used to convert the first Y-domain remote sensing image into a second X-domain translated image. Skip connections are used between the first encoder and the first decoder, as well as between the second encoder and the second decoder, to further enhance the feature representation capabilities of the generator.

[0066] The skip connection method is also called residual connection, such as Figure 2 As shown, in a first encoder-first decoder structure of an embodiment of the present invention, the first encoder is responsible for converting the input sequence into an intermediate representation (feature vector), and the first decoder generates an output sequence based on this intermediate representation, that is, the skip connection between the first encoder and the first decoder is to directly pass part of the output information of the first encoder to certain layers of the first decoder without passing through all the intermediate layers. At the same time, the convolution type and parameters of each layer in the first encoder-first decoder structure in an embodiment of the present invention are shown in Table 1, and the second encoder and the second decoder are similar, and the convolution type and parameters are the same as those in the first encoder-first decoder structure.

[0067] Table 1 Schematic diagram of convolution type and parameters of each layer in the encoder-decoder structure

[0068] ;

[0069] In a specific embodiment of the present invention:

[0070] The translated image includes: a second Y-domain translated image corresponding to the first X-domain remote sensing image; a second X-domain translated image corresponding to the first Y-domain remote sensing image; the translated image and the target heterogeneous remote sensing image have the same style but different content. In order to better optimize the image translation network, the present invention adopts a cross-domain triple loss function to reduce the style difference between the translated image and the target image, and amplify the content difference of the changed area between the translated image and the target image, thereby better highlighting the changes between the images. The translation process is as follows Figure 3 As shown; In the embodiment of the present invention, the cross-domain triple loss function is established based on style loss, content loss and reconstruction loss, and is expressed as:

[0071] ;

[0072] in, represents the cross-domain triple loss function, is the style loss, For content loss, To rebuild losses;

[0073] The style loss is expressed as:

[0074] ;

[0075] in, represents the style loss, represents the high frequency component in the first X-domain remote sensing image, represents the high-frequency component in the second X-domain translated image, represents the high-frequency component in the first Y-domain remote sensing image, represents the high-frequency component in the second Y-domain translated image, represents the L1 norm;

[0076] The content loss is expressed as:

[0077] ;

[0078] in, Indicates content loss, Represents the pixel value of the unchanged area in the target heterogeneous remote sensing image, represents the low-frequency component in the first X-domain remote sensing image, represents the low-frequency component in the second X-domain translated image, represents the low-frequency component in the first Y-domain remote sensing image, represents the low-frequency component in the second Y-domain translated image, represents the L1 norm;

[0079] The reconstruction loss is expressed as:

[0080] ;

[0081] in, represents the reconstruction loss, represents the characteristic component in the first X-domain remote sensing image, represents the characteristic component in the third X domain reconstructed image, represents the characteristic component in the first Y-domain remote sensing image, represents the characteristic component in the third Y domain reconstructed image, represents the L1 norm.

[0082] In a specific embodiment of the present invention, when training the image translation network, in addition to using the downloaded remote sensing image dataset, it is also necessary to obtain the label mask matrix corresponding to the remote sensing image dataset. That is, each remote sensing image has a corresponding label mask matrix, and each element in the label mask matrix corresponds to a pixel in the remote sensing image. Since the label mask matrix is randomly initialized, the invariant region in the remote sensing image dataset obtained based on the label mask matrix is not completely correct. Therefore, after completing the training of the image translation network, the label mask matrix can be updated by translating the image to obtain better change detection results. On this basis, the pixel value of the invariant region in the target heterogeneous remote sensing image can be obtained by the following formula:

[0083] ;

[0084] in, is the pixel value of the unchanged area in the target heterogeneous remote sensing image, is the label mask matrix; when the pixel values of the invariant area in the target heterogeneous remote sensing image are obtained After that, the image translation can be performed, and then the difference image can be obtained to generate the corresponding binary change map. As the number of iterative updates increases, the accuracy of the obtained binary change map will gradually improve.

[0085] respectively obtaining a wavelet domain difference image and a spatial domain difference image between the translated image and the target heterogeneous remote sensing image;

[0086] In an embodiment of the present invention, a wavelet domain difference image between a target heterogeneous remote sensing image and a translated image is obtained by: obtaining a first low-frequency component from a first X-domain remote sensing image and a second low-frequency component from a second X-domain translated image; obtaining a first X-domain difference image based on a Euclidean distance between the first low-frequency component and the second low-frequency component; obtaining a third low-frequency component from a first Y-domain remote sensing image and a fourth low-frequency component from the second Y-domain translated image; obtaining a first Y-domain difference image based on a Euclidean distance between the third low-frequency component and the fourth low-frequency component; and fusing the first X-domain difference image and the first Y-domain difference image using an adaptive weight fusion method to obtain a wavelet domain difference image between the target heterogeneous remote sensing image and the translated image.

[0087] The spatial domain difference image between the translated image and the target heterogeneous remote sensing image is obtained by: inputting the first X-domain remote sensing image and the second X-domain translated image into a first encoder to obtain a first eigenvector corresponding to the first X-domain remote sensing image and a second eigenvector corresponding to the second X-domain translated image; obtaining a second X-domain difference image based on the Euclidean distance between the first eigenvector and the second eigenvector; inputting the first Y-domain remote sensing image and the second Y-domain translated image into a second encoder to obtain a third eigenvector corresponding to the first Y-domain remote sensing image and a fourth eigenvector corresponding to the second Y-domain translated image; obtaining a second Y-domain difference image based on the Euclidean distance between the third eigenvector and the fourth eigenvector; and fusing the second X-domain difference image and the second Y-domain difference image using an adaptive weight fusion method to obtain a spatial domain difference image.

[0088] In a specific embodiment of the present invention:

[0089] The adaptive weight fusion method is used to fuse the first X domain difference image and the first Y domain difference image, which can be expressed as:

[0090] ;

[0091] in, represents the difference image in wavelet domain, represents the first X-domain difference image, represents the first Y-domain difference image, represents the standard deviation of the first X-domain difference image, represents the standard deviation of the first Y-domain difference image.

[0092] In a specific embodiment of the present invention:

[0093] The adaptive weight fusion method is used to fuse the second X-domain difference image and the second Y-domain difference image to obtain a spatial domain difference image, which can be expressed as:

[0094] ;

[0095] in, represents the spatial domain difference image, represents the second X-domain difference image, represents the second Y-domain difference image, represents the standard deviation of the second X-domain difference image, represents the standard deviation of the second Y-domain difference image.

[0096] synthesizing a fused difference image between the translated image and the target heterogeneous remote sensing image based on the wavelet domain difference image and the spatial domain difference image;

[0097] In the embodiment of the present invention, the wavelet domain difference image and the spatial domain difference image are synthesized by using an information entropy weighting method to obtain a fused difference image between the translated image and the target heterogeneous remote sensing image. The synthesis process can be expressed as:

[0098] ;

[0099] in, represents the fused difference image between the translated image and the target heterogeneous remote sensing image, represents the difference image in wavelet domain, represents the information entropy of the difference image in the wavelet domain, represents the spatial domain difference image, Represents the information entropy of the spatial domain difference image.

[0100] In a specific embodiment of the present invention:

[0101] Wavelet domain information contains the multi-scale frequency characteristics of the image, and can decompose the image at multiple scales from the frequency domain perspective, effectively capture the changing details of the image at different frequencies, extract the details and texture features of the image, and is particularly good at capturing the local change information of the image; spatial domain information has the original spatial structure and texture information of the image, intuitively reflects the pixel grayscale distribution and spatial structural relationship of the image, and retains the original morphological characteristics of the image. The two complement each other and provide a rich data basis for the construction of the difference map. However, how to reasonably allocate the weights of the two in the process of fusion of wavelet domain difference image and spatial domain difference image becomes the key to improving the fusion effect. To this end, a deep neural network model is constructed in the embodiment of the present invention, and the obtained wavelet domain difference image and spatial domain difference image are respectively used as the input of the network. Through the complex calculation of multiple layers of neurons, the importance of the two information to the construction of the difference image in different scenarios is learned, and the optimal information entropy weight is determined.

[0102] Specifically, when training the deep neural network model, the obtained wavelet domain difference image and spatial domain difference image are first preprocessed to extract the wavelet domain coefficient matrix in the wavelet domain difference image and the spatial domain feature map in the spatial domain difference image respectively; further, the deep neural network model constructed in the embodiment of the present invention adopts a residual network combined with an attention mechanism. The input layer of the network model receives the processed wavelet domain coefficient matrix and the spatial domain feature map, and performs feature extraction on the two types of information respectively through parallel convolution blocks. Each convolution block contains a 3×3 convolution layer, a batch normalization layer and a ReLU activation function, which enhances the network's nonlinear expression ability while suppressing the vanishing gradient.

[0103] In the feature fusion stage of the deep neural network model, an embodiment of the present invention introduces a spatial attention module and a channel attention module. The spatial attention module calculates the spatial correlation between pixels and highlights the spatial position information of the changed area; the channel attention module strengthens key information channels and suppresses redundant information based on the feature response strength between channels. For example, when processing post-earthquake images, the spatial attention module can focus on the spatial distribution area of collapsed buildings, and the channel attention module enhances the feature channels that reflect structural changes, enabling the network to more accurately capture disaster traces.

[0104] In order to further improve the generalization ability of the deep neural network model, the embodiment of the present invention introduces a global average pooling layer and a fully connected layer at the end of the network, maps the fused features into an adaptive weight vector, normalizes the vector through the Softmax function, and performs weighted fusion with the wavelet domain coefficient matrix and the spatial domain feature map to ensure that the sum of the weights is 1, thereby realizing the effective fusion of the two types of information.

[0105] The deep neural network model is trained using an end-to-end supervised learning approach. The loss function is set to obtain the optimal difference image, with the mean square error (MSE) and the structural similarity index (SSIM) as the joint loss function. The Adam algorithm is used as the optimizer, the initial learning rate is set to 0.001, and the cosine annealing learning rate adjustment strategy is adopted to ensure the convergence speed while avoiding falling into the local optimum.

[0106] The information entropy weighted fusion strategy of the embodiment of the present invention, which has been optimized by a neural network, can dynamically and accurately fuse wavelet domain information and spatial domain information, so that the fused difference image finally obtained by this strategy can not only clearly present the changed area and effectively distinguish the changed class from the unchanged class, but also significantly reduce the influence of background and noise. It has broad application prospects and huge technical advantages in the field of remote sensing image change detection.

[0107] The threshold segmentation algorithm is used to divide the pixels in the fused difference image to obtain the change detection results of the target heterogeneous remote sensing image.

[0108] In the embodiment of the present invention, a threshold segmentation algorithm is used to divide the pixels in the fused difference image, dividing the image pixels into two categories (foreground and background). By calculating the inter-class variance of the two categories of pixels under different thresholds, the threshold that maximizes the inter-class variance is selected as the optimal threshold. Generally, the optimal threshold can be determined by the threshold segmentation algorithm, thereby being used to distinguish between changed areas and unchanged areas in the fused difference image. Pixels with changed and unchanged positions are marked as "1" and "0", respectively, in the fused difference image, thereby realizing the detection of deformation of heterogeneous remote sensing images.

[0109] In another specific embodiment of the present invention:

[0110] Based on the heterogeneous remote sensing image change detection method, the present invention also proposes a heterogeneous remote sensing image change detection system, the structure of which is as follows: Figure 5 As shown, the system includes:

[0111] Image acquisition module, used to collect target heterogeneous remote sensing images;

[0112] The image translation module is used to perform stationary wavelet decomposition on the target heterogeneous remote sensing image and process the decomposed target heterogeneous remote sensing image using an image translation network to obtain a translated image;

[0113] A difference image extraction module is used to obtain a wavelet domain difference image and a spatial domain difference image between the translation image and the target heterogeneous remote sensing image respectively;

[0114] A difference image fusion module is used to synthesize a fused difference image between the translation image and the target heterogeneous remote sensing image based on the wavelet domain difference image and the spatial domain difference image;

[0115] The image detection module is used to divide the pixels in the fused difference image using a threshold segmentation algorithm to obtain the change detection results of the target heterogeneous remote sensing image.

[0116] In an experimental embodiment of the present invention:

[0117] The detection method proposed in the present invention was compared with several existing methods. The comparison results are shown in the following figure. Figure 4 As shown in Figure 2, three data sets were selected for testing in the embodiment of the present invention to perform image change detection respectively. The false positives, missed negatives, overall error, overall accuracy, F1 value and Kappa coefficient of the detection results were calculated respectively to evaluate the effects of different detection methods. The quantitative results are shown in Table 2, Table 3 and Table 4 respectively:

[0118] Table 2 Evaluation results on the Sardinia dataset

[0119] ;

[0120] Table 3 Schematic diagram of evaluation results on the Gloucester dataset

[0121] ;

[0122] Table 4 Schematic diagram of evaluation results on the California dataset

[0123] ;

[0124] Figure 4 middle, Figure 4 (a) is the image before the event, Figure 4(b) is the image after the event, Figure 4 (c) is the actual ground change map, Figure 4 (d) is the change detection result using the X-Net network. Figure 4 (e) is the change detection result using the ACE-Net network. Figure 4 (f) is the change detection result using the GIR-MRF method. Figure 4 (g) is the test result using the CAAE method, Figure 4 (h) is the test result using the AGSCC method, Figure 4 (i) is the test result using the SDIR method, Figure 4 (j) is the test result of the method proposed in the present invention, and Figure 4 From top to bottom, the first row is the Sardinia (Sardinia, Italy) dataset, the second row is the Gloucester (Gloucester) dataset, and the third row is the California (California) dataset. Figure 4 The changed and unchanged areas are represented by white and black, respectively. Red represents FP, which is the part that the model mistakenly judges as the changed area but is actually unchanged. Green represents FN, which is the part that the model mistakenly judges as the unchanged area but is actually changed. It can be seen that the change map generated by the method proposed in the present invention is relatively clear, and the highest F1 value and Kappa coefficient are obtained on the three tested data sets, thereby verifying the effectiveness and practicality of the heterogeneous image change detection method proposed in the present invention.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting changes in heterogeneous remote sensing images, characterized in that: include: Acquire target heterogeneous remote sensing images; The target heterogeneous remote sensing image includes a first X-domain remote sensing image and a first Y-domain remote sensing image; performing stationary wavelet decomposition on the target heterogeneous remote sensing image, and processing the decomposed target heterogeneous remote sensing image using an image translation network to obtain a translated image; The translated images include: a second Y-domain translated image corresponding to the first X-domain remote sensing image; a second X-domain translated image corresponding to the first Y-domain remote sensing image; The image translation network includes: a first generator and a second generator; The first generator is used to convert the first X-domain remote sensing image into a second Y-domain translated image, and includes a first encoder and a first decoder; the first encoder and the first decoder adopt a skip connection; The second generator is used to convert the first Y-domain remote sensing image into a second X-domain translated image, and includes a second encoder and a second decoder; the second encoder and the second decoder adopt a skip connection; respectively acquiring a wavelet domain difference image and a spatial domain difference image between the translation image and the target heterogeneous remote sensing image; The wavelet domain difference image is obtained as follows: Acquire a first low-frequency component in the first X-domain remote sensing image and a second low-frequency component in the second X-domain translated image; acquiring a first X-domain difference image according to a Euclidean distance between the first low-frequency component and the second low-frequency component; Acquire a third low-frequency component in the first Y-domain remote sensing image and a fourth low-frequency component in the second Y-domain translated image; acquiring a first Y-domain difference image according to the Euclidean distance between the third low-frequency component and the fourth low-frequency component; fusing the first X-domain difference image and the first Y-domain difference image using an adaptive weight fusion method to obtain the wavelet domain difference image; synthesizing a fused difference image between the translated image and the target heterogeneous remote sensing image according to the wavelet domain difference image and the spatial domain difference image; A threshold segmentation algorithm is used to divide the pixels in the fused difference image to obtain a change detection result of the target heterogeneous remote sensing image.

2. The method for detecting changes in heterogeneous remote sensing images according to claim 1, wherein: The image translation network also includes: Inputting the second Y-domain translated image into the second generator to obtain a third X-domain reconstructed image; The second X-domain translated image is input into the first generator to obtain a third Y-domain reconstructed image.

3. The method for detecting changes in heterogeneous remote sensing images according to claim 2, wherein: The image translation network adopts a cross-domain triple loss function, specifically: The cross-domain triple loss function includes style loss, content loss and reconstruction loss, which is expressed as: ； in, represents the cross-domain triple loss function, is the style loss, For content loss, To rebuild losses; The style loss is expressed as: ； in, represents the style loss, represents the high frequency component in the first X-domain remote sensing image, represents the high-frequency component in the second X-domain translated image, represents the high-frequency component in the first Y-domain remote sensing image, represents the high-frequency component in the second Y-domain translated image, represents the L1 norm; The content loss is expressed as: ； in, Indicates content loss, Represents the pixel value of the unchanged area in the target heterogeneous remote sensing image, represents the low-frequency component in the first X-domain remote sensing image, represents the low-frequency component in the second X-domain translated image, represents the low-frequency component in the first Y-domain remote sensing image, represents the low-frequency component in the second Y-domain translated image, represents the L1 norm; The reconstruction loss is expressed as: ； in, represents the reconstruction loss, represents the characteristic component in the first X-domain remote sensing image, represents the characteristic component in the third X domain reconstructed image, represents the characteristic component in the first Y-domain remote sensing image, represents the characteristic component in the third Y domain reconstructed image, represents the L1 norm.

4. The method for detecting changes in heterogeneous remote sensing images according to claim 2, wherein: The spatial domain difference image is obtained as follows: Inputting the first X-domain remote sensing image and the second X-domain translated image into a first encoder, obtaining a first feature vector corresponding to the first X-domain remote sensing image and a second feature vector corresponding to the second X-domain translated image; acquiring a second X-domain difference image according to the Euclidean distance between the first eigenvector and the second eigenvector; Inputting the first Y-domain remote sensing image and the second Y-domain translated image into a second encoder, obtaining a third eigenvector corresponding to the first Y-domain remote sensing image and a fourth eigenvector corresponding to the second Y-domain translated image; acquiring a second Y-domain difference image according to the Euclidean distance between the third eigenvector and the fourth eigenvector; The second X-domain difference image and the second Y-domain difference image are fused by adopting an adaptive weight fusion method to obtain the spatial domain difference image.

5. The method for detecting changes in heterogeneous remote sensing images according to claim 1, wherein: The synthesis method of the fused difference image is: The wavelet domain difference image and the spatial domain difference image are fused by using an information entropy weighted fusion method to obtain the fused difference image.

6. A heterogeneous remote sensing image change detection system, which executes a heterogeneous remote sensing image change detection method according to any one of claims 1 to 5, characterized in that: include: Image acquisition module, used to collect target heterogeneous remote sensing images; An image translation module is used to perform stationary wavelet decomposition on the target heterogeneous remote sensing image and process the decomposed target heterogeneous remote sensing image using an image translation network to obtain a translated image; A difference image extraction module is used to respectively obtain a wavelet domain difference image and a spatial domain difference image between the translation image and the target heterogeneous remote sensing image; a difference image fusion module, configured to synthesize a fused difference image between the translation image and the target heterogeneous remote sensing image based on the wavelet domain difference image and the spatial domain difference image; The image detection module is used to divide the pixel points in the fused difference image by using a threshold segmentation algorithm to obtain a change detection result of the target heterogeneous remote sensing image.

Images