Method for detecting building changes in heterogeneous remote sensing images based on mutual distillation and differential evolution

Abstract

The application particularly relates to a kind of heterogenous remote sensing image building change detection methods based on mutual distillation and difference evolution, comprising: obtaining visible light and remote sensing image under different time in the same area;Establish pseudo-twin encoder change detection network, for independently extracting heterogenous image features;Establish structure perception mutual distillation module, through the knowledge mutual distillation of structural representation robust to modal difference in heterogenous features, realize the cross-modal alignment of heterogenous features, enhance the feature extraction performance of encoder;Establish difference dynamic evolution extraction module, the mapping relationship between heterogenous features is modeled as dynamic evolution process in time domain, and based on the distinguishing characteristics of modal difference and change area difference in invariable area in dynamic evolution process, accurately extract change-related representation, effectively enhance the accuracy of change area prediction, and help reduce missed detection.

Description

Technical Field

[0001] This invention relates to the field of change detection technology, specifically to a method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution, which enables accurate detection of changes in building areas in heterogeneous remote sensing image data of synthetic aperture radar (SAR) and visible light images. Background Technology

[0002] Change detection is of great significance for disaster assessment, urban planning, and environmental protection. It can detect changes in objects of interest, providing planners and leaders with comprehensive action plans. Most existing change detection methods utilize homogeneous data acquired from similar types of sensors. However, when faced with tasks like disaster assessment that demand high timeliness and reliability, and where identical modal data may not be available in a short timeframe, homogeneous change detection methods are ineffective. Therefore, researching heterogeneous change detection methods to detect changes in real-time after a disaster, regardless of data modality, is of paramount importance, providing accurate and reliable data support for disaster assessment and subsequent relief efforts.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] This invention provides a method for detecting building changes in heterogeneous remote sensing images based on mutual distillation and differential evolution, which addresses the shortcomings of existing change detection methods that rely heavily on homogeneous data, and enables accurate detection of building change areas in heterogeneous remote sensing image data.

[0005] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0006] According to a first aspect of the present invention, a method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution is provided, the method comprising:

[0007] Acquire visible light and SAR remote sensing images of the same area at different times, label areas where building changes occur, establish a heterogeneous remote sensing image change detection dataset, and divide the dataset into training and test sets;

[0008] A pre-trained heterogeneous change detection network is constructed, comprising a pseudo-twin encoder and a decoder. The two branches of the pseudo-twin encoder have the same structure but do not share parameters, and extract features from heterogeneous images and perform progressive dimensionality reduction. Each layer of the pseudo-twin encoder includes a structured inter-distillation module for achieving cross-modal feature alignment, and a differential dynamic evolution extraction module for discriminatively extracting representations related to real change regions. The differential dynamic evolution extraction module inputs the extracted differential representations related to real change regions into the corresponding layer of the decoder through skip connections, guiding the decoder to generate more accurate and fine-grained change region prediction masks.

[0009] The heterogeneous change detection network is trained using the training set and tested using the test set to obtain the building change detection results from heterogeneous image data.

[0010] In some exemplary embodiments, the method further includes: data augmentation of the training set, the data augmentation including cropping, flipping, and shifting.

[0011] In some exemplary embodiments, the structured inter-distillation module specifically includes the following processing steps:

[0012] The heterogeneous features output by the pseudo-twin encoder are cropped into blocks. The heterogeneous features after being cropped are independently subjected to Fourier transform and high-pass filtering within the image blocks. Then, an inverse Fourier transform is performed and the image blocks are stitched back to the original input feature distribution, outputting an independent structured representation of the heterogeneous features.

[0013] Each element of the structured representation is mapped to a node, and the positional relationship between different elements is mapped to the edge connecting the nodes to construct a grid graph. The structured representation is then smoothed using a two-layer graph convolutional neural network.

[0014] A mutual attention mechanism based on transpose matrix multiplication is used to mask the smoothed structured representation; knowledge inter-distillation is then performed on the masked structured representation.

[0015] In some exemplary implementations, a loss function combining weighted and weight incrementing factors is used to perform knowledge inter-distillation on the masked structured representation.

[0016] In some exemplary embodiments, the difference dynamic evolution extraction module specifically includes the following processing steps:

[0017] The intermediate transition features for the transformation from visible light features to SAR features are obtained by convolving the absolute value of the difference between visible light features and SAR features with the concatenation of the concatenation of visible light features, feature difference, and SAR features in that order.

[0018] The visible light features, intermediate transition features, and SAR features are recombined from the perspective of temporal transformation to obtain the temporal evolution features;

[0019] 3D convolution is used to capture the overall difference representation of temporal evolution features in the temporal domain, including modal differences in unchanged regions and variation differences in changed regions;

[0020] Temporal average pooling is performed on the temporal evolution features, and effective distinction is made based on the distinguishing features exhibited by modal differences and variation differences during the temporal pooling process;

[0021] The convolution kernels of the 3D convolutions used are reconstructed, and the reconstructed 3D convolutions are used to extract the differences in temporal pooling features to obtain modal difference representations.

[0022] The overall difference representation is subtracted from the modal difference representation to obtain the difference representation that is only related to the change region. Then, 3D convolution is used to further refine and optimize the difference representation related to the change region.

[0023] In some exemplary embodiments, training the heterogeneous change detection network using a training set specifically includes: selecting AdamW as the optimizer and optimizing the network model using structured interdistillation loss and change loss.

[0024] In some exemplary embodiments, the optimization of the network model using structured interdistillation loss and variation loss includes:

[0025] We construct the MSE loss as a loss function to supervise the full interaction of structured representation knowledge in each encoder layer, and use a weighted approach to fuse the mutual distillation loss L corresponding to each encoder layer. smd :

[0026]

[0027] Among them, y i and These represent the true value and predicted value of the i-th pixel, respectively, where N is the total number of encoder layers, and λ is the true value and predicted value. w It is the weighting factor for the knowledge distillation loss at each layer;

[0028] The change loss L is constructed by combining DICE loss and BCE loss. cd To address the class imbalance problem between changing and invariant regions in heterogeneous data:

[0029]

[0030]

[0031] L cd =Ldice +λ1*L bce

[0032] Where λ1 is a hyperparameter;

[0033] By combining structural interdistillation loss and change loss, the network model is guided to accurately predict the change region.

[0034] In some exemplary embodiments, according to a second aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the heterogeneous remote sensing image building change detection method based on inter-distillation and differential evolution described in the first aspect.

[0035] According to a third aspect of the present invention, a computer program product is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, the method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution as described in the first aspect is implemented.

[0036] According to a fourth aspect of the present invention, an electronic device is provided, comprising:

[0037] A processor; and a memory for storing executable instructions of the processor;

[0038] The processor is configured to implement the heterogeneous remote sensing image building change detection method based on inter-distillation and differential evolution described in the first aspect above by executing the executable instructions.

[0039] The heterogeneous remote sensing image building change detection method based on inter-distillation and differential evolution provided by embodiments of the present invention first constructs a robust feature representation and knowledge interaction mechanism for modal differences through a pseudo-twin encoder and a structured inter-distillation module. This solves the interference of modal differences between SAR and visible light images on change detection and improves cross-modal alignment capabilities. Simultaneously, by designing a differential dynamic evolution extraction module, the relationship between heterogeneous features is modeled as a temporal evolution process, effectively distinguishing between real changes and modal pseudo-changes. Precise change region features are extracted from complex differential information, significantly improving detection accuracy. Furthermore, a joint optimization strategy of structured inter-distillation loss and ratio change loss guides the model in refined cross-modal alignment in heterogeneous data change detection tasks, improving the accuracy of change region prediction. Compared with existing homogeneous methods, this method significantly reduces dependence on data modality, possesses stronger generalization capabilities, and is applicable to various scenarios such as disaster assessment and urban planning. The method of the present invention has a clear process, an efficient detection network structure design, low computational cost, high operating efficiency, and is easy to deploy and promote, providing reliable technical support for practical engineering applications.

[0040] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0042] Figure 1 This is a flowchart of a heterogeneous remote sensing image building change detection method based on inter-distillation and differential evolution in an exemplary embodiment of the present invention.

[0043] Figure 2 This is a diagram of the heterogeneous change detection network framework in an exemplary embodiment of the present invention;

[0044] Figure 3 This is a diagram of the structured inter-distillation module in an exemplary embodiment of the present invention;

[0045] Figure 4 This is a diagram of the differential dynamic evolution extraction module in an exemplary embodiment of the present invention;

[0046] Figure 5 The heterogeneous building change detection map is generated by the method of an exemplary embodiment of the present invention. Detailed Implementation

[0047] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0048] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0049] To address the shortcomings and deficiencies of existing technologies, this example implementation provides a method for detecting building changes in heterogeneous remote sensing images based on mutual distillation and differential evolution. First, a pseudo-twin encoder change detection network is established to independently extract features from heterogeneous images. Second, a structure-aware mutual distillation module is established. By performing knowledge mutual distillation on the structured representations of heterogeneous features that are robust to modal differences, cross-modal alignment of heterogeneous features is achieved, enhancing the feature extraction performance of the encoder. Furthermore, a differential dynamic evolution extraction module is designed. This module models the mapping relationship between heterogeneous features as a dynamic evolution process in the time domain. Based on the distinguishing characteristics of modal differences in invariant regions and differences in changed regions during the dynamic evolution process, change-related representations are accurately extracted, effectively enhancing the accuracy of predicting changed regions and helping to reduce false negatives.

[0050] Specifically, this may include the following steps:

[0051] Step 1: Acquire visible light and SAR remote sensing images of the same area at different times, label the areas where building changes occur, establish a heterogeneous remote sensing image change detection dataset, and divide the dataset into training and test sets;

[0052] Step 2: Perform data augmentation on the heterogeneous image building change detection training set, such as cropping, flipping, and shifting, to generate training samples with dimensions H×W×3; where H and W are the length and width, respectively;

[0053] Step 3: Input the training samples into the heterogeneous change detection network framework, and use a pseudo-twin encoder to capture the independent representations of heterogeneous features. Each layer of the encoder is connected to a structured inter-distillation module and a differential dynamic evolution extraction module to achieve cross-modal alignment of heterogeneous features and discriminative and accurate extraction of change regions.

[0054] Specifically, the heterogeneous change detection network framework includes a pair of pseudo-twin encoders and a decoder. The two branches of the pseudo-twin encoders have the same structure but do not share parameters, and extract features from heterogeneous images and perform progressive dimensionality reduction. Each layer of the pseudo-twin encoder includes a structured inter-distillation module for achieving cross-modal feature alignment, and a differential dynamic evolution extraction module for discriminatively extracting representations related to real change regions, which enhances the network's robustness to modal differences and improves the accuracy of change detection. The differential dynamic evolution extraction module inputs the extracted differential representations related to real change regions into the corresponding size decoder layer through skip connections, which guides the decoder to generate more accurate and fine-grained change region prediction masks.

[0055] Step 4: Input the output of each layer of the pseudo-twin encoder in Step 3 into the structured inter-distillation module of the current layer. By performing knowledge inter-distillation on the structured representation of heterogeneous features, the feature alignment of the two source inputs is achieved, thereby guiding the encoder to overcome the interference of modal differences on the extraction of the representation of changes in building areas, so as to improve the robustness and discriminative ability of the pseudo-twin decoder.

[0056] Specifically, the structured inter-distillation module includes the following processing steps:

[0057] Step 4-1: The heterogeneous features output by the pseudo-twin encoder are cropped into blocks. Different block sizes are selected according to the different sizes of the encoder output features. The heterogeneous features after being cut into blocks are then subjected to Fourier transform independently within the image blocks. After centering the transformation results, high-pass filtering is performed to filter out low-frequency information in the heterogeneous features, retaining only the transformation and texture information of the reaction features with drastic gradient changes. Then, inverse Fourier transform is performed and the image blocks are stitched back to the original input feature distribution to output an independent structured representation of the heterogeneous features.

[0058] Step 4-2: Construct a graph network, mapping each element of the heterogeneous features to a node, and using edges to represent the distribution between different nodes. That is, each node is only connected to its four adjacent nodes. Embed the original structured representation into the corresponding node, and process the structured representation through a two-layer graph convolutional neural network to achieve smoothness of the structured representation and avoid the original extracted structured representation being too discrete, which would prevent the knowledge distillation process from playing a positive role.

[0059] Step 4-3: The mutual attention mechanism implemented through transpose matrix multiplication focuses on the regions in the heterogeneous features that have not changed. Based on this attention, the structured representation is masked to shield the regions that have actually changed as much as possible, so as to avoid the subsequent knowledge inter-distillation process from affecting the representation of the changed regions and the accuracy of the final change detection.

[0060] Step 4-4: Perform knowledge inter-distillation on the masked structured representation to achieve full knowledge interaction of heterogeneous features, thereby realizing cross-modal alignment of features and improving the robustness of the pseudo-twin encoder to modal differences and its feature extraction capability for real-world variations.

[0061] Step 5: Input the two outputs of each layer of the pseudo-twin encoder in Step 3 into the differential dynamic evolution extraction module. In the module, the transformation correlation between heterogeneous features is modeled as a time-domain evolution process. Then, based on the distinguishing characteristics shown by different differential representations in the time-domain evolution process, the differential representations related to the real changes are accurately extracted.

[0062] Specifically, the differential dynamic evolution extraction module includes the following processing steps:

[0063] Step 5-1: Convolve the absolute value of the difference between visible light features and SAR features with the original features to obtain intermediate transition features that transform from visible light to SAR features;

[0064] Step 5-2: Reorganize the visible light features, intermediate transition features, and SAR features according to the perspective of temporal transformation, and construct a 5-dimensional temporal evolution feature from the three independent 4-dimensional features;

[0065] Step 5-3: Use 3D convolution to capture the overall difference representation of the temporal evolution features of Step 5-2 in the temporal domain, including modal differences in unchanged regions and change differences in changed regions, which are complexly coupled.

[0066] Step 5-4: Perform temporal 3D average pooling on the temporal evolution features described in Step 5-2, and effectively distinguish them based on the distinguishing features exhibited by the modal differences and variation differences during the temporal pooling process.

[0067] Step 5-5: Reconstruct the 3D convolution used in Step 5-3, and use the reconstructed 3D convolution to extract the differences from the temporal pooling features obtained in Step 5-4, so as to maintain the modal difference representation with the same difference distribution and inductive bias as the overall difference.

[0068] Step 5-6: Subtract the overall difference characterization obtained in Step 5-3 from the modal difference characterization obtained in Step 5-5 to obtain the difference characterization that is only related to the change region.

[0069] Step 6: Input the difference representation of the real change region extracted layer by layer by the difference dynamic evolution extraction module on the pseudo-twin encoder side in Step 5 into the decoder of the corresponding size, and fuse it with the decoder information to guide the decoder to generate a more refined change region mask.

[0070] Step 7: Select AdamW as the optimizer and use structured interdistillation loss and variation loss to optimize the model, achieve cross-modal alignment of heterogeneous features, enhance the robustness of the encoder to modal differences, and improve the accuracy of prediction of change regions.

[0071] The following will describe in more detail each step of the heterogeneous remote sensing image building change detection method based on inter-distillation and differential evolution in this exemplary embodiment, with reference to the accompanying drawings and embodiments.

[0072] refer to Figure 1-5 The specific steps of the method for detecting building changes in heterogeneous remote sensing images based on mutual distillation and differential evolution provided in this invention are as follows:

[0073] Figure 1The flowchart of the method according to an embodiment of the present invention includes the following steps: acquiring heterogeneous building change data samples and labeling the data using Label_cd software; performing data augmentation on the data to generate training and testing sets; constructing a heterogeneous change detection network framework, and building a structured inter-distillation module and a differential dynamic evolution extraction module on it; training the heterogeneous change detection network using the training set; and inputting the test data into the trained heterogeneous change detection network for testing to obtain the building change detection results of the heterogeneous image data. The method specifically includes the following steps:

[0074] Step 1: Acquire large-scale SAR and visible light remote sensing image data of the same area taken at different time points by remote sensing satellites. Select areas where buildings have changed for processing and segmentation. Use Labelme_cd to label the areas where buildings have changed. Segment the large-scale remote sensing images and construct a standard 512×512 dataset by using a 1 / 2 overlap segmentation method. Divide the above samples into training and test sets for the heterogeneous image building change detection dataset in an 8:2 ratio.

[0075] Step 2: Perform data augmentation on the heterogeneous image building change detection training set, such as cropping, flipping, and shifting, to generate training samples with dimensions of 512×512×3;

[0076] Step 3: Input the training samples obtained in Step 2 into the heterogeneous change detection network framework. Use a pseudo-twin encoder to capture the independent representation of heterogeneous features. Each layer of the encoder is connected to a structured inter-distillation module and a differential dynamic evolution extraction module to achieve cross-modal alignment of heterogeneous features and discriminative and accurate extraction of change regions.

[0077] For example, refer to Figure 2 The heterogeneous change detection network framework includes the following steps:

[0078] Step 3-1: The encoder of the heterogeneous change detection network framework is designed based on ResNet18 and adopts a pseudo-twin structure, that is, the two encoder branches have the same network structure, but do not share weights with each other, and process features from the light image and SAR image respectively.

[0079] Step 3-2: The encoder downsamples the image features to 1 / 2, 1 / 4, 1 / 8, and 1 / 16 times their original values ​​through a four-encoder layer, and obtains high-level discriminative semantic features; each encoder layer is followed by a structured inter-distillation module and a differential dynamic evolution extraction module.

[0080] Step 3-3: The semantic features of the two heterogeneous high-level discriminative lines output by the pseudo-twin encoder are concatenated through channels and input into the decoder. The corresponding decoder contains a four-layer structure, each layer containing an upsampling module and a deconvolution module. Layer by layer, it is fused with the real change region representation output by the difference dynamic evolution extraction module connected to the encoder of the same layer. The final output is output as a change region mask through a change detection prediction head composed of two layers of fully convolutional network FCN.

[0081] Step 4: Input the two heterogeneous feature outputs of each layer of the pseudo-twin encoder described in Step 3 into the structured inter-distillation module of the current layer. By extracting the structured representations of the heterogeneous features respectively, a graph convolutional neural network is used to smooth the structured representations. After calculating mutual attention on the original input to mask the structured representations of the changing regions, knowledge inter-distillation is performed on the remaining structured representations to achieve feature cross-modal alignment and enhance the encoder's robustness to modal differences and its ability to discriminate modal difference interference.

[0082] For example, refer to Figure 3 The structured interdistillation module includes the following steps:

[0083] Step 4-1: According to the size of the input layer, the heterogeneous features are cropped into blocks of 16×16, 8×8, and 4×4 respectively, and an independent Fourier high-pass filter is performed in each image block to retain only 10% of the high-frequency information for processing. This information removes low-frequency features related to modality differences and retains only the structured representation. Then, the image blocks are reassembled and stitched together. Block Fourier structure-aware filtering helps to capture more detailed structural representations.

[0084] Step 4-2: Map each element of the structured representation to a node, and map the positional relationship between different elements to the edges connecting the nodes to construct a grid graph. Use a 2-layer graph convolutional neural network to smooth the structured representation, reduce discrete features, and effectively suppress noise to improve the performance of subsequent knowledge inter-distillation.

[0085] Step 4-3: Based on the heterogeneous features before filtering, the mutual attention mechanism of transpose matrix multiplication is used to obtain the attention representation of the region that has not actually changed, and it is used to mask the above smoothed structured representation to ensure that the network focuses on cross-modal alignment of the unchanged region during the mutual distillation process, without affecting the detection and extraction of the changed region.

[0086] Step 4-4: Use a loss function that combines weighting and weight incrementing factors to perform knowledge inter-distillation on the masked heterogeneous structured representation. The superposition of multiple encoders gradually realizes cross-modal alignment of heterogeneous features, improving the robustness of the encoder to modal differences.

[0087] Step 5: Input the two outputs of each layer of the pseudo-twin encoder in Step 3 into the differential dynamic evolution extraction module. Convolve the two outputs with their absolute interpolation and combine them with the original features to construct temporal evolution features. Accurately distinguish the differences in real change regions and modal differences through 3D convolution reconstruction and temporal average pooling.

[0088] For example, refer to Figure 4 The differential dynamic evolution extraction module includes the following steps:

[0089] Step 5-1: Obtain the absolute value of the difference between visible light features and SAR features, concatenate the channels with the original features, and then use convolution to fuse the features, reducing the number of channels to the original input size, as an intermediate state for the conversion of visible light features to SAR features from the temporal perspective.

[0090] Step 5-2: View the relationship between visible light features and SAR features as an evolution process in the time domain. Expand the original features into the time dimension, that is, change the size from [B,C,H,W] to [B,C,T,H,W]. Stack the visible light features, intermediate state features, and SAR features in the time dimension in sequence to construct the transformation process from visible light features to SAR features.

[0091] Step 5-3: Use 3D convolution to extract temporal evolution features from the time, length, and width dimensions to capture the differential representation between heterogeneous features;

[0092] Step 5-4: Perform temporal 3D average pooling on the temporal evolution features. Regions with only modal differences but no changes are enhanced during the average pooling process, while the difference representation of regions with real changes is suppressed during the temporal average pooling process due to spatial discontinuity.

[0093] Step 5-5: Reconstruct the 3D convolution used in Step 5-3, and fuse it using only the convolution kernel weight layer that acts on visible light features and SAR features. Extract modal difference representation from the evolution features after temporal average pooling using the same inductive bias.

[0094] Step 5-6: Subtract the overall difference characterization obtained in Step 5-3 from the modal difference characterization obtained in Step 5-5 to obtain the difference characterization that is only related to the change region.

[0095] Step 6: Input the difference representation of the real change region extracted layer by layer by the difference dynamic evolution extraction module on the pseudo-twin encoder side in Step 5 into the decoder of the corresponding size, and fuse it with the decoder information to guide the decoder to generate a more refined change region mask.

[0096] Step 7: Select AdamW as the optimizer and use a heterogeneous change detection network framework to optimize the loss function, namely the structured cross-distillation loss L. smd With change loss L cd The model is optimized to achieve cross-modal alignment of heterogeneous features, enhance the encoder's robustness to modal differences, and improve the accuracy of predicting changing regions.

[0097] L = L smd +L cd

[0098] For example, optimizing the loss function in a heterogeneous change detection network framework includes the following steps:

[0099] Step 7-1: Construct the MSE loss as the loss function to supervise the full interaction of structured representation knowledge in each encoder layer, and fuse the Hu distillation loss corresponding to each encoder layer in a weighted manner:

[0100]

[0101] Among them, y i and These represent the true value and predicted value of the i-th pixel, respectively, where N is the total number of encoder layers, and λ is the true value and predicted value. w It is a weighting factor of the knowledge distillation loss at each layer. In the early stages of training when the encoder's feature extraction is not accurate enough, it provides a tolerance for knowledge inter-distillation. In the later stages of training, it enhances the strength of knowledge inter-distillation and guides the model's refined cross-modal alignment.

[0102]

[0103] Where t and T are the current training round and the total training round, respectively;

[0104] Step 7-2: Construct the change loss L by combining DICE loss and BCE loss. cd To address the class imbalance problem between changing and invariant regions in heterogeneous data:

[0105]

[0106] L cd =L dice +λ1*L bce

[0107] Wherein, λ1 is a hyperparameter used to balance the DICE and BCE losses, so that DICE does not overemphasize the changing region, which would lead to an increase in misjudgments;

[0108] Step 7-3: By effectively combining variation loss and structural interdistillation loss, guide the model to accurately predict the variation region:

[0109] L = L cd +L smd

[0110] The above-mentioned method for detecting building changes in heterogeneous remote sensing images based on mutual distillation and differential evolution is used to detect changes in SAR and visible light heterogeneous remote sensing images.

[0111] This invention provides a method for detecting building changes in heterogeneous remote sensing images based on mutual distillation and differential evolution, which enables accurate detection of changes in building areas in heterogeneous remote sensing image data of synthetic aperture radar (SAR) and optically reflective images. This includes: acquiring visible light and remote sensing images of the same region at different times; using the Labelme_cd tool to label areas where building changes have occurred; establishing a heterogeneous remote sensing image change detection dataset; performing data augmentation on this dataset to generate training and testing sets; constructing a heterogeneous change detection network; employing a pseudo-twin encoder to independently extract unique representations from the two source images; and designing a single decoder with layer-by-layer residual connections to fuse the two source representations and generate a refined change region prediction mask; constructing a structured mutual distillation module to perform knowledge mutual distillation on the structured representations of the two source images robust to modal differences, achieving cross-modal alignment between SAR and visible light heterogeneous images, solving the interference of modal differences on the change detection process, and improving change detection accuracy; and constructing a difference dynamics and evolution extraction module to model the mutual conversion and correlation of visible light and SAR image features as a temporal evolution process. Based on the distinguishing features of pseudo-changes and real change regions introduced by modality in the temporal evolution process, it effectively and accurately extracts change-related difference feature information from various complexly coupled difference information, guiding the network to reconstruct a more accurate change region mask. This invention can adapt to the huge modal differences in heterogeneous remote sensing data and achieve accurate detection of building change areas. Compared with methods that require homogeneous data, it has stronger generalization ability and is easier to promote and use.

[0112] refer to Figure 5 The method of this invention can effectively achieve cross-modal alignment of heterogeneous features and accurately extract the difference representations related to real changes from complex coupled modal differences and change differences, thereby achieving accurate detection of building changes. The effectiveness of the method of this invention is further illustrated by the following simulation experiments.

[0113] 1. Simulation conditions.

[0114] The method of this invention is a simulation performed using Anaconda software on a CPU with an Intel Core i7-9750H CPU, 32GB of memory, an Nvidia RTX3090 graphics card, and a Windows 10 operating system.

[0115] 2. Simulation content.

[0116] The simulation used 2360 SAR and optically sensitive remote sensing images with a size of 52×512 to describe changes in buildings in the captured area.

[0117] To demonstrate the effectiveness of the method, the method was applied to the aforementioned image dataset and compared with 14 different state-of-the-art methods. Four different evaluation metrics were used to measure the comparison results, which are shown in Table 1.

[0118] As can be seen from Table 1, the damage assessment results of the present invention can achieve the best performance. When applied to disaster assessment and other fields, the present invention can effectively improve execution efficiency and provide more accurate detection results.

[0119] Table 1

[0120]

[0121] It should be noted that, as another aspect, this application also provides a storage medium, which may be included in an electronic device or may exist independently without being assembled into the electronic device. The storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to perform the methods described in the following embodiments.

[0122] In one embodiment, this application provides a computer program product including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0123] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0124] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0125] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is defined only by the appended claims.

Claims

1. A method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution, characterized in that, The method includes: Acquire visible light and SAR remote sensing images of the same area at different times, label areas where building changes occur, establish a heterogeneous remote sensing image change detection dataset, and divide the dataset into training and test sets; A pre-trained heterogeneous change detection network is constructed, comprising a pseudo-twin encoder and a decoder. The two branches of the pseudo-twin encoder have the same structure but do not share parameters, extracting features from heterogeneous images and progressively reducing dimensionality. Each layer of the pseudo-twin encoder includes a structured mutual distillation module for achieving cross-modal feature alignment, and a differential dynamic evolution extraction module for discriminatively extracting representations related to real change regions. The differential dynamic evolution extraction module inputs the extracted differential representations related to real change regions into the corresponding layer of the decoder via skip connections, guiding the decoder to generate more accurate and fine-grained change region prediction masks. The structured mutual distillation module specifically includes the following processing steps: The heterogeneous features output by the pseudo-twin encoder are cropped into blocks. The heterogeneous features after being cropped are independently subjected to Fourier transform and high-pass filtering within the image blocks. Then, an inverse Fourier transform is performed and the image blocks are stitched back to the original input feature distribution, outputting an independent structured representation of the heterogeneous features. Each element of the structured representation is mapped to a node, and the positional relationship between different elements is mapped to the edge connecting the nodes to construct a grid graph. The structured representation is then smoothed using a two-layer graph convolutional neural network. A mutual attention mechanism based on transpose matrix multiplication is used to mask the smoothed structured representation; knowledge inter-distillation is then performed on the masked structured representation. The heterogeneous change detection network is trained using the training set and tested using the test set to obtain the building change detection results from heterogeneous image data.

2. The method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution according to claim 1, characterized in that, The method further includes: data augmentation of the training set, wherein the data augmentation includes cropping, flipping, and shifting.

3. The method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution according to claim 1, characterized in that, A loss function combining weighting and weight incrementing factors is used to perform knowledge inter-distillation on the masked structured representation.

4. The method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution according to claim 1, characterized in that, The difference dynamic evolution extraction module specifically includes the following processing steps: The intermediate transition features for the transformation from visible light features to SAR features are obtained by convolving the absolute value of the difference between visible light features and SAR features with the concatenation of the concatenation of visible light features, feature difference, and SAR features in that order. The visible light features, intermediate transition features, and SAR features are recombined from the perspective of temporal transformation to obtain the temporal evolution features; 3D convolution is used to capture the overall difference representation of temporal evolution features in the temporal domain, including modal differences in unchanged regions and variation differences in changed regions; Temporal average pooling is performed on the temporal evolution features, and effective distinction is made based on the distinguishing features exhibited by modal differences and variation differences during the temporal pooling process; The convolution kernels of the 3D convolutions used are reconstructed, and the reconstructed 3D convolutions are used to extract the differences in temporal pooling features to obtain modal difference representations. The overall difference representation is subtracted from the modal difference representation to obtain the difference representation that is only related to the change region. Then, 3D convolution is used to further refine and optimize the difference representation related to the change region.

5. The method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution according to claim 1, characterized in that, The training of the heterogeneous change detection network using the training set specifically includes: selecting AdamW as the optimizer and optimizing the network model using structured inter-distillation loss and change loss.

6. The method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution according to claim 5, characterized in that, The optimization of the network model using structured interdistillation loss and variation loss includes: We construct the MSE loss as a loss function to supervise the full interaction of structured representation knowledge in each encoder layer, and use a weighted approach to fuse the mutual distillation losses corresponding to each encoder layer. : in, and Representing the first i The true and predicted values ​​of each pixel, where N is the total number of encoder layers. It is the weighting factor for the knowledge distillation loss at each layer; Constructing a change loss by combining DICE loss and BCE loss To address the class imbalance problem between changing and invariant regions in heterogeneous data: in, It's a hyperparameter; By combining structural interdistillation loss and change loss, the network model is guided to accurately predict the change region.

7. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution as described in any one of claims 1 to 6.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for detecting building changes in heterogeneous remote sensing images based on inter-distillation and differential evolution, as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, include: processor; and memory for storing the executable instructions of the processor; The processor is configured to execute the heterogeneous remote sensing image building change detection method based on inter-distillation and differential evolution as described in any one of claims 1 to 6 by executing the executable instructions.

Images