Remote Sensing Image Change Detection Method Based on Dynamic Entropy Driving and Cooperative Enhancement Network

Abstract

The present invention relates to a method for remote sensing image change detection based on dynamic entropy-driven and collaborative enhancement network, which is applicable to the change detection of hyperspectral remote sensing images. The method of extracting differential features of dual-temporal images is adopted by using the method of dynamic entropy-driven differential feature extraction. Among them, the dynamic entropy-driven method is designed according to the hyperspectral differential features, and different values are assigned according to the magnitude of the fluctuation of the differential features, which improves the performance of hyperspectral change detection. The extraction of collaborative features of dual-temporal images based on the collaborative enhancement network adopts an interactive aggregation method. By using the features of dual-temporal hyperspectral images, while extracting collaborative features, the differential features are enhanced, which is superior to the existing hyperspectral change detection methods.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image processing and change detection, and particularly relates to a remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network. Background Art

[0002] Change detection aims to accurately identify changes in the Earth's surface or specific targets by comparing geospatial data acquired at different times. This technology is crucial for monitoring surface dynamics and has been widely applied in fields such as environmental monitoring, urban planning, disaster response, agricultural surveillance, and forest protection. Hyperspectral imaging technology can capture subtle differences in objects or surface materials within a wide spectral range, thus significantly improving the accuracy and sensitivity of change detection.

[0003] Deep learning constructs a deep network model with a hierarchical structure by simulating the hierarchical working mode of the human visual system and has demonstrated excellent performance in the field of hyperspectral image change detection. In 2020, Seydi et al. proposed a novel end-to-end convolutional neural network that combines 1D, 2D, and 3D convolutions to optimize feature extraction, thereby achieving excellent detection results. This is mainly due to the fact that the CNN model can effectively extract spatial features through local connection and weight sharing, while significantly reducing the network training parameters. Subsequently, many scholars have proposed a series of improved models based on CNN. To address the problem of single scale, Li et al. and Luo et al. proposed multi-scale convolution methods in 2022 and 2023 respectively to extract features of bi-temporal hyperspectral images and fuse shallow and deep features, thereby detecting subtle change information in the images and achieving remarkable results. To solve the local receptive field limitation in CNN, in 2022, Chen et al. introduced the Transformer architecture into the change detection field to capture long-term dependencies in the images and further improve the detection performance. Song et al. adopted three Transformer network architectures to learn the spatio-temporal spectral information of hyperspectral images and obtained good performance.

[0004] Although the above deep learning networks can automatically learn spatial and spectral features from data, it can be seen from these research works that there are still significant potential and challenges in extracting change features and better capturing change information from bi-temporal images. First, CNN-based methods face difficulties in extracting long-term dependence information in images, mainly because the local receptive field of CNN limits its ability to capture global features. Although CNN performs excellently in spatial feature extraction through local connection and weight sharing, its performance may be limited when dealing with large-scale and complex changes. Second, although Transformer-based methods can model global dependencies through self-attention mechanisms, they still have deficiencies in making full use of key-value pairs of bi-temporal images to obtain change information. Although the Transformer architecture is good at processing sequence data, in specific change detection tasks, its sensitivity to time series and the ability to capture subtle changes between bi-temporal images may not be sufficient. Finally, although the current change detection methods using attention mechanisms perform well in other tasks, these methods often lack an optimized design specifically for change detection. The existing attention mechanisms mainly focus on global or specific region feature extraction and are not customized for the change detection task, thus may not be able to fully capture the small but important change information in images.

[0005] Therefore, there is an urgent need for a new remote sensing image change detection method to solve the above problems. Summary of the Invention

[0006] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a remote sensing image change detection method based on dynamic entropy-driven and collaborative enhancement network.

[0007] This remote sensing image change detection method based on dynamic entropy-driven and collaborative enhancement network includes the following steps:

[0008] Step 1: Segment the bi-temporal hyperspectral data into multiple samples and divide them into a training set and a test set;

[0009] Step 2: Construct a deep learning network, extract the difference features between bi-temporal hyperspectral images through a dynamic entropy-driven difference extractor; generate the collaborative features after enhancement of bi-temporal hyperspectral images through a temporal collaborative reinforcement;

[0010] Step 3: Input the difference features and collaborative features into a multi-scale feature optimization and integrator for feature screening and fusion processing; send the fused features into a classifier to generate detection results, and save the trained model parameters;

[0011] Step 4: Input the training set samples into the network to obtain prediction results, then calculate the overall accuracy of the dynamic entropy-driven and collaborative enhancement network predictor, and update the network parameters; repeat Steps 2 to 4 until the number of iterations reaches the requirement;

[0012] Step 5: Input the test set samples into the final deep feature extraction network model and output the detected sample results.

[0013] Preferably, in Step 2, the calculation method for extracting the difference features between the two-temporal hyperspectral images is specifically as follows: First, subtract the two-temporal hyperspectral image blocks, input them into the dynamic entropy-driven difference extractor for feature extraction, flatten the features after passing through a linear mapping layer, and add positional encoding.

[0014] Preferably, in Step 2, use probability to represent the proportion of each pixel value in the entire channel, calculate of the entropy value to obtain the attention map of the change information; through the residual connection, perform deeper feature learning, and send the learned features into the channel enhancement module.

[0015] Preferably, in Step 2, perform temporal collaborative enhancement on the two-temporal hyperspectral images through the temporal collaborative enhancer. The feature focusing of the temporal collaborative enhancement includes the following steps: Pass the features of the two-temporal hyperspectral images through a linear layer to obtain Q, K, V, adopt the structure in the feature set, push the adjacent vectors to their nearest features, adjust the direction of each query to the direction of the key features through the norm-preserving mapping, make the similar Q, V pairs close, and separate the dissimilar Q, V pairs to obtain the enhanced collaborative features of the two-temporal hyperspectral images.

[0016] Preferably, in Step 3, input each element in the difference features and collaborative features obtained in Step 2 into the multi-scale feature optimization and integration device, and pass through the forgetting layer and the input layer in sequence, and then fuse the features of the forgetting layer and the input layer.

[0017] The beneficial effects of the present invention are:

[0018] 1) The remote sensing image change detection model based on dynamic entropy-driven and collaborative enhancement network established by the present invention is applicable to the change detection of hyperspectral remote sensing images, and adopts the method of dynamic entropy-driven difference feature extraction to extract the difference features of the two-temporal images; among them, the dynamic entropy-driven method is designed according to the hyperspectral difference features, and different values are assigned according to the magnitude of the fluctuation of the difference features, improving the performance of hyperspectral change detection.

[0019] 2) The extraction of the dual-temporal image collaborative features based on the collaborative enhancement network in the present invention adopts an interactive aggregation method. By utilizing the features of the dual-temporal hyperspectral images, while extracting the collaborative features, the differential features are enhanced. Through the experiments conducted on the public hyperspectral change detection dataset, as far as the currently available information is concerned, the change detection method proposed by the present invention is superior to the existing hyperspectral change detection methods. Description of the Drawings

[0020] Figure 1 It is a schematic flow chart of the method of the present invention;

[0021] Figure 2a It is a false-color image at the moment of synthesizing by selecting 3 bands from 198 original bands;

[0022] Figure 2b It is a false-color image at the moment of synthesizing by selecting 3 bands from 198 original bands;

[0023] Figure 2c It is a ground truth map for change detection;

[0024] Figure 3a It is a ground truth map for change in the River dataset;

[0025] Figure 3b It is a change detection result map of GTMSiam;

[0026] Figure 3c It is a change detection result map of the present invention. Detailed Embodiment

[0027] The following further describes the present invention in conjunction with embodiments. The description of the following embodiments is only for helping to understand the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several modifications can still be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0028] Embodiment 1

[0029] As an embodiment, as Figure 1 shown, this remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network includes the following steps:

[0030] Step 1. Data preprocessing. First, the dual-temporal hyperspectral data is segmented into multiple pixel cubes, and each pixel cube represents a sample. In this embodiment, the data format of the initial image is a three-dimensional data matrix in.mat format. We first segment the data cube, take a 7x7 cube to form a pixel cube. The overall information of the pixel cube is used to replace the information of the central pixel point for processing. X krepresents the spectral information of the k-th pixel cube, represents the spectral information of the pixel at the (i, j) position of the k-th pixel cube.

[0031] Then, the artificially labeled samples are divided into the training set, and the unlabeled samples are divided into the test set. In this embodiment, 1% of the samples are randomly selected from all the preprocessed data samples as the training set for model training; the remaining 99% of the samples are used as the test set to evaluate the performance of the model.

[0032] Step 2: Construct a deep learning network, which includes a dynamic entropy-driven difference extractor and a temporal collaborative enhancer.

[0033] The described dynamic entropy-driven difference extractor extracts difference features by calculating the differences between two-temporal hyperspectral images. The temporal collaborative enhancer uses the two-temporal input data for synchronous enhancement to generate enhanced collaborative features.

[0034] The described dynamic entropy-driven difference extractor combines information entropy and channel enhancement mechanism, and the temporal collaborative enhancer combines feature focusing mechanism and collaborative enhancement mechanism;

[0035] Step 3: Input the difference features and collaborative features obtained in Step 2 into a multi-scale feature optimization and integrator for feature screening and fusion processing. Subsequently, the fused features are fed into a classifier to generate detection results and save the trained model parameters;

[0036] Step 4: Fix the trained network parameters, input the samples in Step 1 into the network to obtain prediction results, and increment the iteration count by 1. Then, compare the predicted values with the corresponding true values, calculate the cross-entropy loss to evaluate the performance of the classifier, and calculate the overall accuracy (OA value) of the dynamic entropy-driven and collaborative enhancement network predictor after this iteration. If the iteration count reaches the preset maximum number, proceed to the next step; otherwise, return to Step 2 and update the network parameters to the parameters after this iteration;

[0037] Step 5: Input the test set samples into the final deep feature extraction network model to output the detected sample results.

[0038] Embodiment 2

[0039] As another embodiment, this Embodiment 2 is proposed based on Embodiment 1, and a more specific remote sensing image change detection method based on a dynamic entropy-driven and collaborative enhancement network:

[0040] In the second step, based on the calculation method of extracting differential features driven by dynamic entropy, first subtract the two-temporal hyperspectral image patches and then input them into the differential extractor driven by dynamic entropy. The feature extraction method of this differential extractor driven by dynamic entropy is as follows: Assume that the input feature of the differential extractor driven by dynamic entropy is, after passing through a linear mapping layer, it is flattened and positional encoding is added. The specific calculation formula is as follows:

[0041] D′ = F(BN(DW)) + E pos (1)

[0042] where W represents the linear layer parameter, F represents the flattening operation, and E pos represents adding relative positional encoding. Subsequently, a probability-based method is used to represent the proportion of each pixel value in the entire channel. The specific calculation formula is as follows:

[0043]

[0044] After that, calculate the entropy value of to obtain the attention map of the change information. The specific calculation formula is as follows:

[0045]

[0046] where C represents the dimension of the channel. Then, through the residual connection, while retaining the original input information, deeper features are learned. The specific calculation formula is as follows:

[0047]

[0048] Finally, the learned features are fed into the channel enhancement module. The specific formula is as follows:

[0049]

[0050] where, represents a 1×1 convolution kernel, and G represents the activation function.

[0051] It should be noted that the same or similar parts in this embodiment and Embodiment 1 can be referred to each other and will not be elaborated in this application.

[0052] Embodiment 3

[0053] As another embodiment, this Embodiment 3 is proposed on the basis of Embodiments 1 and 2. A more specific remote sensing image change detection method based on dynamic entropy-driven and collaborative enhancement network. In the second step, the feature focusing mechanism and collaborative enhancement mechanism of the temporal collaborative intensifier are specifically as follows:

[0054] In step two, the feature focusing mechanism in the temporal collaborative enhancement mainly includes the following steps. For the input feature x, first pass it through a linear layer to obtain Q, K, and V. Different from the traditional dot product attention, in this embodiment, a feature concentration structure is adopted to push adjacent vectors towards their nearest features to highlight similar features. The specific calculation formula is as follows:

[0055] Sim(Q,K)=FRL(Q)·FRL(K) T (7)

[0056] Among them, FRL passes the input feature through an activation function layer, and then adjusts the direction of each query to the direction of the key feature through a norm-preserving mapping, so that similar query-key pairs are closer and dissimilar query-key pairs are separated. The specific calculation formula of the norm-preserving mapping is as follows:

[0057]

[0058] where m **p means multiplying each pixel in m by the p-th power. Then combine the corrected attention feature Sim(Q,K) with W. The specific calculation formula is as follows:

[0059]

[0060] where DWC represents depthwise separable convolution.

[0061] Furthermore, the dual-temporal image collaborative enhancement mechanism in the temporal collaborative enhancement in step two mainly includes the following steps. Assume that the two input features are respectively and. First, pass them through linear mapping layers to obtain W1, Q1, V1, W2, Q2, V2. Then perform feature interaction to obtain collaborative features. The specific calculation formula is as follows:

[0062]

[0063] Through the above interaction, the collaborative features of the dual-temporal images can be extracted while enhancing their difference information.

[0064] It should be noted that the same or similar parts in this embodiment and Embodiment 1 and Embodiment 2 can be referred to each other and will not be elaborated in this application.

[0065] Embodiment 4

[0066] As another embodiment, based on Embodiments 1 to 3, this Embodiment 4 proposes a more specific remote sensing image change detection method based on dynamic entropy-driven and collaborative enhancement network:

[0067] The multi-scale feature optimization and integration mechanism in step 3 mainly includes the following. For a given intermediate feature A, each of its elements is taken out and arranged in a row. For each input, these features will each undergo feature screening and fusion through the multi-scale feature optimization and integrator. For the k-th intermediate step, the feature will first pass through a forgetting layer, and the specific calculation formula is as follows:

[0068] f k =σ(W fa a k +W fh h k-1 +b f ) (12)

[0069] Among them, W fa and W fh are both coefficient matrices, σ represents the activation function, and b f represents the bias term. Then it is sent to the input layer, which is mainly responsible for deciding whether to add new information to the memory cell. The specific calculation formula is as follows:

[0070] i k =δ(W ia a k +W ih h k-1 +b i ) (13)

[0071] Among them, W ia and W ih are both coefficient matrices, and b i represents the bias term of the input layer. Subsequently, the features of the forgetting layer and the input layer are fused, and the specific formula is as follows:

[0072]

[0073] Among them, ⊙ represents pointwise multiplication, W ca and W ch represent weight matrices, and tanh represents the activation function. After that, all the information is fused to obtain the output result, and the specific formula is as follows:

[0074] h k =δ(W oa a k +W oh h k +b o )tanh(c k ) (16)

[0075] Among them, W oa and W oh represent the coefficient matrices of the output layer, and b oRepresents the bias term.

[0076] It should be noted that the parts that are the same or similar to those in the first to third embodiments in this embodiment can be referred to each other and will not be elaborated in this application.

[0077] Embodiment 5

[0078] As another embodiment, based on the first to fourth embodiments, this Embodiment 5 proposes to use this remote sensing image change detection method based on dynamic entropy-driven and collaborative enhancement network, and conduct experiments on the River hyperspectral change detection dataset:

[0079] The River hyperspectral dataset is one of the public datasets widely used in the field of hyperspectral change detection.

[0080] In this embodiment, the River dataset selects a river near Jiangsu Province, China. The data was acquired through the EO-1 Hyperion sensor on May 3, 2013 and December 31, 2013 respectively. The resolution of each image is 463×241 pixels, including 198 spectral bands, the spatial resolution is 30m, and the spectral resolution is 10nm. This dataset is mainly used to study the detection of material changes in the river.

[0081] The River hyperspectral data is as Figures 2a to 2c shown, Figure 2a is a false-color image at the moment of synthesizing by selecting 3 bands from 198 original bands, Figure 2b is a false-color image at the moment of synthesizing by selecting 3 bands from 198 original bands, Figure 2c is the ground truth map for change detection.

[0082] In this experiment, 1% of the training samples were randomly selected for training, and the remaining samples were used as the test set. To verify the effectiveness of the algorithm of the present invention, in this embodiment, the method proposed by the present invention is compared with the latest method of GTMSiam for hyperspectral image change detection based on gated transmission. The change detection results are shown in Table 1.

[0083] Table 1: Comparison of classification results by selecting 5 labeled samples for each class on the SA data

[0084]

[0085] It can be seen from the change detection result table that the accuracy of the present invention in the changed and unchanged regions is significantly better than that of the GTMSiam method. The overall accuracy is increased by more than 2 percentage points compared with GTMSiam, the Kappa value is increased by 15 percentage points, the F1 value is increased by 23 percentage points, the accuracy P is increased by about 2 percentage points, and the recall rate R is increased by 30 percentage points.

[0086] Although the GTMSiam method also uses a deep learning architecture to extract hyperspectral features, its model performance is relatively poor because it fails to fully consider the differential features and collaborative features in dual-temporal change detection. The present invention effectively improves the performance of hyperspectral change detection through a differential feature extraction method driven by dynamic entropy. This method assigns different weights to the differential features according to the fluctuation degree of the hyperspectral differential features, so as to extract more accurate differential information. In addition, through the interactive aggregation method of the collaborative enhancement network, while extracting the collaborative features of the dual-temporal hyperspectral images, the differential features are further strengthened, making the detection effect more significant.

[0087] Figures 3a to 3c are the change detection result maps obtained by different methods, where Figure 3a is the change ground truth map of the River dataset, Figure 3b is the GTMSiam change detection result map, Figure 3b is the change detection result map of the present invention. In the figure, the misdetected pixels and missed detected pixels in the change region are represented by red and blue respectively.

[0088] From Figure 3b it can be seen that in the GTMSiam detection result, there are many misclassifications between the changed region and the unchanged region. Compared with GTMSiam, the change detection result of the method proposed by the present invention is more accurate.

[0089] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts between the various embodiments, reference can be made to each other.

Claims

1. A remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network, characterized in that: The following steps are involved: Step 1: split the dual-phase hyperspectral data into multiple samples and divide them into training sets and test sets; Step 2: Build a deep learning network and extract the difference features between the dual-phase hyperspectral images through a dynamic entropy-driven difference extractor; The collaborative features of the enhanced dual-phase hyperspectral images are generated by the temporal collaborative enhancer; in step 2, the calculation method for extracting the difference features between the dual-phase hyperspectral images is specifically as follows: first, the dual-phase hyperspectral image blocks are subtracted, and then put into the dynamic entropy driven difference extractor for feature extraction, and the features are flattened after passing through a linear mapping layer, and position encoding is added; in step 2, the probability Represents the ratio of each pixel value in the entire channel, calculated The entropy value of is used to obtain the attention map of the change information; through the residual link, a deeper feature learning is carried out, and the learned features are sent to the channel enhancement module; In step 2, the dual-phase hyperspectral image is temporally enhanced by the temporal collaborative enhancer, and the feature focusing of the temporal collaborative enhancement is The following steps are included: the features of the dual-phase hyperspectral image are obtained through a linear layer , , , adopts the structure of feature concentration, pushes the adjacent vectors to their nearest features, and adjusts the direction of each query with the direction of the key features through norm-preserving mapping, so that similar , To be close, not similar , The pairs are separated to obtain the synergistic features after dual-phase hyperspectral image enhancement; Step 3: Input the difference features and collaborative features into the multi-scale feature optimization and integrator for feature screening and fusion processing; send the fused features into the classifier to generate detection results and save the trained model parameters; Step 4: Input the training set samples into the network to obtain the prediction results, then calculate the overall accuracy of the dynamic entropy driven and collaborative enhancement network predictors, and update the network parameters; repeat steps 2 to 4 until the number of iterations reaches the required number; Step 5: Input the test set samples into the final deep learning network and output the sample results after detection.

2. The remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network according to claim 1 is characterized in that: In step three, each element of the difference feature and collaborative feature obtained in step two is input into the multi-scale feature optimization and integrator, passes through the forgetting layer and the input layer in turn, and then the features of the forgetting layer and the input layer are fused.

Images