Remote sensing image change detection method based on dynamic entropy driving and collaborative enhancement network

Abstract

The invention relates to a remote sensing image change detection method based on a dynamic entropy driving and collaborative enhancement network, which is suitable for change detection of a hyperspectral remote sensing image, and adopts a dynamic entropy driving difference feature extraction method to extract difference features of a dual-time image; wherein the dynamic entropy driving method is designed according to hyperspectral difference characteristics, different values are given according to the fluctuation quantity of the difference characteristics, and the hyperspectral change detection performance is improved; an interactive aggregation mode is adopted for extraction of the dual-time image cooperation features based on the cooperation enhancement network, the features of the dual-time hyperspectral image are utilized, the difference features are enhanced while the cooperation features are extracted, and the method is superior to an existing hyperspectral change detection method.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image processing and change detection, and in particular relates to a remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network. Background Art

[0002] Change detection aims to accurately identify changes in the earth's surface or specific targets by comparing geospatial data acquired at different times. This technology is crucial in monitoring surface dynamics and has been widely used in environmental monitoring, urban planning, disaster response, agricultural monitoring and forest protection. Hyperspectral imaging technology can capture subtle differences in objects or surface materials over a wide spectral range, thereby significantly improving the accuracy and sensitivity of change detection.

[0003] Deep learning constructs a deep network model with a hierarchical structure by simulating the hierarchical working mode of the human visual system, and has demonstrated excellent performance in the field of hyperspectral image change detection. In 2020, Seydi et al. proposed a novel end-to-end convolutional neural network that combines 1D, 2D, and 3D convolutions to optimize feature extraction, thereby achieving excellent detection results. This is mainly due to the fact that the CNN model can effectively extract spatial features through local connections and weight sharing, while greatly reducing network training parameters. Subsequently, many scholars have proposed a series of improved models based on CNN. In order to deal with the problem of single scale, Li et al. and Luo et al. proposed multi-scale convolution methods in 2022 and 2023, respectively, to extract features of dual-phase hyperspectral images and fuse shallow and deep features to detect small changes in the image, achieving significant results. In order to solve the local receptive field limitation in CNN, in 2022, Chen et al. introduced the Transformer architecture into the field of change detection, thereby capturing long-term dependencies in the image and further improving the detection performance. Song et al. used three Transformer network architectures to learn the spatiotemporal spectral information of hyperspectral images and achieved good performance.

[0004] Although the above deep learning networks can automatically learn spatial and spectral features from data, it can be seen from these research works that there are still significant potentials and challenges in extracting change features from bi-temporal images and better capturing change information. First, CNN-based methods face difficulties in extracting long-term dependency information in images, mainly because the local receptive field of CNN limits its ability to capture global features. Although CNN performs well in spatial feature extraction through local connections and weight sharing, its performance may be limited when dealing with large-scale and complex changes. Second, although Transformer-based methods can model global dependencies through self-attention mechanisms, they still have shortcomings in fully utilizing the key-value pairs of bi-temporal images to obtain change information. Although the Transformer architecture is good at processing sequence data, its sensitivity to time series and its ability to capture subtle changes between bi-temporal images may not be sufficient in the specific change detection task. Finally, although the current change detection methods using attention mechanisms perform well in other tasks, these methods often lack optimized designs specifically for change detection. Existing attention mechanisms mainly focus on global or region-specific feature extraction without being specifically tailored for change detection tasks, which may fail to fully capture subtle but important change information in images.

[0005] Therefore, a new remote sensing image change detection method is urgently needed to solve the above problems. Summary of the invention

[0006] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network.

[0007] This remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network includes the following steps:

[0008] Step 1: split the dual-phase hyperspectral data into multiple samples and divide them into training sets and test sets;

[0009] Step 2: Build a deep learning network, extract the difference features between the dual-phase hyperspectral images through a dynamic entropy-driven difference extractor, and generate enhanced collaborative features of the dual-phase hyperspectral images through a temporal collaborative enhancer.

[0010] Step 3: Input the difference features and collaborative features into the multi-scale feature optimization and integrator for feature screening and fusion processing; send the fused features into the classifier to generate detection results and save the trained model parameters;

[0011] Step 4: Input the training set samples into the network to obtain the prediction results, then calculate the overall accuracy of the dynamic entropy driven and collaborative enhancement network predictors, and update the network parameters; repeat steps 2 to 4 until the number of iterations reaches the required number;

[0012] Step 5: Input the test set samples into the final deep feature extraction network model and output the sample results after detection.

[0013] Preferably, in step 2, the calculation method for extracting the difference features between the dual-phase hyperspectral images is specifically as follows: first, the dual-phase hyperspectral image blocks are subtracted, and the images are put into a dynamic entropy-driven difference extractor for feature extraction, the features are flattened after passing through a linear mapping layer, and position encoding is added.

[0014] As a preferred method, in step 2, the probability Represents the ratio of each pixel value in the entire channel, calculated The entropy value of the feature vector is used to obtain the attention map of the change information; through the residual link, a deeper feature learning is carried out, and the learned features are sent to the channel enhancement module.

[0015] Preferably, in step 2, the dual-phase hyperspectral image is temporally collaboratively enhanced by a temporal collaborative enhancer, and the feature focusing of the temporal collaborative enhancement includes the following steps: the features of the dual-phase hyperspectral image are passed through a linear layer to obtain Q, K, and V, and the feature concentration structure is adopted to push adjacent vectors to their nearest features. The direction of each query is adjusted with the direction of the key feature through a norm-preserving mapping to make similar Q, V pairs close and dissimilar Q, V pairs separate, so as to obtain the collaborative features after the dual-phase hyperspectral image is enhanced.

[0016] Preferably, in step three, each element of the difference feature and collaborative feature obtained in step two is input into the multi-scale feature optimization and integrator, passes through the forgetting layer and the input layer in sequence, and then the features of the forgetting layer and the input layer are fused.

[0017] The beneficial effects of the present invention are:

[0018] 1) The remote sensing image change detection model based on dynamic entropy driving and collaborative enhancement network established in the present invention is suitable for change detection of hyperspectral remote sensing images, and adopts a dynamic entropy driven difference feature extraction method to extract the difference features of dual-time images; wherein the dynamic entropy driven method is designed based on the hyperspectral difference features, and different values ​​are assigned according to the magnitude of the fluctuation of the difference features, thereby improving the performance of hyperspectral change detection.

[0019] 2) The extraction of collaborative features of dual-time images based on the collaborative enhancement network in the present invention adopts an interactive aggregation method, which utilizes the characteristics of dual-time hyperspectral images to extract collaborative features while enhancing the difference features. After experiments on a public hyperspectral change detection dataset, based on the currently available information, the change detection method proposed in the present invention is superior to the existing hyperspectral change detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0021] Figure 2a This is a pseudo-color image of the moment when three bands are selected from 198 original bands for synthesis;

[0022] Figure 2b This is a pseudo-color image of the moment when three bands are selected from 198 original bands for synthesis;

[0023] Figure 2c is the ground truth map for change detection;

[0024] Figure 3a This is the true value map of the River dataset;

[0025] Figure 3b This is the GTMSiam change detection result diagram;

[0026] Figure 3c This is a diagram of the change detection results of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described below in conjunction with embodiments. The description of the following embodiments is only used to help understand the present invention. It should be noted that for ordinary persons in the art, without departing from the principle of the present invention, the present invention can also be modified in some ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0028] Embodiment 1

[0029] As an example, Figure 1 As shown, this remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network includes the following steps:

[0030] Step 1: Data preprocessing: first, divide the dual-phase hyperspectral data into multiple pixel cubes, each pixel cube represents a sample. In this embodiment, the data format of the initial image is a three-dimensional data matrix in .mat format. We first divide the data cube, take a 7x7 cube, and form a pixel cube. Use the overall information of the pixel cube to replace the information of the central pixel for processing. krepresents the spectral information of the k-th pixel cube, Represents the spectral information of the pixel at the kth pixel cube (i, j).

[0031] Then, the manually labeled samples are divided into the training set, and the unlabeled samples are divided into the test set. In this embodiment, 1% of the samples are randomly selected from all the preprocessed data samples as the training set for model training; the remaining 99% of the samples are used as the test set to evaluate the performance of the model.

[0032] Step 2: Build a deep learning network consisting of a dynamic entropy-driven difference extractor and a temporal collaborative enhancer.

[0033] The dynamic entropy driven difference extractor extracts difference features by calculating the difference between the dual-phase hyperspectral images. The temporal synergy enhancer uses the dual-phase input data to perform synchronous enhancement and generate enhanced synergy features.

[0034] The dynamic entropy driven difference extractor combines information entropy with channel enhancement mechanism, and the temporal synergistic enhancer combines feature focusing mechanism with synergistic enhancement mechanism;

[0035] Step 3: Input the difference features and collaborative features obtained in step 2 into the multi-scale feature optimization and integrator for feature screening and fusion processing. Then, the fused features are sent to the classifier to generate the detection results and save the trained model parameters;

[0036] Step 4: Fix the trained network parameters, input the samples in step 1 into the network to obtain the prediction results, and increase the number of iterations by 1. Next, compare the predicted value with the corresponding true value, calculate the cross entropy loss to evaluate the performance of the classifier, and calculate the overall accuracy of the dynamic entropy driven and collaboratively enhanced network predictor after this iteration, that is, the OA value. If the number of iterations reaches the preset maximum number, proceed to the next step; otherwise, return to step 2 and update the network parameters to the parameters after this iteration;

[0037] Step 5: Input the test set samples into the final deep feature extraction network model and output the sample results after detection.

[0038] Embodiment 2

[0039] As another embodiment, this embodiment 2 proposes, on the basis of embodiment 1, a more specific remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network:

[0040] In the step 2, based on the calculation method of extracting difference features driven by dynamic entropy, the dual-time image hyperspectral image blocks are first subtracted and then put into the dynamic entropy driven difference extractor. The feature extraction method of the dynamic entropy driven difference extractor is: Assuming that the input feature of the dynamic entropy driven difference extractor is, it is flattened after a linear mapping layer and position encoding is added. The specific calculation formula is as follows:

[0041] D′=F(BN(DW))+E pos (1)

[0042] Where W represents the linear layer parameters, F represents the flattening operation, and E pos Indicates the addition of relative position encoding. Subsequently, a probability-based method is used to represent the proportion of each pixel value in the entire channel. The specific calculation formula is as follows:

[0043]

[0044] Afterwards, calculate The entropy value is used to obtain the attention map of the change information. The specific calculation formula is as follows:

[0045]

[0046] Where C represents the dimension of the channel. Then, through the residual link, deeper features are learned while retaining the original input information. The specific calculation formula is as follows:

[0047]

[0048] Finally, the learned features are sent to the channel enhancement module. The specific formula is as follows:

[0049]

[0050] in, represents a 1×1 convolution kernel, and G represents an activation function.

[0051] It should be noted that the parts in this embodiment that are the same or similar to those in the first embodiment can be referenced to each other and will not be described in detail in this application.

[0052] Embodiment 3

[0053] As another embodiment, this embodiment 3 proposes a more specific remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network on the basis of embodiments 1 and 2. In step 2, the feature focusing mechanism and collaborative enhancement mechanism of the temporal collaborative enhancer are specifically as follows:

[0054] The feature focusing mechanism in the temporal collaborative reinforcement in step 2 mainly includes the following steps. For the input feature x, Q, K, and V are first obtained through a linear layer. Different from the traditional dot product attention, this embodiment adopts a feature-focused structure to push adjacent vectors to their nearest features to highlight similar features. The specific calculation formula is as follows:

[0055] Sim(Q,K)=FRL(Q)·FRL(K) T (7)

[0056] Among them, FRL is to pass the input features through a layer of activation function, and then adjust the direction of each query and the direction of the key features through the norm-preserving mapping, so that similar query key pairs are closer and dissimilar query key pairs are separated. The specific calculation formula of the norm-preserving mapping is as follows:

[0057]

[0058] Where m **p It means that each pixel in m is multiplied by p. Then the modified attention feature Sim(Q,K) is combined with W. The specific calculation formula is as follows:

[0059]

[0060] Where DWC stands for depthwise separable convolution.

[0061] Furthermore, the collaborative enhancement mechanism of the dual-time image in the time series collaborative enhancement in step 2 mainly includes the following steps. Assuming that the two input features are and respectively, firstly, they are respectively subjected to the linear mapping layer to obtain W1, Q1, V1, W2, Q2, V2. Then, feature interaction is performed to obtain collaborative features. The specific calculation formula is as follows:

[0062]

[0063] Through the above interaction, the collaborative features of the dual-time images can be extracted while enhancing their difference information.

[0064] It should be noted that the parts in this embodiment that are the same or similar to those in Embodiments 1 and 2 can be referenced to each other and will not be described in detail in this application.

[0065] Embodiment 4

[0066] As another embodiment, this embodiment 4 proposes, on the basis of embodiments 1 to 3, a more specific remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network:

[0067] The multi-scale feature optimization and integration mechanism in step 3 mainly includes the following contents: for a given intermediate feature A, each element is first taken out and arranged in a row. For each input, these features will be screened and fused one by one through the multi-scale feature optimization and integrator. For an intermediate k-th step, the feature will first pass through a forgetting layer. The specific calculation formula is as follows:

[0068] f k =σ(W fa a k +W fh h k-1 +b f ) (12)

[0069] Among them, W fa and W fh are coefficient matrices, σ represents the activation function, b f Represents the deviation term. It is then sent to the input layer, which is mainly responsible for deciding whether to add new information to the memory unit. The specific calculation formula is as follows:

[0070] i k =δ(W ia a k +W ih h k-1 +b i ) (13)

[0071] Among them, W ia and W ih are coefficient matrices, b i Represents the bias term of the input layer. Then the features of the forgetting layer and the input layer are fused. The specific formula is as follows:

[0072]

[0073] Among them, ⊙ represents pointwise multiplication, W ca and W ch Represents the weight matrix, tanh represents the activation function, and then all the information is fused to get the output result. The specific formula is as follows:

[0074] h k =δ(W oa a k +W oh h k +b o )tanh(c k ) (16)

[0075] Among them, W oa and W oh represents the coefficient matrix of the output layer, b oRepresents the bias term.

[0076] It should be noted that the parts in this embodiment that are the same or similar to those in Embodiments 1 to 3 can be referenced to each other and will not be described in detail in this application.

[0077] Embodiment 5

[0078] As another embodiment, based on Embodiments 1 to 4, Embodiment 5 proposes to use this remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network, and conducts experiments on the River hyperspectral change detection dataset:

[0079] The River hyperspectral dataset is one of the widely used public datasets in the field of hyperspectral change detection.

[0080] In this example, the River dataset selects a river near Jiangsu Province, China. The data was acquired by the EO-1 Hyperion sensor on May 3, 2013 and December 31, 2013. Each image has a resolution of 463×241 pixels, contains 198 spectral bands, has a spatial resolution of 30m, and a spectral resolution of 10nm. This dataset is mainly used to study the detection of material changes in rivers.

[0081] River hyperspectral data such as Figure 2a to Figure 2c As shown, Figure 2a This is a pseudo-color image when three bands are selected from the 198 original bands for synthesis. Figure 2b This is a pseudo-color image when three bands are selected from the 198 original bands for synthesis. Figure 2c is the ground-truth map for change detection.

[0082] In this experiment, 1% of the training samples were randomly selected for training, and the remaining samples were used as test sets. In order to verify the effectiveness of the algorithm of the present invention, in this embodiment, the method proposed by the present invention is compared with the latest method for hyperspectral image change detection based on multi-scale network of gated transmission GTMSiam. The change detection results are shown in Table 1.

[0083] Table 1: Comparison of classification results of 5 labeled samples per class on SA data

[0084]

[0085] It can be seen from the change detection result table that the accuracy of the proposed method in the changed and unchanged areas is significantly better than that of the GTMSiam method. The overall accuracy is improved by more than 2 percentage points compared with GTMSiam, the Kappa value is improved by 15 percentage points, the F1 value is increased by 23 percentage points, the precision P is improved by about 2 percentage points, and the recall R is improved by 30 percentage points.

[0086] Although the GTMSiam method also uses a deep learning architecture to extract hyperspectral features, its model performance is relatively poor because it fails to fully consider the difference features and collaborative features in dual-time image change detection. The present invention effectively improves the performance of hyperspectral change detection through a dynamic entropy-driven difference feature extraction method. This method assigns different weights to the difference features according to the degree of fluctuation of the hyperspectral difference features, thereby extracting more accurate difference information. In addition, through the interactive aggregation method of the collaborative enhancement network, while extracting the collaborative features of the dual-time hyperspectral images, the difference features are further strengthened, making the detection effect more significant.

[0087] Figure 3a to Figure 3c The change detection results obtained by different methods are shown in Figure 2. Figure 3a is the true value graph of the River dataset. Figure 3b This is the GTMSiam change detection result diagram. Figure 3b This is a change detection result diagram of the present invention. In the diagram, falsely detected pixels and missed pixels in the change area are represented by red and blue, respectively.

[0088] from Figure 3b It can be seen that in the GTMSiam detection results, there are many misclassifications between the changed areas and the unchanged areas. Compared with GTMSiam, the change detection results of the method proposed in the present invention are more accurate.

[0089] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

Claims

1. A remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network, characterized in that: The following steps are involved: Step 1: split the dual-phase hyperspectral data into multiple samples and divide them into training sets and test sets; Step 2: Build a deep learning network, extract the difference features between the dual-phase hyperspectral images through a dynamic entropy-driven difference extractor, and generate enhanced collaborative features of the dual-phase hyperspectral images through a temporal collaborative enhancer. Step 3: Input the difference features and collaborative features into the multi-scale feature optimization and integrator for feature screening and fusion processing; send the fused features into the classifier to generate detection results and save the trained model parameters; Step 4: Input the training set samples into the network to obtain the prediction results, then calculate the overall accuracy of the dynamic entropy driven and collaborative enhancement network predictors, and update the network parameters; repeat steps 2 to 4 until the number of iterations reaches the required number; Step 5: Input the test set samples into the final deep feature extraction network model and output the sample results after detection.

2. The remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network according to claim 1 is characterized in that: In step 2, the calculation method for extracting the difference features between the dual-phase hyperspectral images is specifically as follows: first, the dual-phase hyperspectral image blocks are subtracted, and the images are put into a dynamic entropy-driven difference extractor for feature extraction, the features are flattened after passing through a linear mapping layer, and position encoding is added.

3. The remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network according to claim 2 is characterized in that: In step 2, we use the probability Represents the ratio of each pixel value in the entire channel, calculated The entropy value of the feature vector is used to obtain the attention map of the change information; through the residual link, a deeper feature learning is carried out, and the learned features are sent to the channel enhancement module.

4. The remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network according to claim 1 is characterized in that: In step 2, the dual-phase hyperspectral image is temporally enhanced by the temporal collaborative enhancer, and the feature focusing of the temporal collaborative enhancement is The method comprises the following steps: the features of the dual-phase hyperspectral image are passed through a linear layer to obtain Q, K, and V; the structure of feature concentration is adopted to push adjacent vectors to their nearest features; the direction of each query is adjusted with the direction of the key feature through a norm-preserving mapping to make similar Q and V pairs close and dissimilar Q and V pairs separate, thereby obtaining the collaborative features after the dual-phase hyperspectral image enhancement.

5. The remote sensing image change detection method based on dynamic entropy drive and collaborative enhancement network according to claim 1 is characterized in that: In step three, each element of the difference feature and collaborative feature obtained in step two is input into the multi-scale feature optimization and integrator, passes through the forgetting layer and the input layer in turn, and then the features of the forgetting layer and the input layer are fused.

Images