A deep neural network change detection method suitable for dual temporal phase remote sensing images

Abstract

The application discloses a kind of deep neural network change detection methods suitable for dual temporal remote sensing image. Mainly include the following steps: in feature extraction layer, construct pyramid spatiotemporal attention module introducing coordinate attention;According to the pyramid spatiotemporal attention module obtained in previous step, construct up-sampling layer;According to the pyramid spatiotemporal attention module obtained in first step and the up-sampling layer obtained in second step, construct network output layer, obtain a kind of change detection deep neural network model suitable for dual temporal remote sensing image;With training data set, introduce deep supervision mechanism design loss function combination to train the deep neural network model constructed in third step;Dual temporal remote sensing image is input to the deep neural network model trained in step four and obtains change detection feature image.The application is good in the change detection of remote sensing image change, and it is an effective deep neural network change detection method of dual temporal remote sensing image.

Description

Technical Field

[0001] The present invention relates to image change detection technology, and in particular to a deep neural network change detection method applicable to dual-temporal remote sensing images, belonging to the field of change detection. Background Art

[0002] Remote sensing change detection is the process of identifying significant differences between multi-temporal remote sensing images. Significant differences often depend on specific applications, such as urbanization monitoring, disaster assessment, and environmental monitoring. Automated change detection techniques have recently garnered significant attention, facilitating the development of remote sensing applications.

[0003] With the development of deep learning, numerous change detection network models have been proposed, and a growing number of change detection models for remote sensing images have emerged. Due to its end-to-end network structure, deep learning directly outputs corresponding feature maps after feature extraction from a bi-temporal image. This not only simplifies the complexity of change detection research but also significantly improves detection accuracy. Chao-Peng Chen et al. proposed a new network, SARAS-Net, with several mechanisms. It uses a relation-aware module before feature subtraction and a scale-aware module and a cross-transformer module after subtraction. Finally, the cross-transformer module integrates multi-level features, focusing on spatial information, making it easier to separate foreground and background, and thus reducing false positives. Experiments have shown that SARAS-Net achieves good results on various datasets. The BIT network, proposed by Hao Chen et al., is a Transformer-based method that uses a Transformer encoder-decoder network to enhance the contextual information of ConvNet features through semantic labeling, and then performs feature differentiation to generate a change map. Zhuo Zheng et al. proposed a Single Temporal Supervision Learning (STAR) model that uses unpaired labeled images to train a high-accuracy change detector and generalizes it to real-world bi-temporal images. Furthermore, to evaluate the effectiveness of STAR, they designed a simple yet effective change detector, ChangeStar, which can reuse any deep semantic segmentation architecture via the ChangeMixin module. Comprehensive experimental results show that ChangeStar significantly outperforms baselines under single-temporal supervision and achieves excellent performance under dual-temporal supervision.

[0004] STANet, a network proposed by Hao Chen et al., integrates a novel self-attention module for change detection. STANet processes feature maps at both temporal and spatial scales, calculating attention weights for different regions, ultimately achieving excellent results. However, in the final feature map processing, STANet applies Euclidean distances to dual temporal distance feature maps and then performs linear interpolation. This results in blurred gaps between image regions with changes, resulting in a high false alarm rate. While the location of changed pixels is identified, detection accuracy is lost. While the PAM design calculates attention weights for different image regions, it is computationally expensive. Furthermore, the feature maps prior to PAM are not effectively utilized. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a deep neural network change detection method applicable to dual-temporal remote sensing images to solve the problems existing in the above-mentioned prior art.

[0006] The present invention proposes a deep neural network change detection method for dual-temporal remote sensing images, which mainly includes the following steps:

[0007] (1) In the feature extraction layer, a pyramid spatiotemporal attention module (PCA) is constructed that introduces coordinate attention (CA);

[0008] (2) constructing an upsampling layer based on the feature extraction layer obtained in step (1);

[0009] (3) constructing a network output layer based on the feature extraction layer obtained in step (1) and the upsampling layer obtained in step (2), thereby obtaining a deep neural network change detection method suitable for dual-temporal remote sensing images;

[0010] (4) Using the training data set, introduce a deep supervision mechanism to design a loss function combination to train the deep neural network model constructed in step 3;

[0011] (5) The dual-phase remote sensing image is input into the deep neural network model trained in step 4 to obtain the change detection feature image. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is the coordinate attention module.

[0013] Figure 2 It is an upsampling module.

[0014] Figure 3 It is the deep neural network model proposed in this invention.

[0015] Figure 4 This is a diagram showing the effect of the STANet method on remote sensing image change detection.

[0016] Figure 5 This is a diagram showing the effect of the present invention on remote sensing image detection.

[0017] Figure 6 is the real label map. DETAILED DESCRIPTION

[0018] The present invention will be further described below in conjunction with the accompanying drawings:

[0019] A deep neural network change detection method for dual-temporal remote sensing images can be divided into the following steps:

[0020] (1) In the feature extraction layer, a pyramid spatiotemporal attention module (PCA) is constructed that introduces coordinate attention (CA);

[0021] (2) constructing an upsampling layer based on the feature extraction layer obtained in step (1);

[0022] (3) constructing a network output layer based on the feature extraction layer obtained in step (1) and the upsampling layer obtained in step (2), thereby obtaining a deep neural network change detection method suitable for dual-temporal remote sensing images;

[0023] (4) Using the training data set, introduce a deep supervision mechanism to design a loss function combination to train the deep neural network model constructed in step 3;

[0024] (5) The dual-temporal remote sensing image is input into the deep neural network model trained in step 4 to obtain the change detection feature image.

[0025] Specifically, if Figure 1 As shown in Figure 1, in step (1), after the ResNet network extracts the features, the feature map is input into the spatiotemporal attention module. CA extracts two feature maps with directional features in the vertical and horizontal directions from the feature map of the image through two one-dimensional global pooling layers. Afterwards, these two directional image features are encoded separately through a convolutional layer to obtain two attention features in different directions. These two attention features are attention feature maps generated by capturing the long-term dependency of image features along the two directions.

[0026] The global pooling of image features by the pooling layer is generally used to encode global spatial information, but this will lose some spatial information in the features. Therefore, the CA attention module uses precise spatial position information to capture the long-term dependencies in the image features, decomposing the global pooling into two one-dimensional encoding operations. Given the input image features, CA uses the pooling kernel (H, 1) and (1, W) in two different spatial ranges from the vertical and horizontal directions to perform channel encoding on the feature map. Therefore, the output of c channels and height h is as shown in Formula 1:

[0027]

[0028] Similarly, the output of the cth channel at width w is as shown in Formula 2:

[0029]

[0030] The above two formulas encode features in the vertical and horizontal directions, respectively, aggregating image features to produce a pair of directional attention feature maps. Furthermore, these two transformation formulas enable the CA module to effectively capture long-term dependencies in a single direction, preserving the precise spatial coordinate information of that direction. This allows CA to more accurately and efficiently calculate regions of interest within image features.

[0031] Formulas 1 and 2 extract global spatial information, increasing the receptive field and obtaining more accurate spatial coordinate encoding information. CA then performs coordinate attention generation to effectively utilize the obtained encoding information. After calculating using Formulas 1 and 2, the resulting feature map is input into a shared 1×1 convolutional layer, i.e., the F1 transform function, resulting in Formula 3:

[0032] f=δ(F1([z h ,z w ])) (3)

[0033] Among them, [·,·] means that the two parts of feature z h and z w Connect along the spatial dimension, δ is the Sigmoid nonlinear activation function, generating It is the spatial encoding feature in the vertical and horizontal directions. Among them, r is used to control the block size reduction rate in the SE module. After that, the obtained features are decomposed along the spatial dimension to obtain two independent tensors f h and f w . Input it into another two 1×1 convolutional layers F h and F w , these two convolutional layers are used to transform f h and f w Converted to features with the same number of channels as the input features, as shown in Formula 4 and Formula 5:

[0034] g h =σ(F h (f h )) (4)

[0035] g w =σ(F w (f w )) (5)

[0036] Among them, δ is still the Sigmoid nonlinear activation function. The adjustment of the reduction rate r can control the number of channels of f, which is conducive to reducing the computational overhead of the model. The output feature g h and g w This is the required attention score. Finally, the output y of the coordinate attention mechanism CA is c It can be expressed as formula 6:

[0037]

[0038] Figure 3 The architecture of PCA is given in . Given a bi-temporal feature map X (1) , X (2) ∈R C×H×W , before inputting the PCA module, the feature map is interpolated to generate X (3) ∈R C×H×W , then stack the three feature maps into a feature tensor X∈R C×H×W×3 Then there are four parallel branches; each branch divides the feature tensor into s×s subregions, where s∈S, S∈{1,2,4,8}, defining four pyramid scales. In the branch of scale s, each region is defined as Four CAs are used for each of these four branches. In each pyramid branch, all sub-regions R s,i,j Apply CA to generate the updated residual feature tensor Y s ∈R C×H×W×3 Then, these feature tensors Y s (s∈S) are stacked and fed into the convolutional layer to generate the final feature tensor Y∈R C×H×W×3 Finally, we add Y to X to get a new tensor Z∈R C ×H×W×3 .

[0039] like Figure 2 In step (2), in order to improve the accuracy of change detection, an upsampling module is constructed, and each convolution block contains two convolution layers, where Cov i , i∈(1,3,5), are two convolution layers with a kernel of 3×3, a stride of 1, and a padding of 1, followed by batch normalization (BatchNorm2d) and ReLU activation function. The feature map is processed by Cov i , i∈(1,3,5) will be added to the input feature map and then input the next Cov i, i∈(2, 4, 6), which represents a convolutional layer with a 1×1 kernel, stride 1, and padding 1, followed by batch normalization (BatchNorm2d) and ReLU activation. The three convolutional blocks extract distance feature maps of different scales and channels, stacking them and feeding them into Cov7, a convolutional layer with a 1×1 kernel and stride 1. Finally, the upsampling module outputs a distance feature map of the same size as the input image.

[0040] like Figure 3 As shown, according to the feature extraction layer obtained in step (1) and the upsampling layer obtained in step (2), a network output layer is constructed to obtain a deep neural network change detection method suitable for dual-temporal remote sensing images.

[0041] like Figure 3 As shown in Figure 1, the training process is divided into two branches after the feature extraction layer for remote sensing image change detection. The number of changed samples and unchanged samples varies greatly. In many cases, the changed pixels only account for a small fraction of all pixels, which can introduce some bias to the network during training. To reduce the impact of class imbalance, the batch-balanced contrast loss (BCL) is used and a deep supervision mechanism is added. The calculation formula of this loss function is as follows:

[0042]

[0043] loss=loss_1+α(1-epoch / n_epoch)×loss_2 (10)

[0044] Among them, D * Represents the Euclidean distance feature map between two feature maps, M * It is a binary mapping of the true label, b represents the number of training sets, i and j represent height and width. u 、n c They represent the number of unchanged pixel pairs and the number of changed pixel pairs respectively.

[0045] like Figure 3 As shown in the figure, loss_1 represents the loss value calculated using Equation 9 after directly upsampling the features obtained from the feature extraction layer without passing through the pyramid spatiotemporal attention module (Loss1 in the figure). loss_2 represents the loss value calculated by the change detection features of the entire network (Loss2 in the figure). epoch represents the current number of iterations reached, n_epoch represents the total number of training iterations, and α is the weight parameter that controls the attenuation.

[0046] In the step (5), the dual-temporal remote sensing image is input into the deep neural network model trained in the step (4) to obtain a change feature map.

[0047] The present invention proposes a deep neural network extraction method suitable for core image particles, which uses a spatiotemporal attention module to obtain spatiotemporal attention features and an upsampling module to improve detection accuracy. It accelerates the convergence of the network through a deep supervision mechanism, effectively improving the accuracy of remote sensing image change detection. Figure 4-Figure 5 The STANet method and the effect diagram of the present invention on remote sensing image change detection are given. Figure 6 is the real label map.

[0048] At the same time, the present invention uses precision (Pr), recall (Recall, Re), and F1 to evaluate the training effect of the algorithm on the remote sensing image dataset. The results are compared with the STANet method, and are shown in Table 1. It can be seen that the present invention improves on all indicators of remote sensing image change detection compared to the STANet method. Among them, the improvement in F1 score is particularly significant. This is because the present invention greatly improves the accuracy of remote sensing image change detection and effectively reduces the false alarm rate.

[0049] Table 1 Comparative experiment

[0050]

[0051] In summary, compared to the STANet method, the present invention has obvious advantages in remote sensing image change detection and objective evaluation. Therefore, the present invention is an effective dual-temporal remote sensing image change detection method.

[0052] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A deep neural network change detection method for dual-temporal remote sensing images, characterized by The following steps are involved: Step 1: In the feature extraction layer, construct a pyramid spatiotemporal attention module PCA that introduces coordinate attention CA, as shown in the following formula: f=δ(F1([z h ,z w ])) (3) g h =σ(F h (f h )) (4) g w =σ(F w (f w )) (5) in, represents the output of the c-th channel with a height of h, represents the output of the cth channel at width w, [·,·] represents the combination of the two features z h and z w Connect along the spatial dimension, δ is the Sigmoid nonlinear activation function, generating is the spatial encoding feature in the vertical and horizontal directions, r is the reduction rate, f h and f w The features are decomposed along the spatial dimension to obtain two independent tensors, F h and F w Represents two 1×1 convolutional layers, σ is still the Sigmoid nonlinear activation function, g h and g w That is the required attention score. Specifically, given the dual-time feature map X (1) , X (2) ∈R C×H×W , before entering the PCA module, the feature map is interpolated to generate X (3) ∈R C×H×W , and then stack the three feature maps into a feature tensor X∈R C×H×W×3 , then there are four parallel branches; each branch divides the feature tensor into s×s subregions, where s∈S, S∈{1, 2, 4, 8}, and four pyramid scales are defined. In the branch of scale s, each region is defined as Four CAs are used for each of these four branches. In each pyramid branch, all sub-regions R s，i，j Apply CA to generate the updated residual feature tensor Y s ∈R C×H×W×3 , then, these feature tensors Y s (s∈S) are stacked and fed into the convolutional layer to generate the final feature tensor Y∈R C×H×W×3 Finally, we add Y to X to get a new tensor Z∈R C×H×W×3 ; Step 2: Construct an upsampling layer based on the pyramid spatiotemporal attention module obtained in step 1; Step 3: Based on the pyramid spatiotemporal attention module obtained in step 1 and the upsampling layer obtained in step 2, a network output layer is constructed to obtain a deep neural network model suitable for change detection in dual-temporal remote sensing images. Step 4: Use the training dataset and introduce a deep supervision mechanism to design a loss function combination to train the deep neural network model constructed in step 3. The calculation formula of the loss function is as follows: loss=loss_1+α(1-epoch / n_epoch)×loss_2 (10) Among them, D * Represents the Euclidean distance feature map between two feature maps, M * It is a binary mapping of the true label, b represents the number of training sets, i and j represent height and width, and n u 、n c They represent the number of unchanged pixel pairs and the number of changed pixel pairs respectively. loss_1 represents the loss value calculated by using formula (9) by directly inputting the upsampling feature obtained by the feature extraction layer without passing through the pyramid spatiotemporal attention module to obtain the change detection feature. loss_2 represents the loss value calculated by the change detection feature of the entire network. epoch represents the number of iterations currently reached, n_epoch represents the total number of training iterations, and α is the weight parameter that controls the attenuation. Step 5: Input the dual-temporal remote sensing image into the deep neural network model trained in step 4 to obtain the change detection feature image.

2. A deep neural network change detection method for dual-temporal remote sensing images according to claim 1, characterized in that In step 1, a coordinate attention module is added to calculate the spatiotemporal attention scores of the feature map after the feature extraction layer.

3. The deep neural network change detection method for dual-temporal remote sensing images according to claim 1 is characterized in that The upsampling layer constructed in step 2 upsamples the obtained bi-temporal feature map using the residual network to obtain an accurate change detection feature map.

4. The deep neural network change detection method for dual-temporal remote sensing images according to claim 1 is characterized in that The deep neural network model proposed in step 3 improves the change detection accuracy by increasing the weight of the changed area through the spatiotemporal attention module, deep supervision mechanism and upsampling operation, and finally obtains an accurate change detection map.

5. The deep neural network change detection method for dual-temporal remote sensing images according to claim 1 is characterized in that The loss function combination designed in step 4 uses a deep supervision mechanism to solve the problems of slow convergence and gradient vanishing in deep neural networks. It adds a branching strategy to the hidden layer of the deep neural network to supervise the entire network and play a role in regularization, thereby promoting faster convergence of the network.

Images