A remote sensing change detection method based on multi-scale feature mining and reflection mechanism

By embedding multi-scale feature mining and reflection neural network modules in remote sensing change detection neural network, the problems of low feature extraction efficiency and missed detection and missed detection in the existing technology are solved, and more efficient remote sensing change detection and refined result generation are achieved .

CN118334488BActive Publication Date: 2024-11-01CHUZHOU UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410558337.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-08
Publication Date
2024-11-01
Estimated Expiration
2044-05-08

AI Technical Summary

Technical Problem

The existing deep learning remote sensing change detection methods have limitations in feature extraction and training efficiency, and are prone to missed or missed to detect change areas that are difficult to detect.

Method used

Embed lightweight multi-scale feature mining modules and reflection neural network modules in remote sensing change detection neural networks, optimize context extraction capabilities through multi-scale feature mining, and generate refined change detection results through reflection mechanisms, focusing on missed detection and false detection areas .

Benefits of technology

The training efficiency and change detection accuracy of the model are improved, and effective mining of difficult-to-detect areas and generation of refined detection results are realized.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118334488B_ABST
    Figure CN118334488B_ABST
Patent Text Reader

Abstract

The present invention discloses a remote sensing change detection method based on multi-scale feature mining and reflection mechanism, which belongs to the field of remote sensing image processing technology and is applicable to the fields of land survey, urban building monitoring and ecological monitoring. The implementation steps are: 1. Construction of a multi-scale feature mining module (MSFE); 2. Construction of a remote sensing change detection neural network embedded with a multi-scale feature mining module; 3. Construction of a reflective neural network module; 4. Model training and result output. Compared with the existing remote sensing change detection method, the multi-scale feature mining module of the present invention can efficiently mine multi-scale feature information. In addition, the reflective neural network module is a difficult sample mining technology suitable for remote sensing change detection tasks, which can improve the detection result accuracy of the remote sensing change detection neural network.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing change detection method of multi-scale feature mining and reflection mechanism, which can be used in the fields of land survey, urban building monitoring, ecological monitoring, etc. Background Art

[0002] With the acceleration of urbanization, tasks such as urban land surveys, building monitoring, and ecological environment monitoring are becoming increasingly important. Traditional methods based on manual field surveys are not only costly but also inefficient.

[0003] The emergence of remote sensing change detection technology provides a new solution for field surveys. By analyzing remote sensing images taken at different time points, this technology can quickly identify changes in a specific area. Especially with the introduction of deep learning, the accuracy of remote sensing change detection technology has also been greatly improved.

[0004] Although deep learning technology has greatly improved the performance of feature extraction in remote sensing change detection, the current deep learning remote sensing change detection methods still have some limitations. For example, some techniques that optimize feature extraction capabilities, such as pyramid pooling and dense skip connections, are effective, but the training cost is high, which affects the efficiency of model training. In addition, missed detection or false detection of changed areas in remote sensing images is also a problem that needs to be solved. In order to improve the performance of change detection, the model needs to focus more on these difficult-to-detect areas. Summary of the invention

[0005] In order to solve the above problems, the present invention embeds a lightweight multi-scale feature mining (MSFE) module and a reflective neural network module on the basis of a remote sensing change detection neural network, thereby obtaining a remote sensing change detection method based on multi-scale feature mining and reflective mechanism. The lightweight multi-scale feature mining module can optimize the context extraction capability of remote sensing images and improve the training efficiency of the model at the same time. In addition, the reflective neural network module has the ability to generate refined change detection results from preliminary change detection results. Its joint training with the remote sensing change detection neural network can guide the remote sensing change detection neural network to focus on the change areas that are difficult to detect correctly, such as missed detection and false detection, to achieve a difficult sample mining, and enhance the change detection capability of the remote sensing change detection neural network.

[0006] The technical solution of the present invention is a remote sensing change detection method based on multi-scale feature mining and reflection mechanism, comprising the following steps:

[0007] Step 1: Construction of multi-scale feature mining module;

[0008] First, a resampling layer is constructed using the maximum pooling or bilinear interpolation algorithm, and multiple original feature maps are input into the resampling layer for resampling to obtain corresponding feature maps of the same size. Secondly, an efficient channel attention module (ECA) is constructed, and the above original feature maps are input into the efficient channel attention module to obtain the corresponding channel attention weighted value matrix. Subsequently, the corresponding channel attention weighted value matrix is ​​matrix-multiplied with the resampled feature map, and the feature map obtained by the above matrix multiplication is spliced ​​along the channel dimension. Finally, a convolution operation is performed on the spliced ​​result (the convolution kernel is 1*1), and the feature map obtained by the convolution operation is output. Since the original feature maps are extracted by multiple convolutional layers of the model, they have features of different scales (multi-scale).

[0009] Step 2: Construction of remote sensing change detection neural network embedded with multi-scale feature mining module;

[0010] A multi-layer encoder and a multi-layer decoder are set up to construct a remote sensing change detection neural network. The multi-layer encoder has two branches, which are used to extract the feature maps of the remote sensing images of phase 1 and phase 2 respectively. The multi-layer decoder also has two branches, which uses the feature maps extracted by the corresponding encoder to generate the mask of the changed area; the constructed multi-scale feature mining module (MSFE) is embedded in the above multi-layer encoder and multi-layer decoder respectively, and the remote sensing change detection neural network embedded with the multi-scale feature mining module is obtained. Using this network, the preliminary change detection result p1 can be generated.

[0011] Step 3, reflect on the construction of neural network modules;

[0012] First, the maximum index function (argmax) is applied to convert the preliminary change detection result p1 into a single-channel binary change mask p1', and the difference between p1' and the true value gt is calculated as the judgment basis for the difficult-to-detect area, and an attention weighted value matrix Wa is constructed; secondly, the multi-layer feature map output by the decoder in the remote sensing change detection neural network embedded with the multi-scale feature mining module in step 2 is fused to obtain the feature map f, and the attention mechanism is applied to construct an adaptive attention weighted value matrix Wb for the above fused feature map f; finally, based on the two attention weighted value matrices Wa and Wb obtained above, the above fused feature map f is recalibrated to obtain the feature map f', and the recalibrated result f' is input into the reflective decoder to obtain the refined change detection result p2.

[0013] Step 4: Model training and result output;

[0014] Based on the preliminary change detection result p1 and the refined change detection result p2 obtained in the above steps, the loss function values ​​are calculated respectively, and the optimizer updates the entire model according to the sum of the above two loss function values. Repeat steps 1 to 4 until the training is completed. After the training is completed, the refined change detection result p2 output by the model can be used as the final result.

[0015] Furthermore, the specific implementation of step 1 is as follows;

[0016] 1.1. First, a resampling layer is constructed by using a maximum pooling or bilinear interpolation algorithm, and multiple original feature maps (these original feature maps are extracted by the encoder phase 1 convolutional layers, the encoder phase 2 convolutional layers or the decoder fusion modules of the remote sensing change detection neural network model) are input into the resampling layer for resampling to obtain corresponding feature maps of the same size. That is, n original feature maps are input, and n corresponding new feature maps are generated through the resampling layer, and the sizes of these new feature maps are the same as the sizes of the specifically specified original feature maps;

[0017] 1.2. Secondly, the n original feature maps are input into the efficient channel attention (ECA) module to obtain the corresponding n channel attention weighted value matrix;

[0018] 1.3. Again, perform matrix multiplication operations on the n channel attention weighted value matrices obtained in 1.2 and the n new feature maps resampled in 1.1, and concatenate the feature maps obtained by matrix multiplication along the channel dimension;

[0019] 1.4. Finally, perform a convolution operation on the feature map obtained in 1.3 (the convolution kernel is 1*1), adjust its number of channels to the same as the number of channels of the specifically specified original feature map, and finally output the result.

[0020] Furthermore, the specific implementation of step 2 is as follows:

[0021] 2.1. First, the n-layer encoders of the two branches (Phase 1 and Phase 2) encode the remote sensing images of the two phases and extract two sets of feature maps at different levels, namely f11, f12, ..., f1n (the first phase) and f21, f22, ..., f2n (the second phase);

[0022] 2.2. Secondly, the feature maps f11, f12, ..., f1n and f21, f22, ..., f2n of different levels in the two phases are respectively input into two multi-scale feature mining modules (MSFE) that share the weights of the neural network, and feature information is further mined from the feature maps of different levels and the results f1 and f2 are output. f1 and f1n, f2 and f2n are spliced ​​along the channel dimension to enhance the expressiveness of the encoder. The above results are used as the input of the decoder, that is, as the first-layer feature maps df11 and df21 of the decoder;

[0023] 2.3. Again, the n-layer decoder is composed of multiple convolutional layers and upsampling layers, and there is a fusion module (FM) between the two branches of the same layer, which is responsible for extracting the difference features of the dual-phase feature map of the layer, and fusing them to generate feature maps f31, f32, ..., f3n with difference features. The multi-scale feature mining module further performs feature mining and output on f31, f32, ..., f3n to obtain the feature map f3. Among them, the specific operation of the fusion module is: first extract the difference features of the dual-phase feature map of the nth layer, and at the same time upsample the feature map output by the n-1th layer fusion module by 2 times, and then splice the extracted difference feature map with the above-mentioned upsampled feature map along the channel dimension as the output result of the nth layer fusion module;

[0024] 2.4. Finally, the convolution operation of the classification layer and the Sigmoid activation function are applied to the feature map f3 processed by the multi-scale feature mining module to obtain the preliminary change detection result p1.

[0025] Furthermore, the specific implementation of step 3 is as follows:

[0026] 3.1. First, the maximum index function (argmax) is applied to convert the preliminary change detection result p1 into a single-channel binary change mask p1', and the difference between p1' and the true value gt is calculated. According to the difference result, it is determined whether the area is correctly detected. The pixel value of the binary change mask p1' consists of 0 and 1, where 0 represents pixels in the unchanged area and 1 represents pixels in the changed area;

[0027] 3.2. Secondly, for those pixels in the changed area that are correctly detected, a weighted suppression value is set, and for those pixels in the changed area that are missed or misdetected, a weighted gain value is set. Based on this, an attention weighted value matrix Wa consisting of a weighted gain value and a weighted suppression value is constructed;

[0028] 3.3. Again, the n-layer features (f31, f32, ..., f3n) in the remote sensing change detection neural network decoder are fused, and these fused feature maps f are input into the channel attention mechanism and the spatial attention mechanism to obtain two attention weighted value matrices, and the two attention weighted value matrices are fused by matrix multiplication to generate an adaptive attention weighted value matrix Wb;

[0029] 3.4. Finally, based on the Wa and Wb attention weighted value matrices, the fused feature map f is recalibrated to obtain the feature map f', and the feature map f' is input into the reflective decoder to obtain the refined change detection result p2.

[0030] Furthermore, the specific implementation of step 4 is as follows:

[0031] 4.1. Set the optimizer and learning rate for the entire neural network;

[0032] 4.2. Based on the obtained preliminary change detection result p1 and the refined change detection result p2, the loss function values ​​are calculated respectively;

[0033] 4.3. The overall loss function value is obtained by summing the loss function value obtained by p1 and the loss function value obtained by p2;

[0034] 4.4. The optimizer updates the entire model based on the overall loss function value and repeats steps 1, 2, 3 and 4 until the overall loss function value converges and stabilizes, and the model training is completed. After the training is completed, the refined change detection result p2 output by the model can be used as the final change detection result.

[0035] The present invention proposes a new remote sensing change detection method, which jointly trains a remote sensing change detection neural network and a reflective neural network module, and on this basis realizes a multi-scale feature mining and a difficult sample mining, which has the following beneficial effects:

[0036] 1. Mining multi-scale contexts and improving model training efficiency;

[0037] 2. The reflective neural network module has the ability to generate refined change detection results from preliminary change detection results, which can improve the accuracy of remote sensing change detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a technical flow chart of the whole invention;

[0039] Figure 2 Construct a flow chart for the model of the present invention;

[0040] Figure 3A multi-scale feature mining module (MSFE) constructed for the present invention;

[0041] Figure 4 The remote sensing change detection neural network architecture is constructed by embedding a multi-scale feature mining module in the present invention;

[0042] Figure 5 A Fusion Module (FM) constructed for the present invention;

[0043] Figure 6 The reflective neural network module and related processes constructed for the present invention. DETAILED DESCRIPTION

[0044] In order to more clearly illustrate the technical solution implemented by the present invention, the technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0045] The present invention is a remote sensing change detection method based on multi-scale feature mining and reflection mechanism; first, multi-layer feature encoding is performed on dual-phase remote sensing images, and multi-scale feature mining is performed on this basis to capture feature information of different scales in remote sensing images; secondly, differential feature information is decoded from the result of the above feature encoding through upsampling and multi-layer fusion modules. In this step, the multi-layer fusion module plays a key role, which is responsible for integrating dual-phase features and extracting differential information features reflecting changes from them; then, multi-scale feature mining is continued for the multi-layer features extracted by the multi-layer fusion module, and they are mapped to the corresponding category space through the classification layer to generate preliminary change detection results; then, the preliminary change detection results are further refined by using the reflection neural network module to obtain refined change detection results; finally, the loss function value is calculated based on the preliminary change detection results and the refined change detection results, and the neural network is updated through the optimizer to achieve the effect of iterative improvement.

[0046] like Figure 1 As shown, a remote sensing change detection method based on multi-scale feature mining and reflection mechanism includes the following steps:

[0047] (1) Construction of multi-scale feature mining module

[0048] 1.1, such as Figure 2As shown, by constructing a resampling layer using the maximum pooling or bilinear interpolation algorithm, the size of the original feature maps extracted by the encoder phase 1 convolutional layers (when i=1), the encoder phase 2 convolutional layers (when i=2), or the decoder fusion modules (when i=3) of the remote sensing change detection neural network model is resampled to be consistent. The model encoder phase 1 and encoder phase 2 have 4 convolutional layers, and the decoder has 4 fusion modules, which extract the corresponding 4 feature maps f11, f12, ... f13, f21, f22, ... f23, f31, f32, ... f33, respectively. The size of the corresponding 4 feature maps should be adjusted to the same size as the specific feature map (f13 or f23 or f33) through the resampling layer.

[0049] When embedding the encoder or decoder, the resampling layer of the multi-scale feature mining module adopts maximum pooling or bilinear interpolation respectively. Assume Resampling layer representing the multi-scale feature mining module, represents the maximum pooling operation, represents the bilinear interpolation algorithm, and Represents the height and width of the target size, Represents the input feature map, represents the encoder, Represents the decoder, the specific calculation formula is as follows:

[0050]

[0051] 1.2. Secondly, the four original feature maps fi1, fi2, ..., fi4 are input into the efficient channel attention (ECA) module to obtain the corresponding channel attention weighted value matrices wi1', wi2', ..., wi4';

[0052] 1.3. Again, perform matrix multiplication operation on the four channel attention weighted value matrices obtained in the above steps with fi1', fi2', ..., fi4' to obtain fi1'', fi2'', ..., fi4'', and concatenate the two sets of feature maps obtained by matrix multiplication along the channel dimension to obtain fic'';

[0053] 1.4. Finally, perform a convolution operation on f1c'' (the convolution kernel is 1*1), adjust its number of channels to the same as the number of channels of the original feature map fi4, and output the result fi. Concatenate fi and fi4 along the channel dimension to enhance the expressiveness of the encoder.

[0054] (2) Construction of remote sensing change detection neural network embedded with multi-scale feature mining module

[0055] 2.1. First, Figure 3 As shown in the figure, the 4-layer encoders of the two branches (phase 1 and phase 2) encode the remote sensing images of the two phases and extract two sets of feature maps at different levels, namely f11, f12, ..., f14 (the first phase) and f21, f22, ..., f24 (the second phase). The above 4-layer encoder contains 4 layers of ResNet downsampling layers (using the configuration of ResNet18, including convolution, skip connection, and ReLU);

[0056] 2.2. Input the feature maps f11, f12, ..., f14 and f21, f22, ..., f24 of different levels of the two phases into two multi-scale feature mining modules (MSFE) that share neural network weights, and output the results f1 and f2. Concatenate f1 and f14, and f2 and f24 along the channel dimension to enhance the expressiveness of the encoder. The above results are used as the input of the decoder, that is, as the first-layer feature maps df11 and df21 of the decoder;

[0057] 2.3. Again, the 4-layer decoder consists of multiple convolutional layers and upsampling layers (the size of the convolution kernel is 3*3, and each convolution layer performs two consecutive convolutions. Upsampling uses bilinear interpolation), and there is a fusion module (FM) between the two branches of the same layer, which is responsible for extracting the difference features of the dual-phase feature map of the layer. And fusion generates feature maps f31, f32, ..., f34 with difference features. Similar to the steps in 2.2, the multi-scale feature mining module further mines and outputs f31, f32, ..., f34 to obtain the feature map f3. As shown in Figure 4 As shown in the figure, the specific operation of the fusion module is: first calculate the absolute value subtraction result of the dual-phase feature maps df1n and df2n of the nth layer, that is, f = |df1n - df2n|. Next, the channel attention mechanism and the spatial attention mechanism are used to calculate the attention weighted matrices in the channel and spatial dimensions, which are denoted as w1 and w2 respectively. By performing matrix multiplication operations on the absolute value subtraction result f and the above two weighted matrices w1 and w2, we obtain the feature map f3n. At the same time, the feature map output by the n-1th layer fusion module is upsampled by 2 times, and then the feature map f3n is concatenated with the above upsampled feature map along the channel dimension to obtain a new feature map as the final output result f3n of the nth layer fusion module. Assume that Indicates The feature map output by the layer fusion module, Indicates The 2-fold upsampling feature of the feature map output by the layer fusion module, the above steps can be expressed as:

[0058]

[0059] 2.4. Finally, the convolution operation of the classification layer and the Sigmoid activation function are applied to the feature map f3 processed by the multi-scale feature mining module to obtain the preliminary change detection result p1.

[0060] (3) Reflection on the construction of neural network modules

[0061] 3.1. First, the maximum index function (argmax) is applied to convert the preliminary change detection result p1 into a single-channel binary change mask p1', and the difference between p1' and the true value gt is calculated (subtracted by element position). Whether the difference result is 0 can be used to determine whether the area is correctly detected. The pixel value of the binary change mask p1' consists of 0 and 1, where 0 represents pixels in the unchanged area and 1 represents pixels in the changed area;

[0062] 3.2. Secondly, for those pixels in the changed area that are correctly detected (pixels whose difference result is 0), a weighted suppression value is set, and for those pixels in the changed area that are missed or misdetected (pixels whose difference result is not 0), a weighted gain value is set. Based on this, an attention weighted value matrix Wa consisting of weighted gain values ​​and weighted suppression values ​​is constructed;

[0063] 3.3. Again, the four-layer features (f31, f32, ..., f34) in the remote sensing change detection neural network decoder are fused, and these fused feature maps f are input into the channel attention mechanism and the spatial attention mechanism to obtain two attention weighted value matrices w1 and w2, and the two attention weighted value matrices are fused by matrix multiplication to generate an adaptive attention weighted value matrix Wb;

[0064] 3.4. Finally, based on the Wa and Wb attention weighted value matrices, the fused feature map f is recalibrated to obtain the feature map f', and these feature maps f' are input into the reflective decoder (including multi-layer convolution, upsampling, Sigmoid activation function, and embedded with a multi-scale feature mining module) to obtain the refined change detection result p2.

[0065] (4) Model training and result output

[0066] 4.1. Set the optimizer (AdamW) and learning rate (0.002) for the entire neural network.

[0067] 4.2. Based on the obtained preliminary change detection result p1 and the refined change detection result p2, the loss function values ​​are calculated respectively;

[0068] 4.3. The overall loss function value is obtained by summing the loss function value obtained by p1 and the loss function value obtained by p2;

[0069] 4.4. The optimizer updates the entire model based on the overall loss function value and repeats steps 1, 2, 3, and 4 until the overall loss function value converges to a stable value and the model training is completed. After the training is completed, the refined change detection result p2 output by the model can be used as the final change detection result.

[0070] The main features of the present invention are as follows:

[0071] (1) Based on a remote sensing change detection neural network, a lightweight multi-scale feature mining module and a reflective neural network module are embedded to optimize the context extraction capability of remote sensing images and improve the training efficiency of the model;

[0072] (2) The present invention trains the remote sensing change detection neural network and the reflective neural network module to realize a difficult sample mining technology suitable for remote sensing change detection tasks, which can improve the change detection result accuracy of the remote sensing change detection neural network;

[0073] (3) After training, the reflective neural network module has the ability to generate refined change detection results from the preliminary change detection results. The user can simply draw a simple mask at the approximate location, and the reflective neural network module can generate the corresponding refined change mask based on it.

[0074] The specific embodiments mentioned herein are only used to illustrate the core concepts of the present invention. It is entirely possible for professionals working in this technical field to make various adjustments, supplements or adopt equivalent alternatives to these embodiments, and these changes will not change the essence of the invention or exceed the limits defined by the attached claims.

Claims

1. A remote sensing change detection method based on multi-scale feature mining and reflection mechanism, characterized in that: The steps include: Step 1: Construction of multi-scale feature mining module; First, a resampling layer is constructed using a maximum pooling or bilinear interpolation algorithm, and multiple original feature maps are input into the resampling layer for resampling to obtain corresponding feature maps of the same size; Secondly, an efficient channel attention module ECA is constructed, and the above original feature map is input into the efficient channel attention module to obtain the corresponding channel attention weighted value matrix; then, the corresponding channel attention weighted value matrix is ​​matrix multiplied with the resampled feature map, and the feature map obtained by the above matrix multiplication is spliced ​​along the channel dimension; finally, a convolution operation is performed on the spliced ​​result, where the convolution kernel is 1*1, and the feature map obtained by the convolution operation is output; Step 2: Construction of remote sensing change detection neural network embedded with multi-scale feature mining module; A multi-layer encoder and a multi-layer decoder are set to construct a remote sensing change detection neural network. The multi-layer encoder has two branches, which are used to extract the feature maps of the remote sensing images of phase 1 and phase 2 respectively; the multi-layer decoder has two branches, which uses the feature maps extracted by the corresponding encoder to generate a mask of the changed area; the constructed multi-scale feature mining module MSFE is embedded in the above multi-layer encoder and multi-layer decoder respectively, and the remote sensing change detection neural network embedded with the multi-scale feature mining module is obtained. Using this network, a preliminary change detection result p1 is generated; Step 3, reflect on the construction of neural network modules; First, the maximum index function argmax is applied to convert the preliminary change detection result p1 into a single-channel binary change mask p1', and the difference between p1' and the true value gt is calculated as the judgment basis for the difficult-to-detect area, and an attention weighted value matrix Wa is constructed; secondly, the multi-layer feature map output by the decoder in the remote sensing change detection neural network embedded in the multi-scale feature mining module in step 2 is fused to obtain the feature map f, and the attention mechanism is applied to construct an adaptive attention weighted value matrix Wb for the above fused feature map f; finally, based on the two attention weighted value matrices Wa and Wb obtained above, the above fused feature map f is recalibrated to obtain the feature map f', and the recalibrated result f' is input into the reflective decoder to obtain the refined change detection result p2; Step 4: Model training and result output; Based on the preliminary change detection result p1 and the refined change detection result p2 obtained in the above steps, the loss function values ​​are calculated respectively, and the optimizer updates the entire model according to the sum of the above two loss function values; Repeat steps 1 to 4 until the training is completed. After the training is completed, the refined change detection result p2 output by the model is used as the final result.

2. The remote sensing change detection method based on multi-scale feature mining and reflection mechanism according to claim 1 is characterized in that: The step 1 is specifically as follows: 1.

1. A resampling layer is constructed by adopting a maximum pooling or bilinear interpolation algorithm, and multiple original feature maps are input into the resampling layer for resampling to obtain corresponding feature maps of the same size, that is, n original feature maps are input, and n corresponding new feature maps are generated through the resampling layer, and the size of the new feature map is the same as the size of the specifically specified original feature map; the multiple original feature maps are extracted by the encoder phase 1 convolution layers, the encoder phase 2 convolution layers or the decoder fusion modules of the remote sensing change detection neural network model; 1.

2. Input n original feature maps into the efficient channel attention ECA module to obtain the corresponding n channel attention weighted value matrix; 1.

3. Perform matrix multiplication operations on the n channel attention weighted value matrices obtained in 1.2 and the n new feature maps resampled in 1.1, and concatenate the feature maps obtained by matrix multiplication along the channel dimension; 1.

4. Perform a convolution operation on the feature map obtained by splicing in 1.3, where the convolution kernel is 1*1, and adjust its number of channels to the same as the number of channels of the specifically specified original feature map, and finally output the result.

3. The remote sensing change detection method based on multi-scale feature mining and reflection mechanism according to claim 1 is characterized in that: The step 2 is specifically as follows: 2.

1. The n-layer encoders of two branches, phase 1 and phase 2, encode the remote sensing images of the two phases and extract two sets of feature maps at different levels, namely f11, f12, ..., f1n and f21, f22, ..., f2n; 2.

2. The feature maps f11, f12, ..., f1n and f21, f22, ..., f2n at different levels of the two phases are respectively input into two multi-scale feature mining modules MSFE that share neural network weights, and feature information is further mined from the feature maps at different levels and the results f1 and f2 are output. f1 and f1n, f2 and f2n are concatenated along the channel dimension respectively, and the above results are used as the input of the decoder, that is, as the first-layer feature maps df11 and df21 of the decoder; 2.

3. The n-layer decoder is composed of multiple convolutional layers and upsampling layers, and there is a fusion module FM between the two branches of the same layer, which is responsible for extracting the difference features of the dual-phase feature map of the layer and fusing them to generate feature maps f31, f32, ..., f3n with difference features; the multi-scale feature mining module further mines and outputs f31, f32, ..., f3n to obtain the feature map f3, wherein the specific operation of the fusion module is: firstly extract the difference features of the dual-phase feature map of the n-th layer, and at the same time upsample the feature map output by the n-1-th layer fusion module by 2 times, and then splice the extracted difference feature map with the above-mentioned upsampled feature map along the channel dimension as the output result of the n-th layer fusion module; 2.

4. Apply the convolution operation of the classification layer and the Sigmoid activation function to the feature map f3 obtained by the multi-scale feature mining module to obtain the preliminary change detection result p1.

4. The remote sensing change detection method based on multi-scale feature mining and reflection mechanism according to claim 1 is characterized in that: The step 3 is as follows: 3.

1. Apply the maximum index function argmax to convert the preliminary change detection result p1 into a single-channel binary change mask p1', and calculate the difference between p1' and the true value gt. According to the difference result, determine whether the area is correctly detected. The pixel value of the binary change mask p1' consists of 0 and 1. The value 0 represents the pixels in the unchanged area, and the value 1 represents the pixels in the changed area. 3.

2. For those pixels in the changed area that are correctly detected, a weighted suppression value is set, and for those pixels in the changed area that are missed or misdetected, a weighted gain value is set. Based on this, an attention weighted value matrix Wa consisting of a weighted gain value and a weighted suppression value is constructed; 3.

3. Fuse the n-layer features f31, f32, ..., f3n in the remote sensing change detection neural network decoder, and input these fused feature maps f into the channel attention mechanism and the spatial attention mechanism to obtain two attention weighted value matrices, and use matrix multiplication to fuse the two attention weighted value matrices to generate an adaptive attention weighted value matrix Wb; 3.

4. Based on the Wa and Wb attention weighted value matrices, the fused feature map f is recalibrated to obtain the feature map f', and the feature map f' is input into the reflective decoder to obtain the refined change detection result p2.

5. The remote sensing change detection method based on multi-scale feature mining and reflection mechanism according to claim 1 is characterized in that: The step 4 is specifically as follows: 4.

1. Set the optimizer and learning rate for the entire neural network; 4.

2. Based on the obtained preliminary change detection result p1 and the refined change detection result p2, the loss function values ​​are calculated respectively; 4.

3. The overall loss function value is obtained by summing the loss function value obtained by p1 and the loss function value obtained by p2; 4.

4. The optimizer updates the entire model based on the overall loss function value and repeats steps 1, 2, 3 and 4 until the overall loss function value converges and stabilizes. The model training is completed. After the training is completed, the refined change detection result p2 output by the model is used as the final change detection result.

Citation Information

Patent Citations

  • Semantic segmentation method and device for polyp image

    CN113781489A

  • Infrared and visible light image fusion method based on end-to-end attention network

    CN115170915A

  • Multi-scale feature fusion classroom behavior detection method and system

    CN116229319A

  • Double-temporal remote sensing image change detection network based on attention and multiple scales

    CN116343052A

  • Self-attention-based multi-scale coding and decoding essential image decomposition network, method and application

    CN116597273A