A remote sensing image change detection method based on a two-stage feature pyramid network

Abstract

The application discloses a remote sensing image change detection method based on a two-stage feature pyramid network, which comprises the following steps: respectively extracting the encoding features of two images of the same place at different times to obtain two image encoding feature maps; performing similarity processing and self-attention spatial enhancement processing on the encoding feature maps to obtain initial difference attention feature maps; performing first-stage feature pyramid processing and second-stage feature pyramid processing to obtain re-optimized change feature maps; performing deep supervision and optimization through a loss function to obtain an optimized two-stage feature pyramid network model; and performing detection based on the optimized two-stage feature pyramid network model to obtain a detection result. The application can well perform change detection in a high-resolution remote sensing scene, can also achieve good results in the face of a class imbalance and a too large target scale difference remote sensing scene, and improves various measurement indexes of change detection in the high-resolution remote sensing scene.

Description

Technical Field

[0001] This invention relates to the field of change detection technology in optical remote sensing image processing, and specifically to a method for detecting changes in remote sensing images based on a two-stage feature pyramid network. Background Technology

[0002] The purpose of change detection in optical remote sensing images is to detect differences between optical remote sensing images from different periods and obtain information on changes in targets of interest on the Earth's surface. It is an important research direction in remote sensing image processing. Change detection is a crucial technology for obtaining information on the current state of land use and for locational and global detection. It has wide applications in rapidly developing modern society, such as urban sprawl detection, disaster assessment, environmental monitoring, climate monitoring, resource management, evidence collection in public interest litigation, and urban planning. With the development of remote sensing technology, Earth observation has acquired multiple capabilities, including multi-platform, multi-resolution, and multi-temporal capabilities. Simultaneously, the accumulation of data in recent years has formed a natural remote sensing big data platform. Traditional remote sensing manual interpretation and supervised classification algorithms, due to their low efficiency and enormous financial and material costs, are no longer sufficient to meet the application and information service needs of massive amounts of remote sensing data. They exhibit drawbacks such as poor interpretation accuracy, low processing timeliness, poor robustness, and difficulty in handling long-term, high-intensity interpretation tasks. Therefore, research on remote sensing big data is urgently needed to address the problems existing in current technologies. Most existing solutions rely on intelligent interpretation algorithms to efficiently transform remote sensing big data into data service information, reducing reliance on manual interpretation and promoting rapid and automated evaluation of remote sensing images.

[0003] Traditional change detection methods, such as Change Vector Analysis (CVA), Principal Component Analysis (PCA), and Multivariate Change Detection (MAD), primarily detect changes by extracting spectral information from remote sensing images. However, these methods require manual setting of decision boundaries during the decision-making stage, which demands significant expert knowledge and is inefficient. Machine learning algorithms, such as Support Vector Machines, Decision Trees, and K-Nearest Neighbors, can learn an automatic decision-making model from a large number of labeled samples. Therefore, machine learning algorithms are widely used in remote sensing image change detection. However, these methods heavily rely on handcrafted features, making it difficult to capture high-level feature representations, resulting in poor performance in change detection. In recent years, deep learning has made significant breakthroughs in image analysis, natural language processing, and other fields. With improvements in image processors and increased data volumes, deep learning methods using deep networks for feature encoding have greatly reduced the need for expert knowledge and handcrafted features, and have been widely applied to remote sensing tasks such as change detection, achieving remarkable results. The characteristics of deep feature representation and nonlinear modeling make deep learning-based methods more suitable for feature extraction from complex images, attracting attention in the field of remote sensing image change detection.

[0004] In recent years, semantic segmentation networks based on the FCN architecture have been proposed and proven effective for pixel-level change detection. Many deep learning-based remote sensing image change detection methods have been proposed, outperforming traditional methods. These methods can be divided into two categories: metric learning-based methods and semantic segmentation-based methods. The former typically uses Siamese networks with different feature extractors to extract bi-temporal features, followed by metric learning to obtain the change map. The latter uses a semantic segmentation model to solve the change detection problem, feeding channel-dimensional concatenated bi-temporal images into a semantic segmentation network to predict change regions. Although the above methods have achieved good change detection results, they all ignore the spurious change interference caused by the differences in spatial and semantic information of features at different scales, thus affecting the performance of remote sensing image change detection. Furthermore, these models often have complex encoders and simple decoders, failing to fully utilize the multi-scale features extracted by the encoder. Therefore, there is an urgent need to research methods to eliminate spurious change interference caused by semantic and spatial differences in features at different scales, as well as to design feature pyramid networks suitable for change detection that fully utilize the multi-scale information of the encoder. Summary of the Invention

[0005] In view of this, the present invention provides a remote sensing image change detection method based on a two-stage feature pyramid network, comprising: extracting coded features from a first image and a second image respectively to obtain a first image coded feature map and a second image coded feature map; wherein the first image and the second image are images located at the same location at different time points, the first image coded feature map includes four coded feature maps of different scales, namely, a first image level 0 coded feature map, a first image level 1 coded feature map, a first image level 2 coded feature map, and a first image level 3 coded feature map, and the second image coded feature map includes four different scales. The first image encoding feature map and the second image encoding feature map are respectively the 0th level encoding feature map, the 1st level encoding feature map, the 2nd level encoding feature map, and the 3rd level encoding feature map of the second image. Similarity processing is performed on the first image encoding feature map and the second image encoding feature map to obtain an initial difference feature map. The initial difference feature map includes four initial difference feature maps at different scales: the 0th level initial difference feature map, the 1st level initial difference feature map, the 2nd level initial difference feature map, and the 3rd level initial difference feature map. The 3rd level initial difference feature map is subjected to self-attention space enhancement processing to obtain an initial difference attention space enhancement processing. Attention feature map; the initial difference feature map at level 0, level 1, level 2, and the initial difference attention feature map are processed by a first-stage feature pyramid to obtain an initial change feature map; wherein, the initial change feature map includes three levels of initial change feature maps, namely the initial change feature map at level 0, level 1, and level 2; the initial change feature map is then processed by a second-stage feature pyramid to obtain a re-optimized change feature map; wherein, the re-optimized change feature map includes three levels of re-optimized change feature maps, namely the initial change feature map at level 0, level 1, and level 2; The system comprises a first-level re-optimized change feature map, a second-level re-optimized change feature map, and a third-level re-optimized change feature map. Based on the re-optimized change feature maps, it performs change prediction on the first image and the second image. Deep supervision and optimization are performed based on a loss function to obtain an optimized two-stage feature pyramid network model. The optimized two-stage feature pyramid network model is then used to detect the first and second images to be detected, obtaining detection results indicating the changes in the first and second images, where the first and second images are images located at the same location but at different times.

[0006] Further, the similarity processing of the first image encoded feature map and the second image encoded feature map includes: the similarity processing is calculated using the difference method.

[0007] F i =abs(Fi1 -F i2 ), i = 0, 1, 2, 3

[0008] Among them, F i1 F represents the i-th level encoded feature map of the first image. i2 F represents the i-th level encoded feature map of the second image. i This represents the initial difference feature map of level i.

[0009] Further, the self-attention spatial enhancement processing of the third-level initial difference feature map includes: filtering the third-level initial difference feature map to obtain the filtered result of the third-level initial difference feature map; and generating an initial difference attention map based on the filtered result.

[0010] A = softmax(K) T Q)

[0011] F a =VA

[0012] Where A represents the attention map, K represents the query feature map, Q represents the query feature map, T represents the matrix transpose, V represents the representation feature map, and F represents the representation feature map. a This represents the initial difference attention map.

[0013] Further, the first-stage feature pyramid processing of the Level 0 initial difference feature map, the Level 1 initial difference feature map, the Level 2 initial difference feature map, and the initial difference attention feature map includes: generating a Level 2 initial change feature map by performing Level 2 dual attention processing based on the initial difference attention feature map and the Level 2 initial difference feature map; generating a Level 1 initial change feature map by performing Level 1 dual attention processing based on the Level 2 initial change feature map and the Level 1 initial difference feature map; and generating a Level 0 initial change feature map by performing Level 0 dual attention processing based on the Level 1 initial change feature map and the Level 0 initial difference feature map; wherein the Level 2 dual attention processing, the Level 1 dual attention processing, and the Level 0 dual attention processing all include dual attention processing.

[0014] Further, the dual attention processing includes: performing compression activation processing on the first input data and the second input data in the channel dimension to generate a channel attention map; performing fusion processing on the channel attention map to generate a hybrid channel attention map; filtering the first input data in the channel based on the hybrid channel attention map to generate a channel feature map; performing fusion processing on the second input data and the channel feature map to obtain a hybrid feature map; performing compression activation on the hybrid feature map in the spatial dimension to obtain a spatial hybrid attention map; using the spatial hybrid attention map to perform spatial information enhancement on the second input data to generate a spatial feature map; and performing fusion processing on the channel feature map and the spatial feature map to generate an initial change feature map.

[0015] The second-stage feature pyramid processing of the initial change feature map includes: generating a second-level re-optimized change feature map by performing a second-level filtering fusion process based on the initial difference attention feature map and the second-level initial change feature map; generating a first-level re-optimized change feature map by performing a first-level filtering fusion process based on the second-level re-optimized change feature map and the first-level initial change feature map; and generating a first-level re-optimized change feature map by performing a first-level filtering fusion process based on the first-level re-optimized change feature map and the first-level initial change feature map.

[0016] Furthermore, the expression for the second-level filtering and fusion processing is as follows: The expression for the first-level filtering and fusion processing is as follows: The expression for the 0th-level filter fusion process is as follows: Among them, F r,2 F represents the feature map of the second-level re-optimization change. r,1 F represents the feature map of the first-level re-optimization change. r,0 F represents the feature map of the 0th level re-optimization change. R,2 F represents the initial change feature map of the second level. R,1 F represents the initial change feature map of the first level. R,0 F represents the initial change feature map of level 0. a This represents the initial differential attention feature map.

[0017] Further, using the reoptimized change feature map to predict changes in the first image and the second image includes: filtering the second-level reoptimized change feature map to obtain a filtered second-level reoptimized change feature map; and enlarging the filtered second-level reoptimized change feature map to the size of the first image to obtain a second-level change prediction map; filtering the first-level reoptimized change feature map to obtain a filtered first-level reoptimized change feature map; and enlarging the filtered first-level reoptimized change feature map to the size of the first image to obtain a first-level change prediction map; and filtering the zero-level reoptimized change feature map to obtain a filtered zero-level reoptimized change feature map; and enlarging the filtered zero-level reoptimized change feature map to the size of the first image to obtain a zero-level change prediction map.

[0018] Furthermore, the deep supervision and optimization based on the loss function includes: constructing a loss function L, wherein the loss function L includes a binary cross-entropy loss function Li. BCE and Dice loss function L Dice :

[0019] L = L Dice +L BCE

[0020]

[0021]

[0022] Where N is the number of pixels in the transformation graph, y n This represents the state of the nth pixel, where n is the index of the pixel position in the image, and y n =1 indicates a change, y n =0 indicates no change, p n Y represents the probability of a change in the nth pixel, and Y is the actual change image. It is a change prediction chart;

[0023] And based on the overall loss function Ls, deep supervision is performed on the level 2 change prediction map, the level 1 change prediction map, and the level 0 change prediction map. The expression of the overall loss function Ls is as follows:

[0024]

[0025] Among them, L i Let L represent the loss function L calculated between the predicted change map and the actual change map at level i.

[0026] This invention enables efficient and high-precision multi-temporal remote sensing change detection of high-resolution optical remote sensing images. It can alleviate the interference of pseudo-changes caused by semantic differences in features at different scales and the performance degradation of models caused by class imbalance and excessive scale differences. It can achieve good change detection results in remote sensing scenarios with class imbalance and excessive target scale differences, and can alleviate the interference of pseudo-changes caused by semantic differences in features at different scales.

[0027] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of the invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0028] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0029] Figure 1 This is a flowchart of the method of the present invention;

[0030] Figure 2 This is a flowchart of the first stage feature pyramid processing of the present invention;

[0031] Figure 3 This is a flowchart of the second-stage feature pyramid processing of the present invention;

[0032] Figure 4 This is a schematic diagram of the overall structure of the present invention;

[0033] Figure 5 This is a schematic diagram of the self-attention mechanism module structure. Detailed Implementation

[0034] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the invention. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the invention.

[0035] In the description of embodiments of the present invention, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0036] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and descriptions are considered exemplary in nature and not restrictive. There is no requirement for the order of the method steps described, as long as they are achievable, they are within the scope of protection of this invention.

[0037] This invention proposes a remote sensing image change detection method based on a two-stage feature pyramid network, which specifically includes:

[0038] S101: Extract the coding features of the first image and the second image respectively to obtain the coding feature map of the first image and the coding feature map of the second image. The first image and the second image are images at different times located at the same location.

[0039] In some embodiments, a Siamese coding network can be used to extract coded features from an image. The Siamese coding network can employ a hierarchical backbone network such as ResNet, ResNeXt, DenseNet, or Swin Transformer as the feature extractor, using the same network structure and parameters to extract multi-scale features from dual-temporal images. For example, if the input image size is 3×H×W, the extracted multi-scale feature sizes can be N×H / 4×W / 4, 2N×H / 8×W / 8, 4N×H / 16×W / 16, or 8N×H / 32×W / 32, and this invention does not limit these possibilities.

[0040] In some embodiments, the ResNet18 backbone network can be used as the overall framework of the two-stage feature pyramid network model. This network has a fast training speed, saves computational resources, prevents model overfitting, and has strong generalization ability. Combined with... Figure 4 The two-stage feature pyramid network model involved in this invention consists of a Siamese coding network, a self-attention module, a first-stage feature pyramid network, and a second-stage feature pyramid network. A first image T1 and a second image T2 are input into the two-stage feature pyramid network model, and after feature extraction, four scales of coded feature maps are obtained. Specifically, the first image coded feature map includes four different scales: a level 0 coded feature map, a level 1 coded feature map, a level 2 coded feature map, and a level 3 coded feature map; the second image coded feature map also includes four different scales: a level 0 coded feature map, a level 1 coded feature map, a level 2 coded feature map, and a level 3 coded feature map.

[0041] S102: Perform similarity processing on the first image coding feature map and the second image coding feature map to obtain an initial difference feature map. The initial difference feature map includes four initial difference feature maps at different scales, namely the level 0 initial difference feature map, the level 1 initial difference feature map, the level 2 initial difference feature map and the level 3 initial difference feature map.

[0042] In some embodiments, similarity processing can be performed in the form of difference, and the calculation process of the i-th level initial difference feature map is as follows:

[0043] F i =abs(F i1 -F i2 ), i = 0, 1, 2, 3

[0044] Among them, F i1 F represents the i-th level encoded feature map of the first image. i2 F represents the i-th level encoded feature map of the second image. i This represents the initial difference feature map of level i.

[0045] S103: Perform self-attention space enhancement processing on the third-level initial difference feature map to obtain an initial difference attention feature map.

[0046] In some embodiments, this step can be implemented in the following ways:

[0047] S1031: Filter the initial differential feature map of the third level to obtain the filtered result of the initial differential feature map of the third level, wherein the filtered result includes the query feature map Q, the queried feature map K, and the representation feature map V; and

[0048] S1032: Generate attention map A based on the query feature map Q and the queried feature map K; generate initial differential attention feature map F based on attention map A and the representation feature map V. a The calculation is as follows:

[0049] A = softmax(K) T Q)

[0050] F α =VA

[0051] S104: Perform first-stage feature pyramid processing on the 0th-level initial difference feature map, the 1st-level initial difference feature map, the 2nd-level initial difference feature map, and the initial difference attention feature map to obtain an initial change feature map. The initial change feature map includes three levels of initial change feature maps, namely the 0th-level initial change feature map, the 1st-level initial change feature map, and the 2nd-level initial change feature map.

[0052] In some embodiments, this step can be implemented in the following way:

[0053] S1041: Based on the initial difference attention feature map and the second-level initial difference feature map, a second-level initial change feature map is generated after second-level dual attention processing;

[0054] S1042: Based on the second-level initial change feature map and the first-level initial difference feature map, the first-level initial change feature map is generated after the first-level dual attention processing;

[0055] S1043: Based on the first-level initial change feature map and the second-level initial difference feature map, the second-level initial change feature map is generated after the second-level dual attention processing.

[0056] In some embodiments, this step may also be implemented in the following ways:

[0057] S1141: Simultaneously apply the initial differential attention feature map F a The channel attention map A of the initial difference attention feature map is generated by the F2 compression activation of the second-level initial difference feature map. a A a =σ(f 1×1 (GAP(F a ))); and the channel attention map A2 of the second-level initial difference feature map: A2=σ(f 1×1 (GAP(F2)));

[0058] S1142: Channel attention map A based on the initial differential attention feature map a The second-level mixed channel attention map A is generated from the channel attention map A2 of the second-level initial difference feature map. c,2 :

[0059] S1143: Using Level 2 Mixed Channel Attention Map A c,2 The second-level initial difference feature map F2 is filtered across channels to generate the second-level initial difference attention channel feature map F. c,2 :

[0060] S1144: Based on the initial differential attention feature map F a Compared with the initial differential attention channel feature map F of level 2 c,2 The second-level mixed feature map is obtained.

[0061] S1145: For the second-level mixed feature map Compressed activation is performed in the spatial dimension to obtain the second-level spatial hybrid attention map A. s,2 :

[0062] S1146: Using Level 2 Spatial Mixed Attention Map A s,2 For the initial differential attention feature map F a Spatial information enhancement is performed to generate the second-level initial differential attention spatial feature map F. s,2 :

[0063] S1147: Based on the second-level initial differential attention channel feature map F c,2 and the initial differential attention space feature map F s,2 Generate the second-level initial change feature map F R,2 :F R,2 =F s，2 +F c，2 ;

[0064] After obtaining the initial change feature map of level 2, further processing is carried out:

[0065] S1148: Characteristic map of initial change at level 2 F R,2 The channel attention map A of the first-level initial variation feature map is generated by F1 compression activation and the first-level initial variation feature map. R,2: A R,2 =σ(f 1×1 (GAP(F R,2 The channel attention map A1 of the first-level initial difference feature map: A1 = σ(f) 1×1 (GAP(F1)));

[0066] S1149: Channel attention map A based on the second-level initial change feature map R,2 The channel attention map A1 of the first-level initial difference feature map is used to generate the first-level mixed channel attention map A. c,1 :

[0067] S1150: Using Level 1 Mixed Channel Attention Map A c,1 The first-level initial difference feature map F1 is filtered across channels to generate the first-level initial difference attention channel feature map F. c,1 :

[0068] S1151: Based on the second-level initial change feature map F R,2 Compared with the initial differential attention channel feature map F of level 1 c,1 The first-level mixed feature map is obtained.

[0069] S1152: For the first-level mixed feature map Compressed activation is performed in the spatial dimension to obtain the first-level spatial hybrid attention map A. s,1 :

[0070] S1153: Using Level 1 Spatial Mixed Attention Map A s,1 For the second-level initial change feature map F R,2 Spatial information augmentation is performed to generate the first-level initial differential attention spatial feature map F. s,1 :

[0071] S1154: Based on the first-level initial differential attention channel feature map F c,1 And the first-level initial differential attention space feature map F s,1 Generate the first-level initial change feature map F R,1 :F R，1 =F s，1 +F c，1 ;

[0072] After obtaining the initial change feature map at level 1, further processing is performed:

[0073] S1155: Initial change feature map F of level 1 R,1 And the channel attention map A of the initial difference feature map of level 1 and the compression activation of the initial difference feature map of level 0 and the initial difference feature map of level 1. R,1 and the 0th class The initial differential feature map has a channel attention map A0, where: A R，1 =σ(f 1×1 (GAP(F R，1 ))), A0=σ(f 1×1 (GAP(F0)));

[0074] S1156: Channel attention map A based on the first-level initial change feature map R,1 The channel attention map A0 of the initial difference feature map at level 0 is used to generate the mixed channel attention map A at level 0. c,0 :

[0075] S1157: Finally, use the level 0 mixed channel attention map A c,0 The initial difference feature map F0 of level 0 is filtered on each channel to generate the initial difference attention channel feature map F0 of level 0. c,0 :

[0076] S1158: Based on the first-level initial change feature map F R,1 Compared with the initial differential attention channel feature map F at level 0 c,0 The 0th level mixed feature map is obtained.

[0077] S1159: For the level 0 mixed feature map Compressed activation is performed in the spatial dimension to obtain the level 0 spatial hybrid attention map A. s,0 :

[0078] S1160: Using Level 0 Spatial Mixed Attention Map A s,0 For the first-level initial change feature map F R,1 Spatial information augmentation is performed to generate an initial differential attention spatial feature map F at level 0. s,0 :

[0079] S1161: Based on the initial differential attention channel feature map F at level 0 c,0 And the initial differential attention space feature map F at level 0 s,0 Generate the initial change feature map F of level 0. R,0 :F R,0 =F s,0 +F c，0 ;

[0080] Where GAP(.) represents global average pooling, f 1×1 (.) represents a 1×1 convolution, σ(.) represents the sigmoid function, up(.) represents a 2x upsampling, and MAP(.) represents max pooling. This represents pixel-level multiplication. This indicates pixel-level addition.

[0081] S105: Perform a second-stage feature pyramid processing on the initial change feature map to obtain a re-optimized change feature map. The re-optimized change feature map includes three levels of re-optimized change feature maps, namely the level 0 re-optimized change feature map, the level 1 re-optimized change feature map, and the level 2 re-optimized change feature map.

[0082] In some embodiments, this step can be implemented in the following ways:

[0083] S1051: Based on the initial differential attention feature map F a Compared with the second-level initial change feature map F R,2 The second-level filtering and fusion process is used to generate the second-level re-optimized change feature map F. r,2 :

[0084] S1052: Based on the second-level re-optimization change feature map F r,2 Compared with the first-level initial change feature map F R.1 The first-level filtering and fusion process is used to generate the first-level re-optimized change feature map F. r.1 :

[0085] S1053: Based on the first-level re-optimization change feature map F r,1 Compared with the initial change feature map F of level 0 R.0 The 0th-level filtering and fusion process is used to generate the 0th-level re-optimized change feature map F. r,0 :

[0086] The two-stage feature pyramid processing described above can both be implemented using a decoder. This invention proposes a two-stage feature pyramid processing method that fully utilizes the multi-scale features extracted by the encoder network, which can effectively improve the accuracy of image detection.

[0087] S106: Use the re-optimized change feature map to predict changes in the first image and the second image respectively.

[0088] This step can be implemented in the following ways:

[0089] S1061: Filter the second-level re-optimization change feature map to obtain a filtered second-level re-optimization change feature map;

[0090] S1062: Enlarge the filtered second-level re-optimized change feature map to the size of the first image to obtain the second-level change prediction map;

[0091] S1063: Filter the first-level re-optimization change feature map to obtain the filtered first-level re-optimization change feature map;

[0092] S1064: Enlarge the filtered first-level re-optimized change feature map to the size of the first image to obtain the first-level change prediction map;

[0093] S1065: Filter the 0th level re-optimization change feature map to obtain the filtered 0th level re-optimization change feature map;

[0094] S1066: Enlarge the filtered Level 0 reoptimized change feature map to the size of the first image to obtain the Level 0 change prediction map.

[0095] S107: Deep supervision and optimization are performed based on the loss function L to obtain an optimized two-stage feature pyramid network model.

[0096] The loss function L includes the binary cross-entropy loss function L. BCE and Dice loss function L Dice :

[0097] L = L Dice +L BCE

[0098]

[0099]

[0100] Where N is the number of pixels in the transformation graph, y n This represents the state of the nth pixel, where n is the index of the pixel position in the image, and y n =1 indicates a change, y n =0 indicates no change, p n Y represents the probability of a change in the nth pixel, and Y is the actual change plot. This is a change prediction chart.

[0101] The binary cross-entropy function is widely used in binary classification problems. Change detection is a dense binary classification task, therefore the binary cross-entropy loss function L is chosen. BCE As part of the loss function, remote sensing images suffer from a severe imbalance in the distribution of samples in changing and unchanging regions. Cross-entropy loss, being pixel-level loss, tends to bias the model towards the category with more samples, affecting the final performance of network training. Therefore, the Dice coefficient loss function is introduced to mitigate the adverse effects of sample imbalance.

[0102] Based on the overall loss function L s Deep supervision is applied to the change prediction map, and the overall loss function L s The expression is as follows:

[0103]

[0104] Among them, L i Let L represent the loss function L calculated between the predicted change map and the actual change map at level i.

[0105] S108: Based on the optimized two-stage feature pyramid network model, the first image to be detected and the second image to be detected are detected to obtain a detection result. The detection result indicates the changes of the first image to be detected and the second image to be detected, wherein the first image to be detected and the second image to be detected are images at different times in the same location.

[0106] According to the present invention, after the input image passes through the Siamese coding feature extraction stage and the first-stage feature pyramid, it obtains multi-scale initial change features. Since the initial change features are generated by fusing initial difference features, the first-stage feature pyramid network introduces shallow initial difference features to compensate for the lack of spatial structure information during top-down layer-by-layer fusion. However, due to the presence of a large amount of noise and pseudo-change information in the shallow initial difference features, the generated initial change features inevitably still contain some pseudo-change noise interference. To further improve the description of change features, the second-stage feature pyramid network performs another top-down layer-by-layer fusion of the multi-scale initial change features, gradually eliminating pseudo-change noise interference through deep change features, and re-optimizing the change features. Simultaneously, to improve the detection performance of targets changing at different scales, during the training phase, the multi-level re-optimized change feature maps generated by the second-stage feature pyramid network are used for change prediction, and deep supervision is performed using a loss function weighting method. During testing, only the highest resolution prediction result (level 0 change prediction map) is used as the final change detection prediction result.

[0107] This invention first extracts coded features from bi-temporal images using a Siamese coding network and performs differential operations to extract initial difference features. These initial difference features are then input into a self-attention module to structurally enhance the information in the changed regions. Next, the refined initial difference features from the self-attention module and the multi-scale features extracted by the Siamese coding network are input into a first-stage feature pyramid network guided by a dual-attention module. This dual-attention mechanism guides the initial difference features, making the network more focused on the real changed regions in the bi-temporal images, effectively reconstructing the spatial structure and semantic features of the changed regions, and enabling spatial and semantic interaction of features to generate multi-scale initial change features. After obtaining the multi-scale initial change features, these features are input into a second-stage feature pyramid network. The second-stage feature pyramid network performs bottom-up, layer-by-layer fusion of the reconstructed multi-scale initial change features, aligning multi-scale semantic and spatial information, and eliminating spurious change noise interference from shallow features through deep supervision. Finally, by establishing a binary cross-entropy and Dice coefficient loss function, the two-stage feature pyramid network model is trained, effectively improving the problems of sample imbalance and scale differences affecting model performance during training.

[0108] Compared with the prior art, the beneficial effects of the present invention are:

[0109] (1) This invention uses the lightweight backbone network Resnet18 as a subnet of the Siamese coding network, which largely overcomes problems such as gradient vanishing and model performance degradation.

[0110] (2) This invention designs a dual-attention-guided multi-scale differential feature fusion module. Through a spatial-channel dual-attention mechanism, the initial differential features are guided, enabling the network to focus more on the real change regions in the dual-temporal images and effectively reconstruct the spatial structure information of the change regions. Compared with existing feature fusion strategies, the dual-attention fusion module fully utilizes the semantic information of deep features and the spatial information of shallow features. It selects effective semantic information of change on shallow features and enhances the structural information of change regions on deep features.

[0111] (3) This invention designs a two-stage feature pyramid network model to extract change features from multi-scale encoded features in two temporal phases, and aligns and registers the spatial and semantic information of the multi-scale features to suppress pseudo-change noise interference. At the same time, a deep supervision method is used to improve the model's change detection performance at different scales.

[0112] (4) This invention can significantly improve the effect of optical remote sensing image change detection. In particular, for remote sensing change detection scenarios with class imbalance and large scale differences, it improves the accuracy of change area description while increasing the detection rate.

[0113] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in the present invention, or make equivalent substitutions for some of the technical features. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting changes in remote sensing images based on a two-stage feature pyramid network, characterized in that, The method is designed for high-resolution multi-temporal remote sensing data, and includes the following steps: Encoding features are extracted from the first image and the second image respectively to obtain the first image encoding feature map and the second image encoding feature map; wherein, the first image and the second image are images at different time points located at the same location, the first image encoding feature map includes four encoding feature maps of different scales, namely the first image level 0 encoding feature map, the first image level 1 encoding feature map, the first image level 2 encoding feature map and the first image level 3 encoding feature map, and the second image encoding feature map includes four encoding feature maps of different scales, namely the second image level 0 encoding feature map, the second image level 1 encoding feature map, the second image level 2 encoding feature map and the second image level 3 encoding feature map; The first image coding feature map and the second image coding feature map are subjected to similarity processing to obtain an initial difference feature map; wherein, the initial difference feature map includes four initial difference feature maps at different scales, namely the level 0 initial difference feature map, the level 1 initial difference feature map, the level 2 initial difference feature map and the level 3 initial difference feature map. The third-level initial difference feature map is subjected to self-attention space enhancement processing to obtain an initial difference attention feature map; The initial difference feature map at level 0, the initial difference feature map at level 1, the initial difference feature map at level 2, and the initial difference attention feature map are subjected to a first-stage feature pyramid processing to obtain an initial change feature map; wherein, the initial change feature map includes three levels of initial change feature maps, namely the initial change feature map at level 0, the initial change feature map at level 1, and the initial change feature map at level 2. The initial change feature map is subjected to a second-stage feature pyramid processing to obtain a re-optimized change feature map; wherein the re-optimized change feature map includes three levels of re-optimized change feature maps, namely the level 0 re-optimized change feature map, the level 1 re-optimized change feature map, and the level 2 re-optimized change feature map. Based on the re-optimized change feature map, change prediction is performed on the first image and the second image to obtain a change prediction map; Deep supervision and optimization are performed based on the change prediction map and loss function to obtain an optimized two-stage feature pyramid network model. as well as The first image to be detected and the second image to be detected are detected based on the optimized two-stage feature pyramid network model to obtain detection results. The detection results indicate the changes of the first image to be detected and the second image to be detected, wherein the first image to be detected and the second image to be detected are images at different times in the same location. The first-stage feature pyramid processing of the initial difference feature map at level 0, the initial difference feature map at level 1, the initial difference feature map at level 2, and the initial difference attention feature map includes: Based on the initial difference attention feature map and the second-level initial difference feature map, a second-level initial change feature map is generated after second-level dual attention processing; Based on the second-level initial change feature map and the first-level initial difference feature map, a first-level initial change feature map is generated after first-level dual attention processing; and Based on the first-level initial change feature map and the first-level initial difference feature map, the first-level dual attention processing is used to generate the first-level initial change feature map. The second-level dual-attention processing, the first-level dual-attention processing, and the zero-level dual-attention processing all include dual-attention processing. The second-stage feature pyramid processing of the initial change feature map includes: Based on the initial difference attention feature map and the second-level initial change feature map, a second-level filtering fusion process is used to generate a second-level re-optimized change feature map; Based on the second-level reoptimization change feature map and the first-level initial change feature map, a first-level filtering and fusion process is used to generate a first-level reoptimization change feature map; and Based on the first-level re-optimized change feature map and the first-level initial change feature map, the first-level filtering fusion process is used to generate the first-level re-optimized change feature map.

2. The method as described in claim 1, characterized in that, The similarity processing of the first image encoded feature map and the second image encoded feature map includes: The similarity processing is performed using the difference method: in, F i1 Indicates the first image i Level-coded feature map, F i2 Indicates the second image i Level-coded feature map, F i Indicates the first i Initial differential feature map.

3. The method as described in claim 1, characterized in that, The step of performing self-attention spatial enhancement processing on the initial differential feature map of level 3 includes: The third-level initial difference feature map is filtered to obtain the filtered result of the third-level initial difference feature map; and An initial difference attention map is generated based on the filtering results: Where A represents the attention map, K represents the query feature map, Q represents the query feature map, T represents the matrix transpose, and V represents the representation feature map. This represents the initial difference attention map.

4. The method as described in claim 1, characterized in that, The dual attention processing includes: The first and second input data are compressed and activated along the channel dimension to generate a channel attention map; The channel attention maps are fused to generate a hybrid channel attention map; The first input data is filtered across channels based on the hybrid channel attention map to generate a channel feature map; The second input data and the channel feature map are fused to obtain a hybrid feature map; The hybrid feature map is compressed and activated in the spatial dimension to obtain a spatial hybrid attention map; The spatial information enhancement of the second input data is performed using the spatial hybrid attention map to generate a spatial feature map; and The channel feature map and the spatial feature map are fused to generate an initial change feature map.

5. The method as described in claim 1, characterized in that, The expression for the second-level filtering and fusion processing is as follows: ; The expression for the first-level filtering and fusion processing is as follows: ; The expression for the 0th-level filter fusion process is as follows: ; in, F r,2 This represents the feature map of the second-level re-optimization change. F r,1 This represents the feature map of the first-level re-optimization change. F r,0 This represents the feature map of the changes in the 0th level re-optimization. F R,2 This represents the initial change feature map of the second level. F R,1 This represents the initial change feature map of the first level. F R,0 This represents the initial change feature map of level 0. F a This represents the initial differential attention feature map; f 1×1 (.) represents a 1×1 convolution, and up(.) represents a 2x upsampling. This indicates pixel-level addition.

6. The method as described in claim 1, characterized in that, Using the re-optimized change feature map to predict changes in the first image and the second image includes: The second-level re-optimized change feature map is filtered to obtain a filtered second-level re-optimized change feature map; and the filtered second-level re-optimized change feature map is enlarged to the size of the first image to obtain a second-level change prediction map. The first-level reoptimization change feature map is filtered to obtain a filtered first-level reoptimization change feature map; and the filtered first-level reoptimization change feature map is enlarged to the size of the first image to obtain a first-level change prediction map; and The level 0 re-optimization change feature map is filtered to obtain a filtered level 0 re-optimization change feature map; and the filtered level 0 re-optimization change feature map is enlarged to the size of the first image to obtain a level 0 change prediction map.

7. The method as described in claim 1, characterized in that, The deep supervision and optimization based on the loss function includes: Construct a loss function L, which includes a binary cross-entropy loss function. and Dice loss function : in, N It is the number of pixels in the changing image. y n Indicates the first n The state of each pixel, where n is the index of the pixel position in the image. y n = 1 indicates change. y n =0 indicates no change. p n Indicates the first n The probability of a pixel change. Y This is a diagram showing the actual changes. It is a change prediction map; and Based on the overall loss function Ls, deep supervision is performed on the Level 2 change prediction map, the Level 1 change prediction map, and the Level 0 change prediction map. The expression of the overall loss function Ls is as follows: in, L i Indicates the first i The loss function L is calculated from the predicted and actual change maps of the level change.

Images