Remote sensing image change detection method based on feature constraint and cross attention guidance

Abstract

The present invention relates to a remote sensing image change detection method based on feature constraints and cross-attention guidance, including: obtaining high-resolution dual-phase images, and extracting dual-phase features using a dense residual network; constraining and aligning the dual-phase features in the deep feature space through a maximum mean difference constraint algorithm; encoding and decoding the features in combination with the Transformer architecture to aggregate contextual information; and designing a cross-attention mechanism to obtain interactive difference information between dual-phase images. The beneficial effects of the present invention are: the present invention constructs a feature constraint term loss, which effectively alleviates the pseudo-changes caused by style factors such as illumination and cloud occlusion in remote sensing images; and designs a cross-attention module, which can effectively couple the interactive difference discrimination information between dual-phase remote sensing images, and alleviate the information loss caused by adding layers to the deep neural network.

Description

Technical Field

[0001] The invention relates to the field of remote sensing image change detection, and more specifically, to a remote sensing image change detection method based on feature constraints and cross attention guidance. Background Art

[0002] Remote sensing image change detection is the process of identifying the changes between two images of the same scene taken at different times. This technology is of great significance to many fields such as environmental investigation, geological disaster detection, land cover investigation, urban planning, etc. Therefore, how to effectively improve the accuracy of remote sensing image change detection is an urgent problem to be solved.

[0003] With the rapid development and continuous improvement of remote sensing technology, high-resolution remote sensing images can not only provide clearer and more detailed surface information, but also allow more frequent observation and monitoring. Therefore, change detection technology based on high-resolution remote sensing images has important practical significance for urban planning, environmental monitoring, and land cover. Generally speaking, change detection can be divided into traditional change detection methods and deep learning-based change detection methods. Traditional change detection methods mainly focus on threshold segmentation, principal component analysis, and support vector machines. However, when processing remote sensing images of complex objects or mixed pixels, traditional methods often face problems such as object occlusion, shadows, or overlaps, resulting in poor results. In addition, traditional algorithms usually rely on manual feature extraction or threshold setting, which is time-consuming and affected by subjective factors when processing large-scale or complex remote sensing images.

[0004] Deep learning-based change detection methods have been widely used to improve the accuracy of change detection in identifying changed areas, such as residual networks, dense convolutional networks, generative adversarial networks, and Transformers. Compared with traditional change detection algorithms, deep learning models can learn features and patterns from raw data through end-to-end learning without the need to manually design features or rules. This makes the model more adaptable to complex ground object change patterns and reduces the need for manual intervention. In addition, deep learning models can automatically learn the optimal feature representation, making them more adaptable to different scenes and data. This adaptability helps to deal with image changes caused by different lighting, atmospheric conditions, and occlusion effects.

[0005] The mainstream deep learning models at present are mainly CNN and Transformer. The CNN model can effectively extract local features and texture information, but lacks the ability to model long-range dependencies, while the Transformer model is good at capturing the global context information of features. Therefore, the CNN-Transformer model is designed to effectively combine the advantages of both for feature information coupling. Although some scholars have tried to design the CNN-Transformer model for change detection tasks, it has not effectively alleviated the pseudo-changes caused by style changes such as lighting and cloud occlusion. In addition, it does not pay attention to the interactive difference discrimination information between the two input images, which leads to blurred or incorrect boundary detection. Summary of the invention

[0006] The purpose of the present invention is to overcome the deficiencies in the prior art and propose a remote sensing image change detection method based on feature constraints and cross attention guidance.

[0007] First, a remote sensing image change detection method based on feature constraints and cross-attention guidance is provided, including:

[0008] Step 1: Obtain high-resolution dual-phase images and extract dual-phase features using a dense residual network;

[0009] Step 2: Constraining alignment of the bi-phase features in the deep feature space by using the maximum mean difference constraint algorithm;

[0010] Step 3: Encode and decode features in conjunction with the Transformer architecture to aggregate contextual information;

[0011] Step 4: Design a cross attention mechanism, convolve the two-phase image to obtain Q, K and V respectively, perform cross attention operation to obtain interactive feature information, and then perform difference operation on the interactive features to obtain interactive difference information between the two-phase images after processing;

[0012] Step 5: Fuse the features extracted from the dual-phase image, and predict the final required binary change map through a series of convolution operations.

[0013] Preferably, the dense residual network is a pre-trained ResNet-18, and the bi-phase features include features of bi-phase images at three scales, expressed as:

[0014] F′ i ,F″ i ,F″′ i =Res(Img1),F′ j ,F″ j ,F″′ j=Res(Img2)

[0015] Among them, F′ i ,F″ i ,F″′ i and F′ j ,F″ j ,F″′ j To extract features of different scales, Res() is the residual block, and Img1 and Img2 are dual-temporal remote sensing images.

[0016] Preferably, in step 2, the dual-phase features are constrained to align in the deep feature space by using the maximum mean difference constraint algorithm, and the mathematical calculation formula is:

[0017]

[0018] Among them, L mmd (X, Y, H) represents the maximum mean difference algorithm constraint loss term, F i With F j F′ i ,F″ i ,F″′ i and F′ j ,F″ j ,F″′ j The collection of x i With y j Represent the image samples at two moments respectively, n represents the total number of samples, φ() represents the mapping function, H represents the Hilber feature space, Expressed as a Hilbert space constraint term.

[0019] Preferably, in step 3, the feature is encoded and decoded in combination with the Transformer architecture, and its mathematical expression is:

[0020] F i =enc(F i ),F j =enc(F j )

[0021] F i =dec(F i ),F j =dec(F j )

[0022] Among them, enc() is the Transformer encoder and dec() is the Transformer decoder.

[0023] Preferably, in step 4, the expression of the cross attention mechanism is:

[0024]

[0025]

[0026]

[0027]

[0028]

[0029] Among them, Conv() represents the convolution operation, Concat() represents the concatenation operation, and Q i and Q j Indicates the query key extracted from the two phase images, K i and K j Indicates the key key extracted from the two phase images before and after, V i and V j Indicates the Value extracted from the before and after phase images, d i Expressed as a scaling factor, F diff For the difference feature, and are the interactive features extracted from the previous and next phases respectively. and This is the final feature obtained by coupling the dual-phase image information through the cross-attention mechanism.

[0030] Preferably, the mathematical expression in step 5 is:

[0031]

[0032] Among them, up() is a bilinear interpolation function, P cat It is a binary image with 2 channels that is finally mapped through convolution operation.

[0033] In a second aspect, a remote sensing image change detection system based on feature constraints and cross-attention guidance is provided, which is used to execute any remote sensing image change detection method based on feature constraints and cross-attention guidance described in the first aspect, including:

[0034] An acquisition module is used to acquire high-resolution dual-phase images and extract dual-phase features using a dense residual network;

[0035] A constrained alignment module, used for constraining the alignment of the bi-phase features in the deep feature space by using a maximum mean difference constraint algorithm;

[0036] Aggregation module, which encodes and decodes features in combination with the Transformer architecture to aggregate contextual information;

[0037] A cross attention module is used to convolve the two-phase image to obtain Q, K and V respectively and perform cross attention operation to obtain interactive feature information, and then perform difference operation on the interactive features to obtain interactive difference information between the two-phase images after processing;

[0038] The fusion module is used to fuse the features extracted from the dual-phase images and predict the final required binary change map through a series of convolution operations.

[0039] In a third aspect, a computer storage medium is provided, characterized in that a computer program is stored in the computer storage medium; when the computer program is run on a computer, the computer executes any remote sensing image change detection method based on feature constraints and cross-attention guidance as described in the first aspect.

[0040] The beneficial effects of the present invention are:

[0041] 1. The present invention constructs a feature constraint loss and performs feature alignment on the multi-scale features extracted from the dual-phase images in the deep feature space, which effectively alleviates the pseudo-changes in the remote sensing images caused by style factors such as illumination and cloud occlusion.

[0042] 2. The present invention designs a cross-attention module, which can effectively couple the interactive difference discrimination information between dual-phase remote sensing images and alleviate the information loss caused by adding layers of deep neural networks. In addition, the changed area only occupies a small part of the original image. The previous technical methods will cause blurring and misdetection when detecting the boundaries of buildings, farmland, etc. The present invention can achieve clearer boundary detection and improve the accuracy and robustness of the change detection task. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of the remote sensing image change detection method based on feature constraints and cross attention guidance;

[0044] Figure 2 A comparison diagram of binary change results generated by the present method and the prior art;

[0045] Figure 3 This is another comparison diagram of binary change results generated by the present method and the prior art. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with embodiments. The description of the following embodiments is only used to help understand the present invention. It should be noted that for ordinary persons in the art, without departing from the principle of the present invention, the present invention can also be modified in some ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0047] Embodiment 1:

[0048] Embodiment 1 of the present application provides a remote sensing image change detection method based on feature constraints and cross-attention guidance, including:

[0049] Step 1: Obtain high-resolution bi-phase images and extract bi-phase features using a dense residual network.

[0050] In step 1, the dense residual network is a pre-trained ResNet-18, including a 7×7 convolutional layer and 4 residual layers, and the bi-temporal features include features of the bi-temporal image at three scales, expressed as:

[0051] F′ i ,F″ i ,F″′ i =Res(Img1),F′ j ,F″ j ,F″′ j =Res(Img2)

[0052] Among them, F′ i ,F″ i ,F″′ i and F′ j ,F″ j ,F″′ j To extract features of different scales, Res() is the residual block, and Img1 and Img2 are dual-temporal remote sensing images.

[0053] Step 2: Constrained alignment of the bi-phase features in the deep feature space is performed using a maximum mean difference constraint algorithm.

[0054] In step 2, the improvement of the present invention is to design feature constraints to align the dual-phase multi-scale features in the deep feature space to alleviate the pseudo-changes caused by style changes in remote sensing images. The mathematical calculation formula is:

[0055]

[0056] Among them, L mmd (X, Y, H) represents the maximum mean difference algorithm constraint loss term, F i With F j F′ i ,F″ i ,F″′ i and F′ j ,F″ j ,F″′ j The collection of x i With y jRepresent the image samples at two moments respectively, n represents the total number of samples, φ() represents the mapping function, H represents the Hilbert space, Expressed as a Hilbert space constraint term.

[0057] Step 3: Encode and decode features in conjunction with the Transformer architecture to aggregate contextual information.

[0058] In step 3, the feature is encoded and decoded in combination with the Transformer architecture, and its mathematical expression is:

[0059] F i =enc(F i ),F j =enc(F j )

[0060] F i =dec(F i ),F j =dec(F j )

[0061] Among them, enc() is the Transformer encoder and dec() is the Transformer decoder.

[0062] Step 4: Design a cross-attention mechanism, convolve the dual-phase image to obtain Q, K and V respectively, and perform cross-attention operations to obtain interactive feature information, and then perform difference operations on the interactive features. After processing, the interactive difference information between the dual-phase images is obtained, which effectively improves the feature representation.

[0063] Step 5: Fuse the features extracted from the dual-phase image, and predict the final required binary change map through a series of convolution operations.

[0064] Embodiment 2:

[0065] Based on Example 1, Example 2 of the present application provides a more specific remote sensing image change detection method based on feature constraints and cross attention guidance, including:

[0066] Step 1: Obtain high-resolution dual-phase images and use a dense residual network to extract dual-phase features. The purpose is to perform preliminary feature extraction on the data source for subsequent operations.

[0067] Step 2: Construct a domain adaptive constraint loss term to perform deep feature space alignment on the bi-temporal features extracted in the previous stage, in order to reduce the distribution differences between features. Although the bi-temporal remote sensing images come from the same sensor, the different shooting angles and the images with style changes such as lighting and cloud cover may cause differences in the feature distribution of the images, thus causing pseudo changes. By performing feature constraints, feature inconsistency can be minimized to help the model reduce the false detection rate.

[0068] Step 3: Encode and decode features of three different scales in combination with the Transformer architecture. The global context information of the input features is encoded through the Transformer encoder, and the features are converted into three-dimensional tokens of the target size as embeddings. Then, the Transformer self-attention mechanism is used for calculation. During the Transformer decoding process, the original input features and the Transformer high-level features are concatenated to effectively aggregate the feature information.

[0069]

[0070] Among them, Softmax() is a normalization function. is the scaling factor.

[0071] Step 4: Design a cross attention mechanism. First, the input dual-phase features are converted into three-dimensional tokens through a 1×1 convolution layer as embedding. Then, the Q of the T1 feature and the K of the T2 feature are extracted and transposed to calculate their probability distribution. Then, the softmax function is used to normalize and multiply the V of the T2 feature to obtain the interaction feature. Next, the difference between the interaction features at the two moments is used to obtain the intermediate difference feature. The difference feature is concatenated with the interaction features at the two moments in the channel dimension, and finally the feature of the interaction difference discrimination information of the coupled dual-phase image is obtained.

[0072] In step 4, the present invention takes into account the information loss caused by adding layers of the deep neural network and the limitation that the changed area only occupies a small part of the original image, and thus designs a cross-attention mechanism to effectively couple the interactive difference information between the two-phase images. The expression of the cross-attention mechanism is:

[0073]

[0074]

[0075]

[0076]

[0077]

[0078] Among them, Conv() represents the convolution operation, Concat() represents the concatenation operation, and Q i and Q j Indicates the query key extracted from the two phase images, K i and K j Indicates the key key extracted from the two phase images before and after, V i and V j Indicates the Value extracted from the two phase images before and after, d i Expressed as a scaling factor, F diff For the difference feature, and are the interactive features extracted from the previous and next phases respectively. and This is the final feature obtained by coupling the dual-phase image information through the cross-attention mechanism.

[0079] Step 5: Fuse the features extracted from the dual-phase image, and predict the final required binary change map through a series of convolution operations.

[0080] The mathematical expression in step 5 is:

[0081]

[0082] Among them, up() is a bilinear interpolation function, P cat It is a binary image with 2 channels that is finally mapped through convolution operation.

[0083] Specifically, the features of the two moments are concatenated and fused in the channel dimension, and then upsampled and passed through two 1×1 convolutional layers to map the number of channels of the fused features to 2 channels, represented as foreground and background. During the training process, we calculate the cross entropy loss between the predicted binary change map and the label map for optimization.

[0084]

[0085] Among them, L ce (Y,P) is expressed as the cross entropy loss between the prediction map and the label map.

[0086] In the change detection task, the previous CNN-Transformer model cannot effectively alleviate the pseudo-changes caused by feature differences, and cannot effectively utilize the interactive difference discrimination information between the two-phase images, which will eventually lead to fuzzy or incorrect detection of building and farmland boundaries. The method proposed in this paper can achieve more effective and accurate boundary detection. Figure 2 and Figure 3The following is a comparison of the binary change result graphs generated by this method and the prior art.

[0087] The change detection results of this method are compared with those of nine mainstream methods using four sets of data sets. The results are shown in Table 1. The F1 score and intersection over union (IOU) of this method are significantly better than those of other methods.

[0088] Table 1 Quantitative results on four data sets (bold indicates the best)

[0089]

[0090] It should be noted that the parts in this embodiment that are the same or similar to those in Embodiment 1 can be referenced to each other and will not be described in detail in this application.

[0091] Embodiment 3:

[0092] Based on embodiments 1 and 2, embodiment 3 of the present application provides a remote sensing image change detection system based on feature constraints and cross attention guidance, including:

[0093] An acquisition module is used to acquire high-resolution dual-phase images and extract dual-phase features using a dense residual network;

[0094] A constrained alignment module, used for constraining the alignment of the bi-phase features in the deep feature space by using a maximum mean difference constraint algorithm;

[0095] Aggregation module, which encodes and decodes features in combination with the Transformer architecture to aggregate contextual information;

[0096] A cross attention module is used to convolve the two-phase image to obtain Q, K and V respectively and perform cross attention operation to obtain interactive feature information, and then perform difference operation on the interactive features to obtain interactive difference information between the two-phase images after processing;

[0097] The fusion module is used to fuse the features extracted from the dual-phase images and predict the final required binary change map through a series of convolution operations.

[0098] Specifically, the system provided in this embodiment is a system corresponding to the methods provided in Embodiments 1 and 2. Therefore, the parts in this embodiment that are the same or similar to those in Embodiment 1 can be referenced to each other and will not be repeated in this application.

[0099] In summary, this application proposes a change detection model guided by feature constraints and cross-attention mechanisms for remote sensing image change detection tasks. Specifically, this application first extracts features of three different scales through CNN, and uses feature constraints to align features to reduce the distribution differences between input images. Then Transformer performs encoding and decoding stages to aggregate contextual information. The proposed cross-attention module effectively utilizes the interactive difference discrimination information of the input image to dynamically minimize the interference of irrelevant information. This application has conducted extensive experiments on four datasets: LEVIR-CD, CDD, CLCD, and WHU-CD. The results show that compared with other methods, the method of this application achieves SOTA performance in terms of accuracy and effectiveness, and has important application significance.

Claims

1. A remote sensing image change detection method based on feature constraints and cross attention guidance, characterized in that: include: Step 1: Obtain high-resolution dual-phase images and extract dual-phase features using a dense residual network; The dense residual network is a pre-trained ResNet-18, and the bi-phase features include features of bi-phase images at three scales, expressed as: F′ i ,F″ i ,F″′ i =Res(Img1),F′ j ,F′ j ,F″′ j =Res(Img2) Among them, F′ i , F″ i , F″′ i and F′ j , F″ j , F″′ j To extract features of different scales, Res() is the residual block, Img1 and Img2 are dual-temporal remote sensing images; Step 2: Constraining alignment of the bi-phase features in the deep feature space by using the maximum mean difference constraint algorithm; In step 2, the dual-phase features are constrained to align in the deep feature space using the maximum mean difference constraint algorithm, and the mathematical calculation formula is: Among them, L mmd (X, Y, H) represents the maximum mean difference algorithm constraint loss term, F i With F j F′ i , F″ i , F″′ i and F′ j , F″ j , F″′ j The collection of x i With y j Represent the image samples at two moments respectively, n represents the total number of samples, φ() represents the mapping function, H represents the Hilbert space, Expressed as a Hilbert space constraint term; Step 3: Encode and decode the bi-temporal features in combination with the Transformer architecture to aggregate contextual information; Step 4: Design a cross attention mechanism, convolve the two-phase image to obtain Q, K and V respectively, perform cross attention operation to obtain interactive feature information, and then perform difference operation on the interactive features to obtain interactive difference information between the two-phase images after processing; In step 4, the expression of the cross attention mechanism is: Among them, Conv() represents the convolution operation, Concat() represents the concatenation operation, and Q i and Q j Indicates the query key extracted from the two phase images, K i and K j Indicates the key key extracted from the two phase images before and after, V i and V j Indicates the Value extracted from the two phase images before and after, d i Expressed as a scaling factor, F diff For the difference feature, and are the interactive features extracted from the previous and next phases respectively. and The final feature obtained by coupling the bi-phase image information through the cross attention mechanism; Step 5: Fusing the bi-phase features extracted from the bi-phase image, and predicting the final required binary change map through a series of convolution operations; The mathematical expression in step 5 is: Among them, up() is a bilinear interpolation function, P cat It is a binary image with 2 channels that is finally mapped through convolution operation.

2. A remote sensing image change detection system based on feature constraints and cross-attention guidance, characterized in that: The method for remote sensing image change detection based on feature constraints and cross-attention guidance according to claim 1 comprises: An acquisition module is used to acquire high-resolution dual-phase images and extract dual-phase features using a dense residual network; A constrained alignment module, used for constraining the alignment of the bi-phase features in the deep feature space by using a maximum mean difference constraint algorithm; Aggregation module, which is used to encode and decode bi-temporal features in combination with the Transformer architecture to aggregate contextual information; A cross attention module is used to convolve the two-phase image to obtain Q, K and V respectively and perform cross attention operation to obtain interactive feature information, and then perform difference operation on the interactive features to obtain interactive difference information between the two-phase images after processing; The fusion module is used to fuse the features extracted from the dual-phase images and predict the final required binary change map through a series of convolution operations.

3. A computer storage medium, characterized in that: The computer storage medium stores a computer program; when the computer program is run on a computer, the computer executes the remote sensing image change detection method based on feature constraints and cross-attention guidance as described in claim 1.

Images