Lightweight remote sensing geographic image change detection method and system based on transformer

Abstract

The present application relates to the technical field of image processing, and discloses a change detection method and system for lightweight remote sensing geographic images based on a Transformer, wherein a first image and a second image in a to-be-detected image pair are respectively input into two parallel and weight-shared encoding branches, initial feature maps of different scales corresponding to each image are obtained, and feature enhancement is performed to obtain enhanced feature maps of different scales for each image; two enhanced feature maps of the same scale in the first image and the second image are fused to obtain preliminary fusion feature maps of each scale; a preliminary prediction map of the preliminary fusion feature map with the smallest resolution is obtained; the first image and the second image are fused after being spliced to obtain a bottom feature map; the preliminary fusion feature maps corresponding to each scale and the bottom feature map are sequentially fused, and the preliminary prediction map is used for weighting at the same time to obtain a target fusion map, classification prediction is performed, and a change area prediction map is obtained.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a lightweight method and system for detecting changes in remote sensing geographic images based on Transformer. Background Technology

[0002] Change detection is a crucial task in remote sensing image analysis, widely applied in land use / cover change monitoring, disaster assessment, and urban sprawl analysis. Traditional change detection methods are mainly divided into two categories: pixel-level and object-level. Pixel-level methods directly compare image pixels, but are sensitive to noise and registration errors; object-level methods analyze segmented objects, mitigating the shortcomings of pixel-level methods, but their performance is highly dependent on segmentation quality. With the development of deep learning technology, change detection methods based on convolutional neural networks (CNNs) have gradually become mainstream. These methods extract multi-level features from dual-temporal image pairs, demonstrating superior accuracy and robustness compared to traditional methods.

[0003] However, existing CNN-based methods still have significant limitations. Because CNNs typically use fixed-size convolutional kernels, their receptive field is limited, making it difficult to model long-range contextual dependencies. In change detection tasks, bi-temporal images come from different time points within the same geographical region; effectively capturing the spatiotemporal dependencies between them is crucial for accurately distinguishing between changed and invariant regions. The inadequacy of CNNs in global context modeling limits their discriminative performance in complex scenes.

[0004] In recent years, the Transformer architecture has received widespread attention in the field of computer vision due to its powerful global context modeling capabilities. Its core self-attention mechanism can establish a connection between any two positions in a sequence, thereby achieving effective capture of global information. However, the computational complexity of Transformer's global self-attention is proportional to the square of the length of the input sequence. When processing high-resolution remote sensing images, this leads to huge computational overhead and memory consumption, severely limiting the model's feasibility in practical applications, especially on edge devices with limited computing resources.

[0005] Therefore, how to fully utilize the global modeling advantages of Transformer while effectively controlling its computational cost in change detection tasks, and develop a lightweight change detection network model that combines powerful long-distance context modeling capabilities with efficient computational performance, has become an urgent technical problem to be solved. Summary of the Invention

[0006] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the existing technology cannot take into account both global image features and computational resource overhead, resulting in poor detection accuracy and low efficiency.

[0007] To address the aforementioned technical problems, this invention provides a lightweight change detection method for remote sensing geographic images based on Transformer, comprising:

[0008] The first and second images in the image pair to be detected are respectively input into two parallel and weight-shared Transformer encoding branches to obtain multiple initial feature maps of different scales for each image;

[0009] For the initial feature maps of each image at each scale, frequency separation is performed to obtain each high-frequency component, and spatial attention operation, skip connections and depthwise separable convolution are performed to obtain multiple enhanced feature maps of different scales for each image.

[0010] For the first image and the second image, the two enhanced feature maps of the same scale are concatenated according to the channel dimension to obtain multiple concatenated feature maps, which are then fused to obtain preliminary fused feature maps for each scale.

[0011] Input the initial fused feature map with the lowest resolution into the MLP to obtain the initial prediction map;

[0012] The first image and the second image are stitched together and then fused to obtain the low-level feature map;

[0013] The preliminary fusion feature maps and the underlying feature maps corresponding to each scale are fused sequentially from low to high resolution, and the preliminary prediction map is used for weighting to obtain the target fusion map.

[0014] Classify and predict the target fusion map to obtain a predicted map of the changed area.

[0015] Preferably, the image input Transformer encoding branch includes:

[0016] The image is passed through a feedforward network with a predetermined number of stacked layers connected sequentially along the forward propagation direction to obtain an initial feature map at the first scale;

[0017] The initial feature map at the first scale is passed through a feedforward network with a preset number of stacked layers connected sequentially along the forward propagation direction to obtain the initial feature map at the second scale.

[0018] The initial feature map of the second scale is passed through a depth-separable convolutional attention block with a preset number of stacked layers connected sequentially along the forward propagation direction, and a feedforward network with a preset number of stacked layers to obtain the initial feature map of the third scale.

[0019] Preferably, the multiple initial feature maps of different scales corresponding to each image are represented as follows:

[0020] ;

[0021] ;

[0022] ;

[0023] in, , This represents the first image in the image pair to be detected. This represents the second image in the image pair to be detected; , and These represent the first and second parts of the image pair to be detected. image The first-scale initial feature map, the second-scale initial feature map, and the third-scale initial feature map; Indicates the preset stacking layer number; Indicates a feedforward network. This represents a feedforward network with a predetermined number of stacked layers connected in series along the forward propagation direction; This indicates a convolutional layer with a kernel size of 3 and a stride of 2; This represents a depthwise separable convolutional attention block. This represents a depth-separable convolutional attention block consisting of a preset number of stacked layers connected sequentially along the forward propagation direction.

[0024] Preferably, the implementation of depthwise separable convolutional attention blocks includes:

[0025] The input features are processed through depthwise separable convolution and then added to the initial input features to obtain intermediate features;

[0026] The intermediate features are fed through a feedforward network and then added to the initial intermediate features to obtain the output features of the depthwise separable convolutional attention block.

[0027] Among them, input features The output of depthwise separable convolution , is represented as:

[0028] ;

[0029] in, Represents a linear mapping. This indicates bilinear interpolation upsampling. This indicates a bullish attention-based strategy; This represents a depthwise separable convolutional unit that downsamples the input features to obtain... , , .

[0030] Preferably, the initial feature maps at various scales of each image are subjected to frequency separation to obtain each high-frequency component, and spatial attention operation, skip connections, and depthwise separable convolution are performed to obtain multiple enhanced feature maps at different scales for each image, including:

[0031] After performing average pooling and bilinear interpolation on the initial feature map of any scale of any input image, the low-frequency component is obtained, and the corresponding high-frequency component is obtained by subtracting the low-frequency component from the initial feature map.

[0032] After performing spatial attention operations on the high-frequency components, they are skipped connections with the initial feature map and convolved to output the high-frequency intermediate feature map;

[0033] After performing channel attention operations on the high-frequency intermediate feature map, it is skipped and connected to the high-frequency intermediate feature map, and then passed through a depthwise separable convolution to output the corresponding enhanced feature map.

[0034] Preferably, after performing spatial attention operation on the high-frequency components, a skip connection is made with the initial feature map, and a high-frequency intermediate feature map is output via convolution, including:

[0035] Perform spatial attention operation on high-frequency components to obtain spatial attention feature maps. , is represented as:

[0036] ;

[0037] Spatial attention feature map With the initial feature map Skip connections, convolution outputs high-frequency intermediate feature maps , is represented as: ;

[0038] in, Represents high-frequency components. This represents the activation function. This represents a convolution with a kernel size of 7. This indicates that the data is assembled according to the channel dimension. Indicates average pooling. This indicates max pooling. This represents a convolution with a kernel size of 3.

[0039] Preferably, after performing channel attention operation on the high-frequency intermediate feature map, it is skip-connected with the high-frequency intermediate feature map and then subjected to depthwise separable convolution to output the corresponding enhanced feature map, represented as:

[0040] Perform channel attention operation on the high-frequency intermediate feature map to obtain the channel attention feature map. , is represented as:

[0041] ;

[0042] Channel attention feature map High-frequency intermediate feature map Skip connections and pass through depthwise separable convolutions to output enhanced feature maps. , is represented as: ;

[0043] in, This represents a multilayer perceptron.

[0044] Preferably, for the first image and the second image, the two enhanced feature maps of the same scale are concatenated according to the channel dimension to obtain multiple concatenated feature maps, which are then fused separately to obtain preliminary fused feature maps corresponding to each scale, as shown below:

[0045] ;

[0046] in, Indicates the first Preliminary scalar fusion of feature maps Indicates a feedforward network. This indicates that the data is concatenated according to the channel dimension; and These represent the first image and the second image respectively. Scale-enhanced feature map.

[0047] Preferably, the preliminary fused feature maps and the underlying feature maps corresponding to each scale are fused sequentially from low to high resolution, and the preliminary prediction map is used for weighting to obtain the target fused map, including:

[0048] Preliminary fusion of feature maps at the third scale Upsampling to the initial fusion feature map with the second scale Once the dimensions are the same, the two are merged, and the preliminary prediction map is used. Weighted summation to obtain the first intermediate feature map , is represented as:

[0049] ;

[0050] The first intermediate feature map Upsampled to the initial fusion feature map with the first scale Once the dimensions are the same, the two are merged and weighted using the preliminary prediction map. Obtain the second intermediate feature map , is represented as:

[0051] ;

[0052] The second intermediate feature map Upsampling to the lower-level feature map Once the dimensions are the same, the two are merged and weighted using the preliminary prediction map. Obtain the target fusion image , is represented as:

[0053] ;

[0054] in, Indicates a feedforward network. This indicates splicing along the channel dimension. This indicates bilinear interpolation.

[0055] This embodiment provides a system for a lightweight change detection method for remote sensing geographic images based on Transformer, including:

[0056] The feature encoding module is used to input the first and second images of the image pair to be detected into two parallel and weight-shared Transformer encoding branches to obtain multiple initial feature maps of different scales for each image.

[0057] The feature enhancement module is used to perform frequency separation on the initial feature maps of each image at each scale, obtain each high-frequency component, and perform spatial attention operation, skip connections and depthwise separable convolution to obtain multiple enhanced feature maps of each image at different scales.

[0058] The feature fusion module is used to stitch together two enhanced feature maps of the same scale corresponding to the first image and the second image according to the channel dimension to obtain multiple stitched feature maps, and then fuse them separately to obtain preliminary fused feature maps corresponding to each scale.

[0059] The preliminary fusion module is used to input the preliminary fusion feature map with the lowest resolution into the MLP to obtain the preliminary prediction map;

[0060] The low-level feature acquisition module is used to stitch the first image and the second image together and then fuse them to obtain the low-level feature map.

[0061] The weighted fusion module is used to fuse the preliminary fusion feature maps and the underlying feature maps at each scale in order of resolution from low to high, and at the same time, it uses the preliminary prediction map for weighting to obtain the target fusion map.

[0062] The prediction module is used to classify and predict the target fusion map and obtain the predicted map of the changed area.

[0063] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0064] The lightweight change detection method for remote sensing geographic images based on Transformer described in this invention addresses the high computational complexity of existing Transformer architectures in remote sensing image feature extraction by constructing a hierarchical, hybrid lightweight backbone network. Simultaneously, it strengthens feature representation capabilities through frequency-separated feature enhancement and multi-scale fusion incorporating underlying information. This ensures that, despite a significant reduction in parameters and computational cost, the model retains strong long-range context modeling capabilities and a keen ability to capture subtle changes. It successfully achieves high-precision remote sensing change detection in resource-constrained environments, improving the accuracy of detecting changed regions in remote sensing geographic images. Attached Figure Description

[0065] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0066] Figure 1 This is a flowchart of the steps of the method for detecting changes in remotely sensed geographic images according to the present invention;

[0067] Figure 2 This is a schematic diagram of a lightweight backbone network based on Transformer;

[0068] Figure 3 This is a schematic diagram showing the visualization results of the method of the present invention;

[0069] Figure 4 This is a visual comparison chart of the results of this invention and existing technologies. Detailed Implementation

[0070] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0071] Reference Figure 1 The flowchart of the method for detecting changes in remote sensing geographic images according to the present invention is shown below, with specific steps as shown in S101 to S107.

[0072] S101: Input the first and second images of the image pair to be detected into two parallel and weight-shared Transformer encoding branches to obtain multiple initial feature maps of different scales for each image.

[0073] Reference Figure 2 The image shown is a schematic diagram of a lightweight backbone network based on Transformer; specifically, the image to be detected is... Inputting a lightweight backbone network based on Transformer yields multi-scale initial feature maps, represented as follows:

[0074] ;

[0075] ;

[0076] ;

[0077] in, , This represents the first image in the image pair to be detected. This represents the second image in the image pair to be detected; , and These represent the first and second parts of the image pair to be detected. image The first-scale initial feature map, the second-scale initial feature map, and the third-scale initial feature map; This represents a feedforward network consisting of two point convolutional layers with kernel size 1 and a depthwise convolutional layer with kernel size 3 sandwiched between them. This represents a feedforward network with a predetermined number of stacked layers connected in series along the forward propagation direction; This indicates a convolutional layer with a kernel size of 3 and a stride of 2, used to upsample and downsample features to half their previous size on both the length and width scales; This represents a depthwise separable convolutional attention block. This represents a depth-separable convolutional attention block consisting of a preset number of stacked layers connected sequentially along the forward propagation direction. Indicates the preset stacking layer number, subscript Indicates stacking Such layers, through a lightweight backbone network formed by stacking these modules, can extract data from the image to be detected. Extracting multi-scale features , , .

[0078] The specific implementation of depthwise separable convolutional attention blocks is represented as follows:

[0079] ;

[0080] ;

[0081] in, The specific implementation can be represented as:

[0082] ;

[0083] in, This indicates a depthwise separable convolution, consisting of a depthwise convolutional layer with a kernel size of 3 and a stride of 2, and a pointwise convolutional layer with a kernel size of 1. It is used to downsample the input features and obtain... , , In other words, the query, key, and value required by Multi-Head Attention (MHSA) are represented as: ; obtained , , It will then use permute and reshape operations to... Form transformed In the form of, This indicates the number of heads in the bulls' self-attention. The sequence length is... Subsequently, on , , Perform multi-head self-attention and restore the features to their original size and number of channels through bilinear interpolation upsampling and linear mapping to match the original features. Perform skip connections; the final FFN of the module is used to implement non-linear transformations in the channel dimension. The two sums represent skip connections, which are used to preserve the original feature information.

[0084] S102: For the initial feature maps of each image at each scale, perform frequency separation to obtain each high-frequency component, and perform spatial attention operation, skip connections and depthwise separable convolution to obtain multiple enhanced feature maps of different scales for each image.

[0085] Specifically, the feature enhancement module enhances the extracted multi-stage initial features; the multi-scale features extracted from the backbone network are frequency-separated to obtain high-frequency components, represented as: ;

[0086] in, This is average pooling, used to extract low-frequency components; This is a bilinear interpolation used to restore low-frequency components to the same size as the original features, while the subtraction operation is used to obtain high-frequency components.

[0087] Then, spatial attention and skip connections are applied to the high-frequency components:

[0088] ;

[0089] ;

[0090] in As an activation function, it controls the range of output values ​​between (0, 1); This indicates an operation that splices data along the channel dimension.

[0091] Then, channel attention and skip connections are applied to the intermediate features, and finally, the output of the feature enhancement module is obtained through a depthwise separable convolution with a kernel size of 7.

[0092] ;

[0093] .

[0094] S103: For the first image and the second image, the two enhanced feature maps of the same scale are concatenated according to the channel dimension to obtain multiple concatenated feature maps, and then fused separately to obtain the preliminary fused feature maps corresponding to each scale.

[0095] No. Preliminary Scale Fusion Feature Map , is represented as:

[0096] ;

[0097] in, Indicates a feedforward network. This indicates that the data is concatenated according to the channel dimension; and These represent the first image and the second image respectively. Scale-enhanced feature map.

[0098] S104: Input the initial fused feature map with the lowest resolution into the MLP to obtain the initial prediction map, represented as: .

[0099] S105: After concatenating the first image and the second image, fuse them to obtain the low-level feature map, represented as: .

[0100] S106: Fuse the preliminary fusion feature maps and the low-level feature maps corresponding to each scale in order of increasing resolution, while using the preliminary prediction map for weighting, to obtain the target fusion map, including:

[0101] Preliminary fusion of feature maps at the third scale Upsampling to the initial fusion feature map with the second scale Once the dimensions are the same, the two are merged, and the preliminary prediction map is used. Weighted summation to obtain the first intermediate feature map , is represented as:

[0102] ;

[0103] The first intermediate feature map Upsampled to the initial fusion feature map with the first scale Once the dimensions are the same, the two are merged and weighted using the preliminary prediction map. Obtain the second intermediate feature map , is represented as:

[0104] ;

[0105] The second intermediate feature map Upsampling to the lower-level feature map Once the dimensions are the same, the two are merged and weighted using the preliminary prediction map. Obtain the target fusion image , is represented as:

[0106] ;

[0107] in, Indicates a feedforward network. This indicates splicing along the channel dimension. This indicates bilinear interpolation.

[0108] S107: Classify and predict the target fusion map to obtain a predicted map of the changed area.

[0109] Specifically, the target fusion map is input into the prediction head, and the change detection prediction result is obtained through the pixel-level prediction head. The specific implementation is as follows: For the output of the multi-scale feature fusion part, its length and width dimensions are already consistent with the original image pair. Therefore, it is directly compressed in the channel dimension through an MLP layer, resulting in an output channel number of 2, which is divided into two categories: changed and unchanged. The predicted classification is represented as: .

[0110] In this embodiment of the invention, the change detection network, which consists of a lightweight backbone network based on Transformer, a feature enhancement module, a feature fusion module, and a pixel-level prediction head, is optimized using the cross-entropy loss function.

[0111] The cross-entropy loss function is expressed as:

[0112] ;

[0113] in, C represents the number of image pairs in each batch, and C represents the number of categories. Indicates the first in the current batch The true value of the detection of each image pair Indicates the first in the current batch The network prediction results for each image pair are obtained by minimizing... To optimize.

[0114] In this embodiment, change detection simulation experiments were conducted using the method described in this application and two existing change detection methods: SNUNet (a multi-level feature stitching method in which densely connected Siamese networks are used for change detection) and ChangeFormer (a pure Transformer method specifically designed for change detection tasks, including a hierarchical Transformer encoder and a simple MLP decoder in a Siamese architecture), and the experimental results were compared.

[0115] All experiments were performed on an NVIDIA RTX 4090 GPU, and all image pairs used in the experiments were from the DSIFN-CD remote sensing dataset. The evaluation metrics used were F1-score and IoU, which quantitatively compare the change detection accuracy of different methods. The F1-score is defined as follows: The definition of IoU , where TP, TN, FP, and FN represent the number of true positive, true negative, false positive, and false negative pixels, respectively. The values ​​of F1-score and IoU range from (0,1), with larger values ​​indicating higher model detection accuracy.

[0116] Select an image to be detected from the DSIFN and perform detection using the method described in this application, referring to... Figure 3 The image shown is a visualization of the results of the method of this invention; from left to right, it represents the original image A, the original image B, the ground truth label, the model prediction result, true positives (TP) represented by white, true negatives (TN) represented by black, false positives (FP) represented by red, and false negatives (FN) represented by green. Fewer red and green pixels indicate better results.

[0117] Five images to be detected were selected from DSIFN and detected using the method described in this application, SNUNet and ChangeFormer, with reference to... Figure 4 The image shows a comparison of the visualization results of the present invention and existing technologies. Columns 1, 2, and 3 represent the original image A, original image B, and ground truth labels; columns 4 and 5 represent the results from SNUNet and ChangeFormer, respectively. Column 6 shows the results of the method proposed in this invention. It can be observed that the method proposed in this invention significantly improves the prediction of building change areas in remote sensing images. For example, in the second row, for the detection of large building change areas, the method described in this application has significantly fewer false positive pixels than the comparison method. Similarly, in the fifth row, for the detection of small change areas, the method described in this application has significantly fewer false negative pixels than the comparison method. The quantitative results of the method described in this application and the other two comparison methods on the entire DSIFN-CD test set are shown in Table 1.

[0118] Table 1 Comparison of Method Indicators

[0119] method F1 score IoU SNUNet 82.03 69.53 ChangeFormer 86.48 76.48 This invention 87.49 77.77

[0120] Based on the above embodiments, this invention provides a system for a lightweight remote sensing geographic image change detection method based on Transformer. Specifically, the system may include:

[0121] The feature encoding module is used to input the first and second images of the image pair to be detected into two parallel and weight-shared Transformer encoding branches to obtain multiple initial feature maps of different scales for each image.

[0122] The feature enhancement module is used to perform frequency separation on the initial feature maps of each image at each scale, obtain each high-frequency component, and perform spatial attention operation, skip connections and depthwise separable convolution to obtain multiple enhanced feature maps of each image at different scales.

[0123] The feature fusion module is used to stitch together two enhanced feature maps of the same scale corresponding to the first image and the second image according to the channel dimension to obtain multiple stitched feature maps, and then fuse them separately to obtain preliminary fused feature maps corresponding to each scale.

[0124] The preliminary fusion module is used to input the preliminary fusion feature map with the lowest resolution into the MLP to obtain the preliminary prediction map;

[0125] The low-level feature acquisition module is used to stitch the first image and the second image together and then fuse them to obtain the low-level feature map.

[0126] The weighted fusion module is used to fuse the preliminary fusion feature maps and the underlying feature maps at each scale in order of resolution from low to high, and at the same time, it uses the preliminary prediction map for weighting to obtain the target fusion map.

[0127] The prediction module is used to classify and predict the target fusion map and obtain the predicted map of the changed area.

[0128] The Transformer-based lightweight remote sensing geographic image change detection system of this embodiment is used to implement the aforementioned Transformer-based lightweight remote sensing geographic image change detection method. Therefore, the specific implementation of the Transformer-based lightweight remote sensing geographic image change detection system can be found in the embodiment section of the Transformer-based lightweight remote sensing geographic image change detection method above. For example, the feature encoding module, feature enhancement module, feature fusion module, preliminary fusion module, low-level feature acquisition module, weighted fusion module, and prediction module are respectively used to implement steps S101 to S107 in the aforementioned Transformer-based lightweight remote sensing geographic image change detection method. Therefore, its specific implementation can be referred to the description of the corresponding embodiments, and will not be repeated here.

[0129] This invention addresses the high computational cost and deployment limitations of existing Transformer-based change detection methods by constructing a lightweight backbone network for feature extraction. Building upon the powerful long-range context modeling capabilities of the Transformer architecture, it achieves high-precision change detection with low parameter count and computational complexity through innovative lightweight design and effective feature enhancement. This invention constructs a hierarchical, hybrid (CNN-Transformer) lightweight backbone network, solving the problem of high computational complexity in remote sensing image feature extraction using existing Transformer architectures. Simultaneously, it enhances feature representation capabilities through frequency-separated feature enhancement and multi-scale fusion incorporating underlying information. This ensures that while significantly reducing parameter count and computational cost, the model still possesses strong long-range context modeling capabilities and a keen ability to capture subtle changes, successfully achieving high-precision remote sensing change detection even in resource-constrained environments.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A lightweight change detection method for remote sensing geographic images based on Transformer, characterized in that, include: The first and second images in the image pair to be detected are respectively input into two parallel and weight-shared Transformer encoding branches to obtain multiple initial feature maps of different scales for each image; For the initial feature maps of each image at each scale, frequency separation is performed to obtain each high-frequency component. Then, spatial attention operation, skip connections, and depthwise separable convolution are performed to obtain multiple enhanced feature maps of different scales for each image. This includes: performing spatial attention operation on the high-frequency components, skip connections with the initial feature map, and convolution to output a high-frequency intermediate feature map; performing channel attention operation on the high-frequency intermediate feature map, skip connections with the high-frequency intermediate feature map, and passing it through depthwise separable convolution to output the corresponding enhanced feature map. For the first image and the second image, the two enhanced feature maps of the same scale are concatenated according to the channel dimension to obtain multiple concatenated feature maps, which are then fused to obtain preliminary fused feature maps for each scale. The initial fused feature map with the lowest resolution is input into a multilayer perceptron (MLP) to obtain an initial prediction map. The first image and the second image are stitched together and then fused to obtain the low-level feature map; The preliminary fused feature maps and the underlying feature maps corresponding to each scale are fused sequentially from low to high resolution, while the preliminary prediction map is used for weighting to obtain the target fused map. This includes: upsampling the preliminary fused feature map of the third scale to the same size as the preliminary fused feature map of the second scale, fusing the two, and using the preliminary prediction map for weighting to obtain a first intermediate feature map; upsampling the first intermediate feature map to the same size as the preliminary fused feature map of the first scale, fusing the two, and using the preliminary prediction map for weighting to obtain a second intermediate feature map; upsampling the second intermediate feature map to the same size as the underlying feature map, fusing the two, and using the preliminary prediction map for weighting to obtain the target fused map. Classify and predict the target fusion map to obtain a predicted map of the changed area.

2. The lightweight remote sensing geographic image change detection method based on Transformer according to claim 1, characterized in that, The image input Transformer encoding branch includes: The image is passed through a feedforward network with a predetermined number of stacked layers connected sequentially along the forward propagation direction to obtain an initial feature map at the first scale; The initial feature map at the first scale is passed through a feedforward network with a preset number of stacked layers connected sequentially along the forward propagation direction to obtain the initial feature map at the second scale. The initial feature map of the second scale is passed through a depth-separable convolutional attention block with a preset number of stacked layers connected sequentially along the forward propagation direction, and a feedforward network with a preset number of stacked layers to obtain the initial feature map of the third scale.

3. The lightweight remote sensing geographic image change detection method based on Transformer according to claim 2, characterized in that, Each image corresponds to multiple initial feature maps at different scales, represented as follows: ； ； ； in, , This represents the first image in the image pair to be detected. This represents the second image in the image pair to be detected; , and These represent the first and second parts of the image pair to be detected. image The first-scale initial feature map, the second-scale initial feature map, and the third-scale initial feature map; Indicates the preset stacking layer number; Indicates a feedforward network. This represents a feedforward network with a predetermined number of stacked layers connected in series along the forward propagation direction; This indicates a convolutional layer with a kernel size of 3 and a stride of 2; This represents a depthwise separable convolutional attention block. This represents a depth-separable convolutional attention block consisting of a preset number of stacked layers connected sequentially along the forward propagation direction.

4. The lightweight remote sensing geographic image change detection method based on Transformer according to claim 3, characterized in that, The implementation of depthwise separable convolutional attention blocks includes: The input features are processed through depthwise separable convolution and then added to the initial input features to obtain intermediate features; The intermediate features are fed through a feedforward network and then added to the initial intermediate features to obtain the output features of the depthwise separable convolutional attention block. Among them, input features The output of depthwise separable convolution , is represented as: ； in, Represents a linear mapping. This indicates bilinear interpolation upsampling. This indicates a bullish focus operation; This represents a depthwise separable convolutional unit that downsamples the input features to obtain the query required by Multi-Head Attention (MHSA). ,key Sum .

5. The lightweight remote sensing geographic image change detection method based on Transformer according to claim 1, characterized in that, For the initial feature maps of each image at each scale, frequency separation is performed to obtain each high-frequency component, including: After performing average pooling and bilinear interpolation on the initial feature map of any scale of any input image, the low-frequency component is obtained. Then, the low-frequency component is subtracted from the initial feature map to obtain the corresponding high-frequency component.

6. The lightweight remote sensing geographic image change detection method based on Transformer according to claim 5, characterized in that, After performing spatial attention on the high-frequency components, skip connections are made with the initial feature map, and convolution is performed to output a high-frequency intermediate feature map, including: Perform spatial attention operation on high-frequency components to obtain spatial attention feature maps. , is represented as: ； Spatial attention feature map With the initial feature map Skip connections, convolution outputs high-frequency intermediate feature maps , is represented as: ; in, Represents high-frequency components. This represents the activation function. This represents a convolution with a kernel size of 7. This indicates that the data is assembled according to the channel dimension. Indicates average pooling. This indicates max pooling. This represents a convolution with a kernel size of 3.

7. The lightweight remote sensing geographic image change detection method based on Transformer according to claim 6, characterized in that, After performing channel attention on the high-frequency intermediate feature map, a skip connection is made with the high-frequency intermediate feature map, and then passed through a depthwise separable convolution to output the corresponding enhanced feature map, represented as: Perform channel attention operation on the high-frequency intermediate feature map to obtain the channel attention feature map. , is represented as: ； Channel attention feature map High-frequency intermediate feature map Skip connections and pass through depthwise separable convolutions to output enhanced feature maps. , is represented as: ; in, This represents a multilayer perceptron.

8. The lightweight remote sensing geographic image change detection method based on Transformer according to claim 1, characterized in that, For the first and second images, their corresponding enhanced feature maps of the same scale are concatenated according to the channel dimension to obtain multiple concatenated feature maps. These concatenated feature maps are then fused separately to obtain preliminary fused feature maps for each scale, as shown below: ； in, Indicates the first Preliminary scalar fusion of feature maps Indicates a feedforward network. This indicates that the data is concatenated according to the channel dimension; and These represent the first image and the second image respectively. Scale-enhanced feature map.

9. The lightweight remote sensing geographic image change detection method based on Transformer according to claim 8, characterized in that: Preliminary fusion of feature maps at the third scale Upsampling to the initial fusion feature map with the second scale Once the dimensions are the same, the two are merged, and the preliminary prediction map is used. Weighted summation to obtain the first intermediate feature map , is represented as: ； The first intermediate feature map Upsampled to the initial fusion feature map with the first scale Once the dimensions are the same, the two are merged, and the preliminary prediction map is used. Weighted summation to obtain the second intermediate feature map , is represented as: ； The second intermediate feature map Upsampling to the lower-level feature map Once the dimensions are the same, the two are merged, and the preliminary prediction map is used. Weighted summation to obtain the target fusion graph , is represented as: ； in, Indicates a feedforward network. This indicates splicing along the channel dimension. This indicates bilinear interpolation.

10. A system based on the lightweight remote sensing geographic image change detection method according to any one of claims 1 to 9, characterized in that, include: The feature encoding module is used to input the first and second images of the image pair to be detected into two parallel and weight-shared Transformer encoding branches to obtain multiple initial feature maps of different scales for each image. The feature enhancement module is used to perform frequency separation on the initial feature maps of each image at each scale, obtain each high-frequency component, and perform spatial attention operation, skip connections and depthwise separable convolution to obtain multiple enhanced feature maps of each image at different scales. The feature fusion module is used to stitch together two enhanced feature maps of the same scale corresponding to the first image and the second image according to the channel dimension to obtain multiple stitched feature maps, and then fuse them separately to obtain preliminary fused feature maps corresponding to each scale. The preliminary fusion module is used to input the preliminary fused feature map with the lowest resolution into the multilayer perceptron (MLP) to obtain a preliminary prediction map. The low-level feature acquisition module is used to stitch the first image and the second image together and then fuse them to obtain the low-level feature map. The weighted fusion module is used to fuse the preliminary fusion feature maps and the underlying feature maps at each scale in order of resolution from low to high, and at the same time, it uses the preliminary prediction map for weighting to obtain the target fusion map. The prediction module is used to classify and predict the target fusion map and obtain the predicted map of the changed area.

Images