Deep learning-based remote sensing image change detection method, computer device and readable storage medium
Patent Information
- Application Number
- CN202410057654.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-16
- Publication Date
- 2026-10-27
- Estimated Expiration
- 2044-01-16
Smart Images
Figure CN117994675B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of computer vision remote sensing images, and more specifically, to a method for detecting changes in remote sensing images based on deep learning, a computer device, and a computer-readable storage medium. Background Technology
[0002] With the rapid development of deep learning technology, remote sensing imagery has become a research area of great interest to many scholars. Deep learning has wide applications in remote sensing image processing. Remote sensing images are image data of the Earth's surface acquired through long-range sensors such as satellites and aircraft. Deep learning algorithms analyze and process remote sensing images, and this technology has significant application prospects in many fields. For example: remote sensing target detection and recognition, classification of ground features in remote sensing images, environmental monitoring, urban planning, and earthquake disaster assessment.
[0003] Despite significant advancements in existing deep learning-based remote sensing image change detection methods, several technical limitations remain:
[0004] (1) Resolution limitation: The resolution of remote sensing images is limited, especially for large-scale land cover. Lower resolution may lead to loss of detail, limiting the accurate extraction and analysis of certain objects or features.
[0005] (2) Cloud, fog and shadow interference: Clouds, fog and shadows have a negative impact on the quality and interpretation of remote sensing images. They can cause problems such as image occlusion, uneven lighting and color distortion, thereby reducing the usability and interpretability of the images.
[0006] (3) Temporal nature of remote sensing images: Static remote sensing images are difficult to provide dynamic information on changes in the Earth's surface. Therefore, for some applications that require time-series data, such as environmental monitoring and disaster management, static images cannot provide complete information.
[0007] (4) Data acquisition cost and frequency: Acquiring high-resolution remote sensing images typically requires expensive equipment and sensors, and the frequency of data acquisition is limited. This limits the feasibility of applications for real-time or frequent monitoring.
[0008] (5) Challenges of Automatic Interpretation and Analysis: Remote sensing images often require automatic interpretation and analysis to extract useful information. However, automated processing still faces some challenges, such as semantic segmentation, object detection, and classification in remote sensing images.
[0009] (6) Data volume and processing capacity (i.e., computational and memory challenges): Remote sensing image data is massive, placing enormous demands on storage and processing capabilities. Effectively processing and analyzing large-scale remote sensing image data remains a challenge, especially in real-time or near real-time applications.
[0010] Therefore, it is necessary to study an innovative deep learning-based method for remote sensing image change detection to address the technical challenges of computation and memory management faced by traditional remote sensing image detection models when processing high-resolution images. Summary of the Invention
[0011] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.
[0012] Therefore, the first objective of this invention is to propose a method for detecting changes in remote sensing images based on deep learning.
[0013] The second objective of this invention is to provide a computer device.
[0014] A third objective of this invention is to provide a computer-readable storage medium.
[0015] The first aspect of the present invention provides a method for detecting changes in remote sensing images based on deep learning, comprising: step S1: acquiring a dataset of remote sensing images; wherein each image in the dataset includes a previous time-phase remote sensing image, a subsequent time-phase remote sensing image, and a corresponding building change label image; step S2: dividing the dataset of remote sensing images into a training set, a validation set, and a test set according to a preset ratio; wherein the training set is a labeled training set, and the validation set and the test set are both unlabeled test sets; step S3: preprocessing the dataset; step S4: constructing a self-attention network model of Dual-Stream Swin-Transformer HRNet; the self-attention network model includes: a BackBone module, a Dual-Stream Swin-Transformer module, and a PredictHead module; the BackBone module includes: a previous time-phase remote sensing image receiving module, a subsequent time-phase remote sensing image receiving module, a first HRNet module, and a second HRNet module; the Dual-Stream Swin-Transformer module includes: a first Patch Merging module, a second Patch Merging module, and a second Patch Merging module. The self-attention network model consists of a Merging module, an SW-Block module, a first ConCat module, a second ConCat module, a first W-Block module, a second W-Block module, and a Sub&Abs module; Step S5: Train the self-attention network model based on the preprocessed training set, input the preprocessed validation set into the trained self-attention network model for optimization, save the final optimized self-attention network model, and define the final optimized self-attention network model as the optimal model; Step S6: Input the preprocessed test set into the optimal model to perform change detection on remote sensing images, and obtain the change detection results of the remote sensing images corresponding to the preprocessed test set; wherein, the previous time phase The remote sensing image receiving module is used to receive the pre-temporal remote sensing image corresponding to each image in the preprocessed training set, validation set, or test set; the post-temporal remote sensing image receiving module is used to receive the post-temporal remote sensing image corresponding to each image in the preprocessed training set, validation set, or test set; the first HRNet module is used to extract features from the received pre-temporal remote sensing image to obtain a first feature map; wherein, the first HRNet module uses HRNet as the backbone network; the second HRNet module is used to extract features from the received post-temporal remote sensing image to obtain a second feature map; the second HRNet module uses HRNet as the backbone network; the first Patch Merging module is used to perform spatial compression and semantic information compression operations on the first feature map to obtain a third feature map;The second Patch Merging module is used to perform spatial and semantic information compression operations on the second feature map to obtain a fourth feature map; the SW-Block module is used to receive a feature map C1 obtained by concatenating the third and fourth feature maps, and to extract features from the concatenated feature map C1 to obtain a fifth feature map, which is then used to segment the fifth feature map into two feature maps C of the same size; the first ConCat module is used to concatenate the first feature map and the feature map C to obtain a sixth feature map, and to send the sixth feature map to the first W-Block module; the second ConCat module is used to concatenate the second feature map and the feature map C to obtain a seventh feature map, and to send the seventh feature map to the second W-Block module. -Block module; the first W-Block module and the second W-Block module are used to receive the sixth feature map and the seventh feature map respectively, and to extract features from the sixth feature map and the seventh feature map respectively to obtain the corresponding eighth feature map and ninth feature map respectively; the Sub&Abs module is used to subtract the corresponding positions of the eighth feature map and the ninth feature map to obtain the tenth feature map; wherein, the tenth feature map highlights the changes between the previous and subsequent time-phase remote sensing images corresponding to each image in the preprocessed training set, validation set or test set; the PredictHead module is used to receive the tenth feature map to obtain the change detection results of the remote sensing images corresponding to the preprocessed training set, validation set or test set.
[0016] Preferably, the first Patch Merging module is specifically used for: dividing the first feature map into several non-overlapping fixed-size regions, and flattening each region into a one-dimensional vector to form a corresponding feature matrix; merging adjacent patches into a new patch to reduce the spatial resolution of the first feature map while maintaining the number of channels; and the second Patch Merging module is specifically used for: dividing the second feature map into several non-overlapping fixed-size patches, and flattening each patch into a one-dimensional vector to form a corresponding feature embedding matrix; merging adjacent patches into a new patch to reduce the spatial resolution of the second feature map while maintaining the number of channels.
[0017] Preferably, the SW-Block module specifically includes: a first normalization layer, an SW-MSA attention mechanism module, a second normalization layer, and a first multilayer perceptron; the first multilayer perceptron consists of multiple fully connected layers; wherein, the first normalization layer is used to receive a feature map C1 obtained by concatenating the third feature map and the fourth feature map, and normalize the concatenated feature map C1, and then send the normalized feature map C2 to the SW-MSA attention mechanism module; the SW-MSA attention mechanism module is used to perform a cyclic shift operation on the input feature map C2, and segment the image after the cyclic shift operation into multiple... Non-overlapping local regions are then processed by multi-head self-attention (MSA) to obtain feature map C3. Feature map C3 is then concatenated with feature map C1 to obtain feature map C4, which is then fed to the second normalization layer. The second normalization layer receives feature map C4, normalizes it, and then sends the normalized feature map C5 to the first multilayer perceptron. The first multilayer perceptron receives feature map C5 and extracts features from it to obtain feature map C6, which is then concatenated with feature map C4 to obtain the fifth feature map.
[0018] Preferably, the first W-Block module specifically includes: a third normalization layer, a first W-MSA attention mechanism module, a fourth normalization layer, and a second multilayer perceptron; the second multilayer perceptron consists of multiple fully connected layers; the third normalization layer is used to receive the sixth feature map, normalize the sixth feature map, and then send the normalized feature map C7 to the first W-MSA attention mechanism module; the first W-MSA attention mechanism module is used to segment the input feature map C7 into multiple non-overlapping local regions, and then perform multi-head self-attention in each local region. The force calculation yields a feature map C8 after multi-head self-attention calculation, which is then used to concatenate feature map C8 with the sixth feature map to obtain feature map C9, which is then sent to the fourth normalization layer. The fourth normalization layer receives feature map C9, normalizes it, and then sends the normalized feature map C10 to the second multilayer perceptron. The second multilayer perceptron receives feature map C10 and extracts features from it to obtain feature map C11, which is then concatenated with feature map C9 to obtain the eighth feature map. The second W-Block module specifically includes: a fifth normalization layer, a second W-MSA attention mechanism module, a sixth normalization layer, and a third multilayer perceptron; the third multilayer perceptron consists of multiple fully connected layers; the fifth normalization layer is used to receive the seventh feature map, normalize the seventh feature map, and then send the normalized feature map C12 to the second W-MSA attention mechanism module; the second W-MSA attention mechanism module is used to segment the input feature map C12 into multiple non-overlapping local regions, and then perform multi-head self-attention calculation in each local region. The system calculates and obtains a feature map C13 after multi-head self-attention computation. This feature map C13 is then concatenated with the seventh feature map to obtain a feature map C14, which is then sent to the sixth normalization layer. The sixth normalization layer receives the feature map C14, normalizes it, and then sends the normalized feature map C15 to the third multilayer perceptron. The third multilayer perceptron receives the feature map C15 and extracts features from it to obtain a feature map C16, which is then concatenated with the feature map C14 to obtain the ninth feature map.
[0019] Preferably, the calculation formula for multi-head self-attention in the SW-Block module, the first W-Block module, and the second W-Block module is as follows:
[0020]
[0021] In the formula, Q, K, and V are the Query matrix, Key matrix, and Value matrix, respectively. Q, K, and V are all... d represents the dimensions of the Query and Key matrices; B represents the relative position encoding. M 2 This represents the number of patches within the local window.
[0022] Preferably, the step of preprocessing the dataset includes performing data augmentation operations on the dataset.
[0023] A second aspect of the present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the deep learning-based remote sensing image change detection method as described in any of the above technical solutions.
[0024] The third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the deep learning-based remote sensing image change detection method as described in any of the above technical solutions.
[0025] The beneficial effects of this invention are:
[0026] (1) This invention constructs an innovative self-attention network model of Dual-Stream Swin Transformer HRNet, replacing the backbone network HRNet to enable the model to extract features better, and introduces a shifted window mechanism to construct self-attention. The core idea of this method is to decompose the complex self-attention operation into simpler parts and combine it with the shifted window mechanism to enhance the model's receptive field and modeling ability. Specifically, this method implements the Dual-Stream structure by decomposing the Transformer encoder layer into two sub-layers, namely the Patch Merging layer and the Window-based Multi-head Self-Attention (W-MSA) layer. The image patch fusion layer segments the input feature map into non-overlapping local regions and merges these regions into larger feature map patches. This can improve computational efficiency and capture local information. The W-MSA layer independently applies the multi-head self-attention mechanism within each feature map patch to establish global and local associations. In addition, Dual-Stream Swin Transformer HRNet also introduces a shifted window mechanism, which enhances the model's ability to model long-distance dependencies through cyclic shift operations. Cyclic shifting operations allow each window to access information from its neighboring windows, thereby expanding the model's receptive field. This mechanism helps capture global contextual information and improves modeling capabilities.
[0027] (2) Traditional Transformer models require significant computational and storage resources when processing high-resolution images because they need to perform global self-attention computation on the entire image. This leads to a surge in computational and memory demands, limiting the scalability of Transformer when processing large images. However, the model of this invention employs an innovative attention mechanism, SW-Block and W-Block, to address this issue. It segments the image into multiple spatially overlapping windows and performs self-attention computation within each window, rather than computing the entire image. This method significantly reduces computational and memory requirements, enabling Swin Transformer to efficiently process high-resolution images. By segmenting the image into windows and performing local self-attention computation, Swin Transformer maintains locality when modeling dependencies within the image. Each window only needs to consider its own relationship with neighboring windows, without having to consider every location in the entire image. This locality allows Swin Transformer to better handle large images while maintaining good modeling capabilities while reducing computational and memory demands.
[0028] (3) The improved Patch Fusion image patch fusion mechanism in this paper ensures that the attention of each window is only related to its neighboring windows, further reducing computational complexity. Through this windowing mechanism, the Swin Transformer can process high-resolution visual input with higher efficiency. It does not need to perform global self-attention calculations on the entire image, but only focuses on local dependencies at the window level. This combination of decomposition and windowing mechanism makes the Swin Transformer an ideal choice for processing high-resolution images. Compared with traditional Transformer models, it maintains high accuracy while having higher computational efficiency and lower memory consumption. This enables the Dual-Stream Swin Transformer HRNet to exhibit excellent performance in tasks such as image classification, object detection, and semantic segmentation. Through the windowing mechanism, the Dual-Stream Swin Transformer HRNet can transform complex global self-attention operations into local intra-window self-attention calculations, thereby reducing computational complexity. It only needs to focus on the relative positional information within each window, without having to deal with the absolute positional relationships of the entire image. This mechanism not only improves computational efficiency but also enhances the model's ability to model long-distance dependencies.
[0029] Additional aspects and advantages of the invention will become apparent from the description which follows, or may be learned by practice of the invention. Attached Figure Description
[0030] Figure 1 A schematic flowchart of a deep learning-based remote sensing image change detection method according to an embodiment of the present invention is shown.
[0031] Figure 2 A schematic diagram of the structure of a self-attention network model of Dual-Stream Swin-Transformer HRNet according to an embodiment of the present invention is shown;
[0032] Figure 3 A schematic diagram of the structure of HRNet, a backbone network according to an embodiment of the present invention, is shown.
[0033] Figure 4 A schematic diagram of the structure of the PatchMerging module according to an embodiment of the present invention is shown;
[0034] Figure 5 A schematic diagram of the structure of the SW-Block module and the W-Block module according to an embodiment of the present invention is shown;
[0035] Figure 6 A schematic block diagram of a computer device according to an embodiment of the present invention is shown;
[0036] Figure 7 A schematic diagram showing the result of remote sensing image change detection according to an embodiment of the present invention is illustrated. Detailed Implementation
[0037] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0038] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.
[0039] Figure 1 A schematic flowchart of a deep learning-based remote sensing image change detection method according to an embodiment of the present invention is shown. Figure 1 As shown, the detection method includes:
[0040] Step S1: Obtain the dataset of remote sensing images;
[0041] Step S2: Divide the remote sensing image dataset into a training set, a validation set, and a test set according to a preset ratio;
[0042] Step S3: Preprocess the dataset;
[0043] Step S4: Build a self-attention network model of Dual-Stream Swin-Transformer HRNet;
[0044] Step S5: Train the self-attention network model based on the preprocessed training set, input the preprocessed validation set into the trained self-attention network model for optimization, save the final optimized self-attention network model, and define the final optimized self-attention network model as the optimal model;
[0045] Step S6: Input the preprocessed test set into the optimal model to perform change detection on the remote sensing image, and obtain the change detection results of the remote sensing image corresponding to the preprocessed test set.
[0046] In this embodiment, each image in the remote sensing image dataset includes a previous-temporal remote sensing image, a subsequent-temporal remote sensing image, and a corresponding building change label map. The training set is a labeled training set, while the validation set and test set are both unlabeled test sets. Figure 2As shown, the self-attention network model of Stream Swin-Transformer HRNet includes: BackBone module, Dual-StreamSwin-Transformer module, and PredictHead module; BackBone module includes: pre-temporal remote sensing image receiving module, post-temporal remote sensing image receiving module, first HRNet module and second HRNet module; Dual-StreamSwin-Transformer module includes: first Patch Merging module, second PatchMerging module, SW-Block module, first ConCat module, second ConCat module, first W-Block module, second W-Block module and Sub&Abs module.
[0047] The system includes a pre-temporal remote sensing image receiving module, used to receive the pre-temporal remote sensing image corresponding to each image in the preprocessed training set, validation set, or test set; a post-temporal remote sensing image receiving module, used to receive the post-temporal remote sensing image corresponding to each image in the preprocessed training set, validation set, or test set; a first HRNet module, used to extract features from the received pre-temporal remote sensing images to obtain a first feature map; wherein the first HRNet module uses HRNet as its backbone network; the structure diagram of the HRNet is shown below. Figure 3 As shown; the second HRNet module is used to extract features from the received post-temporal remote sensing image to obtain a second feature map; the second HRNet module uses HRNet as its backbone network; the structure of this HRNet is shown in the diagram. Figure 3As shown; the first Patch Merging module is used to perform spatial and semantic information compression operations on the first feature map to obtain a third feature map; the second Patch Merging module is used to perform spatial and semantic information compression operations on the second feature map to obtain a fourth feature map; the SW-Block module is used to receive the feature map C1 obtained by concatenating the third and fourth feature maps, and to extract features from the concatenated feature map C1 to obtain a fifth feature map, which is then used to divide the fifth feature map into two feature maps C of the same size; the first ConCat module is used to concatenate the first feature map and the feature map C to obtain a sixth feature map, and to send the sixth feature map to the first SW-Block module; the second ConCat module is used to concatenate the second feature map and the feature map C to obtain a seventh feature map, and to send the seventh feature map to the second SW-Block module. The W-Block module comprises a first W-Block module and a second W-Block module, used to receive the sixth feature map and the seventh feature map respectively, and to extract features from the sixth feature map and the seventh feature map respectively to obtain the corresponding eighth feature map and ninth feature map; the Sub&Abs module is used to subtract the corresponding positions of the eighth feature map and the ninth feature map to obtain the tenth feature map; wherein the tenth feature map highlights the changes between the preceding and following time-phase remote sensing images corresponding to each image in the preprocessed training set, validation set, or test set; the PredictHead module is used to receive the tenth feature map to obtain the change detection results of the remote sensing images corresponding to the preprocessed training set, validation set, or test set.
[0048] In this embodiment, the present invention constructs an innovative self-attention network model of Dual-Stream Swin Transformer HRNet. Self-attention is constructed by decomposing the traditional Transformer encoder layer into smaller building blocks and introducing a shifted window mechanism. Traditional Transformer models require significant computational and storage resources when processing high-resolution images because they need to perform self-attention calculations on the entire image. However, SwinTransformer significantly reduces computational and memory requirements by segmenting the image into multiple spatially overlapping windows and performing self-attention calculations within each window. Furthermore, the introduced shifted window mechanism ensures that the attention of each window is only related to its neighboring windows, further reducing computational complexity. This combination of decomposition and windowing mechanisms makes Swin Transformer an ideal choice for processing high-resolution visual inputs, maintaining high accuracy while achieving higher computational efficiency and lower memory consumption. In a specific embodiment, the self-attention network model of the present invention mainly consists of three parts: Backbone, Dual-Stream Swin-TransformerHRNet, and PredictHead. The preceding temporal image (Image_A) and the following temporal image (Image_B) undergo feature extraction via a backbone network (HRNet-18). These features are then processed by a dual-stream Swin-Transformer, primarily consisting of Patch Merging, the sw-block, and the w-block. Patch Merging compresses spatial and semantic information before the sw-block begins. The two feature maps are then concatenated and passed through the sw-block, and finally segmented into two identical feature maps. These two segmented feature maps are then concatenated with the feature maps output from the backbone and fed into the w-block. In the w-block, a subtraction operation is performed between these two sets of feature maps to highlight the changes between the preceding and following temporal images. Finally, the resulting feature maps are used for prediction via a predictive head to obtain the final change detection result.
[0049] Furthermore, this invention designs a special backbone network structure, HRNet. HRNet (High-Resolution Network) is a deep convolutional neural network used for tasks such as image segmentation and pose estimation. Its design is inspired by the concept of high-resolution image processing, which aims to improve image processing performance by utilizing image information from multiple resolutions. HRNet introduces this idea into neural networks, using feature maps from multiple resolutions to construct the network for better processing of high-resolution images. The core idea of HRNet is to construct the network using feature maps from multiple resolutions. Specifically, it employs a structure called "High-Resolution," which acquires more complete feature information by constructing feature pyramids at different resolutions. The HRNet network structure mainly consists of two parts: High-Resolution Representation Learning and High-Resolution Fusion. The former is used to extract multi-scale feature representations by constructing feature pyramids to obtain more comprehensive feature information; the latter is used to fuse feature maps from different resolutions to obtain better feature representations. Experiments have shown that, compared to ResNet and EfficientNet, using HRNet as the backbone network has stronger feature representation capabilities. HRNet is based on a multi-branch convolutional neural network structure, which enables information exchange between different branches of the network, better fusing features from different levels and thus improving feature representation capabilities. In contrast, ResNet and EfficientNet are based on single-branch network structures, which struggle to fully fuse features from different levels and perform poorly when processing some high-resolution images. Therefore, this invention selects HRNet as the backbone network.
[0050] Furthermore, this invention constructs a novel extension strategy. It utilizes an improved PatchFusion image patch fusion mechanism, inspired by PatchMerging.
[0051] Furthermore, this invention introduces SW-Block and W-Block to construct a structure that balances efficiency and performance. Traditional Transformers compute attention (Multi-headed Self-attention, MSA) globally, resulting in very high computational complexity. The Swing Transformer, however, employs both Window Multi-headed Self-attention (W-MSA) and Sliding Window Multi-headed Self-attention (SW-MSA). This invention improves upon this algorithm by separating the SW-MSA and W-MSA blocks. Based on this design, these blocks are named SW-BLOCK and W-BLOCK, respectively.
[0052] In one embodiment of the present invention, the first Patch Merging module is specifically used to: divide the first feature map into several non-overlapping fixed-size regions, and flatten each region into a one-dimensional vector to form a corresponding feature matrix; merge adjacent patches into a new patch to reduce the spatial resolution of the first feature map while maintaining the number of channels unchanged; and the second Patch Merging module is specifically used to: divide the second feature map into several non-overlapping fixed-size patches, and flatten each patch into a one-dimensional vector to form a corresponding feature embedding matrix; merge adjacent patches into a new patch to reduce the spatial resolution of the second feature map while maintaining the number of channels unchanged.
[0053] In this embodiment, an improved Patch Fusion image patch fusion mechanism is used, which is based on the idea of the PatchMerging module. A schematic diagram of the PatchMerging module is shown below. Figure 4As shown, the implementation of this PatchMerging module involves dividing the input image into several non-overlapping, fixed-size patches and flattening each patch into a one-dimensional vector, forming a feature embedding matrix. Then, adjacent patches are fused into a new patch, thereby reducing spatial resolution and increasing the number of channels. This fusion process is implemented using a fully connected layer to mix adjacent patch embeddings together. By progressively reducing the spatial resolution of the feature map through Patch Merging, deep neural networks can better understand the image. However, regarding the size of the feature map, unlike Patch Merging which quadruples the channel dimension and halves the spatial resolution, this invention's Patch Fusion reduces the spatial resolution of the feature map by half while maintaining the same number of channels. Experiments have demonstrated that Patch Fusion is a simple and effective downsampling method that can be used in deep neural networks to reduce spatial resolution while keeping the number of channels constant, enabling image understanding at higher layers and thus improving feature representation and model performance.
[0054] In one embodiment of the present invention, the SW-Block module specifically includes: a first normalization layer, an SW-MSA attention mechanism module, a second normalization layer, and a first multilayer perceptron; the first multilayer perceptron is composed of multiple fully connected layers; wherein, the first normalization layer is used to receive a feature map C1 obtained by concatenating the third feature map and the fourth feature map, and to normalize the concatenated feature map C1, and then send the normalized feature map C2 to the SW-MSA attention mechanism module; the SW-MSA attention mechanism module is used to perform a cyclic shift operation on the input feature map C2, and to divide the image after the cyclic shift operation into... The system is divided into multiple non-overlapping local regions, and multi-head self-attention is calculated in each local region to obtain a feature map C3 after multi-head self-attention calculation. This feature map C3 is then concatenated with the feature map C1 to obtain a feature map C4, which is then sent to the second normalization layer. The second normalization layer receives the feature map C4, normalizes it, and then sends the normalized feature map C5 to the first multilayer perceptron. The first multilayer perceptron receives the feature map C5 and extracts features from it to obtain a feature map C6, which is then concatenated with the feature map C4 to obtain the fifth feature map.
[0055] This embodiment describes the structure of the SW-Block. For example... Figure 5As shown, the SW-Block module specifically includes: two normalization layers, an SW-MSA attention mechanism module, and a multilayer perceptron. Because the subsequent W-Block module has the same structure as the SW-Block module, to distinguish it from the W-Block module, this invention sets the SW-Block module to specifically include: a first normalization layer, an SW-MSA attention mechanism module, a second normalization layer, and a first multilayer perceptron. The core idea of the W-MSA module is to segment the input image into multiple non-overlapping local regions (windows), and then perform multi-head self-attention operations within each region. In this way, the model can focus on information within local regions, thereby improving computational efficiency. The SW-MSA module introduces a cyclic shift operation on top of the W-MSA module to improve the model's ability to capture long-range dependencies. Specifically, SW-MSA first performs a cyclic shift on the input feature map, and then performs the operations to be performed by the W-MSA module. In this way, each window can obtain information from its neighboring windows, thus giving the model a larger receptive field. It is precisely because of these features of the W-MSA and SW-MSA modules that the Dual-Stream Swin-Transformer can effectively capture local and global information while maintaining high computational efficiency.
[0056] In one embodiment of the present invention, the first W-Block module specifically includes: a third normalization layer, a first W-MSA attention mechanism module, a fourth normalization layer, and a second multilayer perceptron; the second multilayer perceptron is composed of multiple fully connected layers; the third normalization layer is used to receive the sixth feature map, perform normalization processing on the sixth feature map, and then send the normalized feature map C7 to the first W-MSA attention mechanism module; the first W-MSA attention mechanism module is used to segment the input feature map C7 into multiple non-overlapping local regions, and then perform processing on each local region. Multi-head self-attention is performed to obtain feature map C8, which is then concatenated with the sixth feature map to form feature map C9, which is then fed to the fourth normalization layer. The fourth normalization layer receives feature map C9, normalizes it, and then sends the normalized feature map C10 to the second multilayer perceptron. The second multilayer perceptron receives feature map C10 and extracts features from it to obtain feature map C11, which is then concatenated with feature map C9 to form the eighth feature map. The second W-Block module specifically includes: a fifth normalization layer, a second W-MSA attention mechanism module, a sixth normalization layer, and a third multilayer perceptron; the third multilayer perceptron consists of multiple fully connected layers; the fifth normalization layer is used to receive the seventh feature map, normalize the seventh feature map, and then send the normalized feature map C12 to the second W-MSA attention mechanism module; the second W-MSA attention mechanism module is used to segment the input feature map C12 into multiple non-overlapping local regions, and then perform multi-head self-attention in each local region. The calculation yields a feature map C13 after multi-head self-attention calculation, which is then used to concatenate the feature map C13 with the seventh feature map to obtain a feature map C14, which is then sent to the sixth normalization layer. The sixth normalization layer receives the feature map C14, performs normalization processing on the feature map C14, and then sends the normalized feature map C15 to the third multilayer perceptron. The third multilayer perceptron receives the feature map C15 and performs feature extraction on the feature map C15 to obtain a feature map C16, which is then concatenated with the feature map C14 to obtain the ninth feature map.
[0057] This embodiment describes the structure of the W-Block. For example... Figure 5As shown, the W-Block module specifically includes: two normalization layers, a W-MSA attention mechanism module, and a multilayer perceptron. Because the SW-Block module has the same structure as the W-Block module, to differentiate it from the SW-Block module, the first W-Block module is configured to include: a third normalization layer, a first W-MSA attention mechanism module, a fourth normalization layer, and a second multilayer perceptron; the second multilayer perceptron consists of multiple fully connected layers. To further differentiate the structures of the first and second W-Block modules, the second W-Block module is configured to include: a fifth normalization layer, a second W-MSA attention mechanism module, a sixth normalization layer, and a third multilayer perceptron. The core idea of the W-MSA module is to segment the input image into multiple non-overlapping local regions (windows), and then perform multi-head self-attention operations within each region. In this way, the model can focus on information within local regions, thereby improving computational efficiency. It is precisely because of these characteristics of the W-MSA module that the Dual-Stream Swin-Transformer can effectively capture local and global information while maintaining high computational efficiency.
[0058] In one embodiment of the present invention, the multi-head self-attention calculation formula in the SW-Block module, the first W-Block module, and the second W-Block module is as follows:
[0059]
[0060] In the formula, Q, K, and V are the Query matrix, Key matrix, and Value matrix, respectively. Q, K, and V are all... d represents the dimensions of the Query and Key matrices; B represents the relative position encoding. M 2 This represents the number of patches within the local window.
[0061] In one embodiment of the present invention, the step of preprocessing the dataset includes performing data augmentation operations on the dataset.
[0062] like Figure 6 As shown, a computer device 600 includes: a memory 602, a processor 604, and a computer program stored in the memory 602 and executable on the processor 604. When the processor 604 executes the computer program, it implements the steps of the deep learning-based remote sensing image change detection method as described in any of the above embodiments.
[0063] The computer device 600 provided by the present invention, when the processor 604 executes a computer program, constructs an innovative self-attention network model of Dual-Stream Swin Transformer HRNet, which constructs self-attention by decomposing the traditional Transformer encoder layer into smaller building blocks and introducing a shifted windows mechanism.
[0064] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the deep learning-based remote sensing image change detection method as described in any of the above embodiments.
[0065] The computer-readable storage medium provided by this invention constructs an innovative self-attention network model of Dual-Stream Swin Transformer HRNet when the computer program is executed by the processor. This model constructs self-attention by decomposing the traditional Transformer encoder layer into smaller building blocks and introducing a shifted windows mechanism.
[0066] The technical solution of the present invention will be illustrated below with a specific embodiment.
[0067] The specific implementation steps of the deep learning-based remote sensing image change detection method in this embodiment are as follows:
[0068] (1) Download the dataset file from the LEVIR-CD dataset. Since the dataset is crucial for deep learning, and remote sensing image change detection requires a large amount of data for training, this invention uses LEVIR-CD, which was specifically created to solve the problem of building change detection in remote sensing images. This dataset contains 1000 samples, of which 637 samples are used as a labeled training set, and the other 363 samples are used as unlabeled validation and test sets. Each sample includes a previous-phase remote sensing image, a subsequent-phase remote sensing image, and a corresponding building change label map. The remote sensing images in the LEVIR-CD dataset are 1024×1024 in size and have a pixel resolution of 0.5 meters. The label map is the same size as the image, with a pixel value of 0 or 255, where 0 indicates no building change and 255 indicates building change.
[0069] This dataset was created based on the open-source invention of the LEVIR team at Beihang University. It aims to provide a standard dataset for research in the field of building change detection in remote sensing images. The LEVIR-CD dataset is representative and valuable, and can help researchers better explore and solve the problem of building change detection in remote sensing images.
[0070] By using the LEVIR-CD dataset, this invention enables the evaluation of the performance of the proposed algorithm on building change detection tasks, and allows for comparison and analysis with other methods. This helps verify the effectiveness of the algorithm and provides a benchmark for future research on building change detection in remote sensing images.
[0071] (2) Preprocessing the dataset. Since the Dual-Stream Swin-Transformer HRNet self-attention network model did not use a pre-trained model and the training dataset was limited, we used standard data augmentation techniques: randomly adjusting image size between 0.5 and 2, horizontally flipping images, and changing image saturation, brightness, and contrast. By using data augmentation techniques to expand the dataset, we could more effectively train the network's feature extraction network.
[0072] To enhance the data, this embodiment employs various data augmentation strategies. These include random cropping with larger crop sizes to increase the size of the images input to the network; random flipping and rotation to enhance the robustness of the model; and random blurring, random swapping of A and B images, and color jitter to increase data diversity and the model's generalization ability.
[0073] Specifically, RandomCrop is used for random image cropping, RandomFlipOrRotation for random image flipping and rotation, RandomBlur for random image blurring, RandomSwap for random swapping of adjacent images, and RandomDistort for color jittering. This combination of data augmentation strategies effectively improves the model's robustness and generalization ability, resulting in better performance.
[0074] (3) Construct a self-attention network model of Dual-Stream Swin-Transformer HRNet. This self-attention network model includes: a Backbone module, a Dual-Stream Swin-Transformer module, and a PredictHead module; the Backbone module includes: a pre-temporal remote sensing image receiving module, a post-temporal remote sensing image receiving module, a first HRNet module, and a second HRNet module; the Dual-Stream Swin-Transformer module includes: a first Patch Merging module, a second Patch... The system comprises a Merging module, an SW-Block module, a first ConCat module, a second ConCat module, a first W-Block module, a second W-Block module, and a Sub&Abs module; wherein, a pre-temporal remote sensing image receiving module is used to receive the pre-temporal remote sensing image corresponding to each image in the preprocessed training set, validation set, or test set; a post-temporal remote sensing image receiving module is used to receive the post-temporal remote sensing image corresponding to each image in the preprocessed training set, validation set, or test set; a first HRNet module is used to extract features from the received pre-temporal remote sensing image to obtain a first feature map; wherein, the first HRNet module uses HRNet as the backbone network; a second HRNet module is used to extract features from the received post-temporal remote sensing image to obtain a second feature map; the second HRNet module uses HRNet as the backbone network; a first Patch Merging module is used to perform spatial compression and semantic information compression operations on the first feature map to obtain a third feature map; a second ... The Merging module performs spatial and semantic information compression operations on the second feature map to obtain a fourth feature map; the SW-Block module receives a feature map C1 obtained by concatenating the third and fourth feature maps, and extracts features from the concatenated feature map C1 to obtain a fifth feature map, which is then used to segment the fifth feature map into two feature maps C of the same size; the first ConCat module concatenates the first feature map and the feature map C to obtain a sixth feature map, and sends the sixth feature map to the first W-Block module; the second ConCat module concatenates the second feature map and the feature map C to obtain a seventh feature map, and sends the seventh feature map to the second W-Block module; the first W-Block module and the second W-Block module receive the sixth feature map and the seventh feature map respectively, and extract features from the sixth feature map and the seventh feature map respectively to obtain corresponding eighth and ninth feature maps;The Sub&Abs module is used to subtract corresponding positions from the eighth and ninth feature maps to obtain the tenth feature map. The tenth feature map highlights the changes between the preceding and following time-phase remote sensing images corresponding to each image in the preprocessed training, validation, or test sets. The PredictHead module receives the tenth feature map to obtain the change detection results for the remote sensing images corresponding to the preprocessed training, validation, or test sets.
[0075] Specifically, the first Patch Merging module is used to: divide the first feature map into several non-overlapping fixed-size regions, and flatten each region into a one-dimensional vector to form a corresponding feature matrix; and merge adjacent patches into a new patch to reduce the spatial resolution of the first feature map while maintaining the number of channels. The second Patch Merging module is used to: divide the second feature map into several non-overlapping fixed-size patches, and flatten each patch into a one-dimensional vector to form a corresponding feature embedding matrix; and merge adjacent patches into a new patch to reduce the spatial resolution of the second feature map while maintaining the number of channels. The SW-Block module specifically includes: a first normalization layer, an SW-MSA attention mechanism module, a second normalization layer, and a first multilayer perceptron; the first multilayer perceptron consists of multiple fully connected layers; wherein, the first normalization layer is used to receive a feature map C1 obtained by concatenating the third feature map and the fourth feature map, normalize the concatenated feature map C1, and then send the normalized feature map C2 to the SW-MSA attention mechanism module; the SW-MSA attention mechanism module is used to perform a cyclic shift operation on the input feature map C2, and segment the image after the cyclic shift operation into multiple non-overlapping segments. The multi-head self-attention algorithm is used to calculate the feature map C3 in each local region, and then the feature map C4, which is obtained by concatenating the feature map C3 with the feature map C1, is sent to the second normalization layer. The second normalization layer receives the feature map C4, performs normalization processing on the feature map C4, and then sends the normalized feature map C5 to the first multilayer perceptron. The first multilayer perceptron receives the feature map C5 and performs feature extraction on the feature map C5 to obtain the feature map C6, which is then concatenated with the feature map C4 to obtain the fifth feature map.The first W-Block module specifically includes: a third normalization layer, a first W-MSA attention mechanism module, a fourth normalization layer, and a second multilayer perceptron; the second multilayer perceptron consists of multiple fully connected layers; the third normalization layer is used to receive the sixth feature map, normalize the sixth feature map, and then send the normalized feature map C7 to the first W-MSA attention mechanism module; the first W-MSA attention mechanism module is used to segment the input feature map C7 into multiple non-overlapping local regions, and then perform multi-head self-attention calculation in each local region. The multi-head self-attention calculation yields a feature map C8, which is then concatenated with the sixth feature map to obtain a feature map C9, which is then fed to the fourth normalization layer. The fourth normalization layer receives feature map C9, normalizes it, and then sends the normalized feature map C10 to the second multilayer perceptron. The second multilayer perceptron receives feature map C10 and extracts features from it to obtain feature map C11, which is then concatenated with feature map C9 to obtain the eighth feature map. The two W-Block modules specifically include: a fifth normalization layer, a second W-MSA attention mechanism module, a sixth normalization layer, and a third multilayer perceptron; the third multilayer perceptron consists of multiple fully connected layers; the fifth normalization layer is used to receive the seventh feature map, normalize the seventh feature map, and then send the normalized feature map C12 to the second W-MSA attention mechanism module; the second W-MSA attention mechanism module is used to segment the input feature map C12 into multiple non-overlapping local regions, and then perform multi-head self-attention calculation in each local region. The feature map C13, after multi-head self-attention calculation, is obtained. This feature map C13 is then concatenated with the seventh feature map to form feature map C14, which is then sent to the sixth normalization layer. The sixth normalization layer receives feature map C14, normalizes it, and then sends the normalized feature map C15 to the third multilayer perceptron. The third multilayer perceptron receives feature map C15 and extracts features from it to obtain feature map C16, which is then concatenated with feature map C14 to obtain the ninth feature map. The multi-head self-attention calculation formula in the SW-Block module, the first W-Block module, and the second W-Block module is:
[0076]
[0077] In the formula, Q, K, and V are the Query matrix, Key matrix, and Value matrix, respectively. Q, K, and V are all... d represents the dimensions of the Query and Key matrices; B represents the relative position encoding. M 2 This represents the number of patches within the local window.
[0078] In change detection, the output head, specifically the Sub&Abs module, is a crucial component. It generates a change map to represent the differences between two images. The output head is designed based on feature maps generated by the w-block, which already contain information about the differences between the two images. Typically, the output head employs the concept of a Fully Convolutional Network (FCN) to upsample and convolve the feature maps, preserving more spatial and semantic information. The main goal of the output head is to subtract corresponding positions from the two images to generate the change map. Each pixel value in the change map represents the degree of difference between the two images at that location, and can be used to analyze changes in buildings in remote sensing images, detect the appearance and disappearance of targets, and perform other tasks. The change map generated by the output head provides a visual understanding of the changes between the two images. Larger pixel difference values indicate significant changes at that location, while smaller difference values indicate less noticeable changes. The change map offers a visual way to analyze and understand patterns and trends in image changes. Therefore, the output head plays an important role in change detection. By processing the feature map generated by the w-block, it generates a change map, providing us with an intuitive understanding and analytical ability of image changes.
[0079] (4) The present invention adopts a twin network structure. In order to better train the entire network, each of the two branches of the network adopts a loss function.
[0080] (5) Obtaining the optimal model. After each training round, the preprocessed validation set is input into the trained self-attention network model for optimization. The final optimized self-attention network model is saved and defined as the optimal model. The optimal model is named using the training round number to more intuitively observe whether overfitting has occurred. After the final training, we will obtain the final 1000-round trained model and the optimal model.
[0081] During training, a learning rate scheduler with a fixed decay rate at equal intervals was used, and different decay strategies were explored. A warmup operation was also added to better optimize model performance. This experiment set the learning rate to 0.0004, the batch size to 32, and used the AdamW optimizer. During training, the Momentum optimizer was used, combined with linear learning rate decay and the EMA method. The training process lasted 100 epochs to fully train the model and improve its performance and generalization ability.
[0082] (6) Input the preprocessed test set into the optimal model to detect changes in remote sensing images, and obtain the change detection results of the remote sensing images corresponding to the preprocessed test set. For example... Figure 7 The diagram shown illustrates the results of remote sensing image change detection in this embodiment.
[0083] (7) When using deep learning models for data classification, the present invention uses the following performance metrics to measure the classification effect, namely accuracy, recall, IOU, F1 score, OACC and Kappa coefficient.
[0084] In binary classification problems, the set of samples that are originally positive and are also classified as positive by the classifier is denoted as TP (true positive); the set of samples that are originally positive but are classified as negative by the classifier is denoted as FN (false negative); the set of samples that are originally negative but are classified as positive by the classifier is denoted as FP (false positive); and the set of samples that are originally negative and are also classified as negative by the classifier is denoted as TN (true negative).
[0085] Precision and recall are statistical values, and their respective calculation methods are as follows:
[0086]
[0087]
[0088] The F1 score is the harmonic mean of precision and recall, commonly used for evaluating imbalanced datasets. Its formula is defined as follows:
[0089]
[0090] Intersection over Union (IOU) is a metric used to evaluate the performance of object detection and segmentation. It measures the accuracy of an algorithm by comparing the degree of overlap between predicted results and ground truth values. The formula for calculating IOU is as follows:
[0091]
[0092] Here, A and B represent the regions of the predicted result and the true value, respectively. Intersection(A, B) represents their intersection, and Union(A, B) represents their union. The IOU value is usually between 0 and 1, with a higher value indicating that the predicted result is closer to the true value. When IOU equals 1, the predicted result is exactly the same as the true value; when IOU equals 0, the predicted result and the true value have no overlap.
[0093] OACC (Overall Accuracy) is a metric used to evaluate the performance of a classification model. It represents the weighted average of the model's classification accuracy across all classes. The formula for calculating OACC is as follows:
[0094]
[0095] Where n represents the number of categories, TP i N represents the number of correctly classified samples in the i-th class. i This represents the total number of samples in class i. The OACC value is usually between 0 and 1, with a higher value indicating higher classification accuracy.
[0096] The Kappa coefficient is a metric used to evaluate the performance of classification models. It takes into account the influence of random factors in the classification results, thus more accurately assessing the model's classification accuracy. The formula for calculating the Kappa coefficient is as follows:
[0097]
[0098] Where P0 represents the accuracy of the model, P e Pe represents the expected accuracy of random factors in the classification results. The formula for calculating Pe is as follows:
[0099]
[0100] Among them, T i O represents the number of samples whose true class is the i-th class. i This represents the number of samples predicted by the model to be of class i, and N represents the total number of samples.
[0101] The Kappa coefficient ranges from -1 to 1, with a higher value indicating better classification accuracy. A Kappa coefficient of 1 indicates that the model's classification result is completely correct; a Kappa coefficient of 0 indicates that the model's classification result is the same as the random result; and a Kappa coefficient less than 0 indicates that the model's classification result is poor.
[0102] (7) This embodiment compares different types of backbone networks. The experimental results of different types of backbone networks are shown in Table 1.
[0103] Table 1. Backbone Network Experiment Results
[0104] Resnet18 0.8888 0.8897 0.9561 0.8642 Resnet64 0.8976 0.8928 0.9633 0.8851 Resnext18 0.8948 0.8930 0.9664 0.8854 SwinTransformer-s 0.9178 0.9214 0.9936 0.9108 Hmet18 0.9176 0.9151 0.9926 0.9112
[0105] Table 1 shows that this embodiment compared different types of backbone networks. Based on a comprehensive comparison and discussion of the experimental results, the following conclusions are drawn: SwinTransformer-s and Hmet18 backbone network models performed well on the evaluation metrics, achieving high scores. Specifically, SwinTransformer-s performed excellently on IOU, F1, and OACC metrics, especially achieving high accuracy on OACC. Hrnet18 achieved high scores on IOU and Kappa metrics, demonstrating good object boundary matching and consistency. Although SwinTransformer-s performed well on the evaluation metrics, its inference speed is relatively slow, which may limit its practical application in large-scale datasets or real-time applications. In contrast, the Hrnet18 model not only performed well on the evaluation metrics but also had a faster inference speed. Furthermore, ResNet64 and Resnext18 models also demonstrated good performance and, compared to SwinTransformer-s and Hmet18 models, have faster inference speeds, making them suitable for processing large amounts of data.
[0106] (7) This embodiment compares the network model of the present invention with other networks. The comparison results of the network model of the present invention with other networks are shown in Table 2.
[0107] Table 2 Comparison results of this invention with other networks
[0108] STANet 0.8874 0.8819 0.9608 0.8765 SNUNet 0.8994 0.8956 0.9706 0.8851 DSIFN 0.9064 0.9004 0.9652 0.8846 Bit 0.9156 0.9101 0.9866 0.8998 Ours 0.9178 0.9151 0.9926 0.9112
[0109] As shown in Table 1, compared with other CD models, the Bit model and the network model of this invention perform exceptionally well in terms of IOU, F1, and OACC metrics, exhibiting higher evaluation scores compared to other models. Particularly noteworthy is the high scores achieved by the Bit model on OACC and Kappa metrics, demonstrating excellent performance in overall accuracy and consistency. Furthermore, the model of this invention slightly surpasses the Bit model in IOU, exhibiting better object boundary matching and overlap. The STANet and SNUNet models also perform well in evaluation metrics, but their performance is slightly inferior to that of the Bit and Ours models.
[0110] The results of comparative experiments on the network model of this invention with other classic network models of the same type show that the Dual-Stream Swin Transformer HRNet in this invention effectively solves the computational and memory challenges of traditional Transformers when processing high-resolution images through innovative decomposition and windowing mechanisms, providing a new solution for efficiently processing large-scale visual data.
[0111] This invention proposes a deep learning-based remote sensing image change detection method that can quantitatively analyze and determine the characteristics and processes of land surface changes from two remote sensing images from different periods. Traditional change detection methods generally analyze changes using the difference between two images, but suffer from sensitivity to noise, occlusion, and complex changes. To address these issues, this invention proposes a change detection method based on a Dual-Stream Swin-Transformer network. This method employs a dual-stream architecture and Swin blocks to better capture change information between two images and improves feature extraction capabilities and accuracy. Compared to traditional change detection methods, the proposed method exhibits better adaptability and higher accuracy, making it more suitable for change detection tasks in complex scenarios. Experiments demonstrate that this method significantly improves performance compared to other change detection networks, demonstrating better feature extraction capabilities, greater adaptability, and higher accuracy. It is better suited for updating geographic data, assessing and predicting disaster development trends, monitoring land cover / use, and next-generation intelligent Earth observation satellites.
[0112] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for detecting changes in remote sensing images based on deep learning, characterized in that, include: Step S1: Obtain the dataset of remote sensing images; wherein, each image in the dataset of remote sensing images includes a previous time-phase remote sensing image, a subsequent time-phase remote sensing image, and a corresponding building change label map; Step S2: Divide the dataset of remote sensing images into a training set, a validation set, and a test set according to a preset ratio; wherein the training set is a labeled training set, and the validation set and the test set are both unlabeled test sets; Step S3: Preprocess the dataset; Step S4: Construct a self-attention network model of Dual-Stream Swin-Transformer HRNet; the self-attention network model includes: a Backbone module, a Dual-Stream Swin-Transformer module, and a PredictHead module; the Backbone module includes: a pre-temporal remote sensing image receiving module, a post-temporal remote sensing image receiving module, a first HRNet module, and a second HRNet module; the Dual-Stream Swin-Transformer module includes: a first PatchMerging module, a second Patch Merging module, a SW-Block module, a first ConCat module, a second ConCat module, a first W-Block module, a second W-Block module, and a Sub&Abs module; Step S5: Train the self-attention network model based on the preprocessed training set, input the preprocessed validation set into the trained self-attention network model for optimization, save the final optimized self-attention network model, and define the final optimized self-attention network model as the optimal model; Step S6: Input the preprocessed test set into the optimal model to perform change detection on the remote sensing image, and obtain the change detection result of the remote sensing image corresponding to the preprocessed test set; The pre-temporal remote sensing image receiving module is used to receive the pre-temporal remote sensing image corresponding to each image in the preprocessed training set, validation set, or test set. The post-temporal remote sensing image receiving module is used to receive the post-temporal remote sensing image corresponding to each image in the preprocessed training set, validation set, or test set. The first HRNet module is used to extract features from the received pre-temporal remote sensing image to obtain a first feature map; wherein, the first HRNet module uses HRNet as the backbone network; The second HRNet module is used to extract features from the received post-temporal remote sensing image to obtain a second feature map; the second HRNet module uses HRNet as its backbone network; The first Patch Merging module is used to perform spatial compression and semantic information compression operations on the first feature map to obtain the third feature map; The second Patch Merging module is used to perform spatial compression and semantic information compression operations on the second feature map to obtain the fourth feature map; The SW-Block module is used to receive a feature map C1 obtained by concatenating the third feature map and the fourth feature map, and to extract features from the concatenated feature map C1 to obtain a fifth feature map, which is then used to divide the fifth feature map into two feature maps C of the same size. The first ConCat module is used to concatenate the first feature map and the feature map C to obtain a sixth feature map, and send the sixth feature map to the first W-Block module; The second ConCat module is used to concatenate the second feature map and the feature map C to obtain a seventh feature map, and send the seventh feature map to the second W-Block module; The first W-Block module and the second W-Block module are used to receive the sixth feature map and the seventh feature map respectively, and to extract features from the sixth feature map and the seventh feature map respectively to obtain the corresponding eighth feature map and the ninth feature map respectively; The Sub&Abs module is used to subtract corresponding positions of the eighth feature map and the ninth feature map to obtain the tenth feature map; wherein the tenth feature map highlights the changes between the previous and subsequent time-phase remote sensing images corresponding to each image in the preprocessed training set, validation set, or test set. The PredictHead module is used to receive the tenth feature map to obtain the change detection results of the remote sensing images corresponding to the preprocessed training set, validation set, or test set.
2. The remote sensing image change detection method based on deep learning according to claim 1, characterized in that, The first Patch Merging module is specifically used for: The first feature map is divided into several non-overlapping regions of fixed size, and each region is flattened into a one-dimensional vector to form a corresponding feature matrix; Adjacent patches are then merged into new patches to reduce the spatial resolution of the first feature map while maintaining the same number of channels; and The second Patch Merging module is specifically used for: The second feature map is divided into several non-overlapping fixed-size patches, and each patch is flattened into a one-dimensional vector to form a corresponding feature embedding matrix; Adjacent patches are then merged into new patches to reduce the spatial resolution of the second feature map while maintaining the same number of channels.
3. The remote sensing image change detection method based on deep learning according to claim 1, characterized in that, The SW-Block module specifically includes: a first normalization layer, an SW-MSA attention mechanism module, a second normalization layer, and a first multilayer perceptron; the first multilayer perceptron is composed of multiple fully connected layers; The first normalization layer is used to receive the feature map C1 after the third feature map and the fourth feature map are concatenated, and to normalize the concatenated feature map C1. Then, the normalized feature map C2 is sent to the SW-MSA attention mechanism module. The SW-MSA attention mechanism module is used to perform a cyclic shift operation on the input feature map C2, segment the image after the cyclic shift operation into multiple non-overlapping local regions, and then perform multi-head self-attention calculation in each local region to obtain a feature map C3 after multi-head self-attention calculation, so that the feature map C4 after the feature map C3 and the feature map C1 are concatenated can be sent to the second normalization layer. The second normalization layer is used to receive feature map C4, normalize feature map C4, and then send the normalized feature map C5 to the first multilayer perceptron. The first multilayer perceptron is used to receive the feature map C5 and extract features from the feature map C5 to obtain feature map C6, which is then used to concatenate feature map C4 and feature map C6 to obtain the fifth feature map.
4. The remote sensing image change detection method based on deep learning according to claim 3, characterized in that, The first W-Block module specifically includes: a third normalization layer, a first W-MSA attention mechanism module, a fourth normalization layer, and a second multilayer perceptron; the second multilayer perceptron is composed of multiple fully connected layers; The third normalization layer is used to receive the sixth feature map, normalize the sixth feature map, and then send the normalized feature map C7 to the first W-MSA attention mechanism module. The first W-MSA attention mechanism module is used to divide the input feature map C7 into multiple non-overlapping local regions, and then perform multi-head self-attention calculation in each local region to obtain the feature map C8 after multi-head self-attention calculation, so that the feature map C8 and the sixth feature map are concatenated to the fourth normalization layer. The fourth normalization layer is used to receive the feature map C9, normalize the feature map C9, and then send the normalized feature map C10 to the second multilayer perceptron. The second multilayer perceptron is used to receive the feature map C10 and extract features from the feature map C10 to obtain feature map C11, which is then concatenated with feature map C9 to obtain the eighth feature map. The second W-Block module specifically includes: a fifth normalization layer, a second W-MSA attention mechanism module, a sixth normalization layer, and a third multilayer perceptron; the third multilayer perceptron is composed of multiple fully connected layers. The fifth normalization layer is used to receive the seventh feature map, normalize the seventh feature map, and then send the normalized feature map C12 to the second W-MSA attention mechanism module. The second W-MSA attention mechanism module is used to divide the input feature map C12 into multiple non-overlapping local regions, and then perform multi-head self-attention calculation in each local region to obtain the feature map C13 after multi-head self-attention calculation, so that the feature map C14 after the feature map C13 is concatenated with the seventh feature map can be sent to the sixth normalization layer. The sixth normalization layer is used to receive feature map C14, normalize feature map C14, and then send the normalized feature map C15 to the third multilayer perceptron. The third multilayer perceptron is used to receive the feature map C15 and extract features from the feature map C15 to obtain feature map C16, which is then concatenated with feature map C14 to obtain the ninth feature map.
5. The remote sensing image change detection method based on deep learning according to claim 4, characterized in that, The calculation formula for multi-head self-attention in the SW-Block module, the first W-Block module, and the second W-Block module is as follows: In the formula, Q, K, and V are the Query matrix, Key matrix, and Value matrix, respectively. Q, K, and V are all... d represents the dimensions of the Query and Key matrices; B represents the relative position encoding. M 2 This represents the number of patches within the local window.
6. The remote sensing image change detection method based on deep learning according to any one of claims 1 to 5, characterized in that, The preprocessing steps for the dataset include: Perform data augmentation operations on the dataset.
7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the deep learning-based remote sensing image change detection method as described in any one of claims 1 to 6.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the deep learning-based remote sensing image change detection method as described in any one of claims 1 to 6.