A high-resolution image change detection method based on CIM-T architecture

Abstract

This invention discloses a high-resolution image change detection method based on the CIM-T architecture, relating to the field of change detection and image processing of two-phase remote sensing images in real-world scenarios. The method includes the following steps: Step 1, acquiring two-phase remote sensing images to obtain a change detection dataset; Step 2, constructing a change detection model using the change detection dataset; Step 3, training the change detection model using the change detection dataset; Step 4, inputting the image dataset to be tested into the change detection model to obtain the change detection result. Specifically, the change detection model in Step 2 is an encoder-decoder model; the encoder includes a backbone network feature extraction module and a label generator module; the decoder includes a dense skip connection module and an ECAM module.

Description

Technical Field

[0001] This invention relates to the field of change detection and image processing of two-phase remote sensing images in real-world scenarios, and particularly to a high-resolution image change detection method based on the CIM-T architecture. Background Technology

[0002] In recent years, with the continuous development of aerospace technology, China's Earth observation capabilities have been continuously improved, and the spatial resolution of images obtained through remote sensing satellites has also been constantly increasing. Compared with ordinary remote sensing imagery, high-resolution remote sensing imagery, specifically high spatial resolution remote sensing imagery, has richer detail information, resulting in greater accuracy and objectivity. Therefore, high-resolution remote sensing imagery is widely used in important fields such as detecting changes in urban illegal construction, detecting changes in land desertification, and analyzing crop area.

[0003] In the field of remote sensing image applications, change detection is one of the most important downstream tasks. Due to the complex ground feature information in high-resolution remote sensing images, past image registration still relied on manual interpretation. However, manual interpretation methods are labor-intensive and suffer from high false positive and false negative rates, failing to meet the demands of real-time detection. To adapt to the current trend of informatization and intelligentization in manufacturing, production processes need to break free from the constraints of traditional manual methods on capacity and efficiency.

[0004] Traditional change detection algorithms for high-resolution remote sensing images primarily apply statistical principles, using individual pixels as the basic unit to analyze and extract change information. However, this method struggles to accurately determine change boundaries and requires significant manpower and time. With the continuous development of machine learning methods, change detection has evolved from pixel-level change detection (focusing on individual pixels) to object-level change detection (aggregating multiple pixels). While machine learning methods can reduce manpower and time costs, they still cannot yield satisfactory results regarding the boundaries of changed regions and the semantic information of the changed content.

[0005] However, with the continuous development of deep learning methods, various change detection algorithms based on neural networks are constantly being proposed. Regarding the issue of varying remote sensing image sizes, Li Junwei et al. in their Chinese invention patent application "A Change Detection Method for Optical Remote Sensing Images Based on Adaptive Fusion NestedUNet" (application number: CN202211047386.2) and Ruan Yongjian et al. in their Chinese invention patent application "A Change Detection Method for Remote Sensing Images Based on Full-Scale Feature Aggregation" (application number: CN202211003665.9) have both made certain improvements to address the problem of inconsistent image resolution. Meanwhile, regarding the issue of inconsistent change objects, Dai Yuqi et al. in their Chinese invention patent application "A Method for Detecting Surface Topography Changes in Planetary Remote Sensing Images Based on Deep Learning" (application number: CN202211136723.5) and Hou Biao et al. in their Chinese invention patent application "A Patent Method for Aircraft Change Detection Based on Deep Learning and Image Registration Algorithms" (application number: CN202210815924.1) have both made certain improvements to address different change objects.

[0006] However, the methods described above often encounter a series of problems when used for change detection in high-resolution remote sensing images. The boundaries of change regions in remote sensing images are often quite fine, and the continuous downsampling process of convolutional neural networks leads to the loss of numerous detailed information from the image, significantly impacting the boundary information of the final change detection results. Furthermore, high-resolution remote sensing images are typically very large, requiring a significantly larger number of parameters and greater computational power for training large-size, high-resolution images.

[0007] Therefore, those skilled in the art are dedicated to developing a high-resolution image change detection method based on the CIM-T architecture to solve the aforementioned problems in the existing technology. Summary of the Invention

[0008] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is how to solve the problems that exist simultaneously in various neural network-based deep learning change detection algorithms, which lead to the loss of a large amount of detailed information in the image, as well as the problem of requiring an excessive number of parameters and training computing power.

[0009] To achieve the above objectives, this invention provides a high-resolution image change detection method based on the CIM-T architecture. This method combines the advantages of a CIM-Transformer network architecture (i.e., the CIM-T architecture, consisting of convolutional-intra-convolutional modules and a Transformer module) with the fewer parameters required by convolutional neural networks and the strong feature extraction capabilities of Transformer networks. While ensuring that the number of training parameters is not excessively large, it significantly improves the extraction of edge information for image change detection. Ultimately, it solves the problem of very imprecise boundary details in the final detection result due to the continuous downsampling process in existing convolutional neural networks, and also addresses the problem of extremely high parameter and computational demands caused by using only the Transformer network architecture. Specifically, the technical solution provided by this invention adopts a classic encoder-decoder structure for network construction: the encoder part proposes a combination of convolution and intra-convolution to initially extract features, and then uses the Transformer module to further optimize the features to obtain more refined features; the decoder part proposes to use a dense skip connection mechanism to resample the features at each level to restore detailed information. Finally, an ECAM module (Overall Channel Attention Mechanism module) is used to make better predictions of features in changed areas.

[0010] Specifically, this invention provides a high-resolution image change detection method based on the CIM-T architecture, comprising the following steps:

[0011] Step 1: Collect two periods of remote sensing images to obtain a change detection dataset;

[0012] Step 2: Construct a change detection model using the change detection dataset;

[0013] Step 3: Train the change detection model using the change detection dataset;

[0014] Step 4: Input the dataset of the image to be tested into the change detection model to obtain the change detection results;

[0015] in,

[0016] The change detection model in step 2 is an encoder-decoder model;

[0017] The encoder includes a backbone network feature extraction module and a label generator module;

[0018] The decoder includes a dense skip connection module and an ECAM module;

[0019] Specifically, step 2 includes the following sub-steps:

[0020] Step 2.1: Extract multi-scale features from the change detection dataset through the backbone network feature extraction module and record them in the first feature map. Then, regenerate the multi-scale features through the label generator module.

[0021] Step 2.2: The extracted multi-scale features are fused and enhanced using the dense skip connection module;

[0022] Step 2.3: Use the ECAM module to perform differential assimilation on the multi-scale features to obtain the change detection model.

[0023] Furthermore, the backbone network feature extraction module in step 2 is composed of a dual-path weight-sharing Siamese network. The change detection dataset obtained from the two periods of remote sensing images is input into the Siamese network, and then the CIM module is used to obtain the multi-scale features in the change detection dataset.

[0024] Furthermore, the CIM module includes a convolution module and an inner convolution module. The input change detection dataset is first processed by the convolution module to obtain an intermediate result, and then the intermediate result is input into the inner convolution module to obtain the multi-scale features.

[0025] Furthermore, the convolution module includes: a 3×3 convolutional layer, a first batch of normalization layers, and an activation layer; the inner convolution module includes: a 1×1 convolutional kernel, a second batch of normalization layers, an activation function, and an element-wise addition module;

[0026] The result obtained by the inner convolution module is added to the change detection dataset by residual connection to obtain the multi-scale feature, which is then recorded in the first feature map with four layers, each of which has a different size and number of channels.

[0027] The CIM module is given by the following formula:

[0028] x = x + CIM(x)

[0029] Where x represents the change detection dataset, CIM represents the result of the convolution-inner convolution operation, and the addition operation represents the residual connection method.

[0030] Furthermore, the tag generator module includes: a tag encoding module and a tag decoding module;

[0031] The label encoding module encodes the multi-scale features obtained by the backbone network feature extraction module through the spatial attention module and the transformer encoding module to generate a label vector with global context information.

[0032] The label decoding module uses a multi-head inter-layer attention approach to re-obtain the label vector and reconnect it with the first feature. Figure 1 A second feature map of sample size with global information.

[0033] Furthermore, for an upsampling unit, the dense skip connection module uses double bilinear interpolation to fuse and enhance the second feature map, resulting in a second feature map with improved resolution. The second feature map is then used as the input for the next upsampling unit, and each layer of the final second feature map has the same size.

[0034] The upsampling unit includes: a 1×1 convolutional layer, a batch normalization layer, an activation function layer, and a double bilinear interpolation layer.

[0035] Furthermore, the input to the ECAM module is a four-layer feature map of the second feature map;

[0036] Step 2.3 includes the following sub-steps:

[0037] Step 2.3.1: Directly add the four feature maps together, specifically as follows:

[0038] F intra =x (0，1) +x (0，2) +x (0，3) +x (0，4)

[0039] Where, x (0，1) This represents the first layer feature map, x. (0，2) This represents the second layer feature map, x. (0，3) This represents the third layer feature map, x. (0，4) F represents the fourth layer feature map. intra This represents the internal relationship of the four layers of feature maps input, and the addition operation represents the element-wise addition of the feature maps of a specific layer;

[0040] Step 2.3.2: Perform a channel-level concatenation operation to obtain the interrelationships between the four input feature maps, specifically:

[0041] F inter =[x (0，1) x (0，2) x (0，3) x (0，4) ]

[0042] Among them, F inter This indicates the relationship between the four layers of input feature maps, and the square brackets represent the concatenation operation on the channel dimension;

[0043] Step 2.3.3, F intra The input channel attention module calculates the channel attention CAM, specifically as follows:

[0044] CAM(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))

[0045] Where F can take the value F intra or F inter F intra or F inter The channel attention calculation formula is the same. MLP is a multilayer perceptron, AvgPool is average pooling, MaxPool is max pooling, and σ represents the sigmoid operation.

[0046] M intra =CAM(F intra )

[0047] Where F takes F intra M intra This represents a relationship within the four-layer feature map;

[0048] Step 2.3.4, F inter Input the channel attention module and compare it with M after repeating it four times. intra Adding them together, we get:

[0049] F inter +repeat (4) (M (intra) )

[0050] Step 2.3.5: Multiply the two results pixel-by-pixel to calculate the overall attention mechanism ECAM, specifically:

[0051]

[0052] Among them, M (inter) This represents a relationship between the four feature maps, and its calculation method is the same as M. intra The calculation method is similar, except that F is... intra Replace with F inter ;

[0053] Step 2.3.6: Obtain the final prediction result through a 1×1 convolution operation, specifically as follows:

[0054] Y = (ECAM(F) ensemble ))

[0055] Among them, F ensemble represent Y represents the final prediction result, and h represents a 1×1 convolution operation.

[0056] Furthermore, the channel attention module includes an average pooling layer, a max pooling layer, a fully connected layer, and an activation function layer.

[0057] Furthermore, in step 3, the total training loss of the change detection model is the sum of the cross-entropy loss and the similarity metric loss:

[0058] Loss = L wce +L dice

[0059] Where Loss is the total training loss, L wce For the cross-entropy loss, L dice The similarity metric loss;

[0060] L wce The calculation process is as follows:

[0061]

[0062] Where H represents the height of the two remote sensing images, W represents the width, wig t[class] represents the weight of each category, and class represents two lists: 0 or 1;

[0063] L dice The calculation process is as follows:

[0064]

[0065] Where Y represents the true value of the label, Representing the prediction result, softmax is the normalization exponential function. The softmax function normalizes the original data of the previous layer, transforming it into a value between (0,1).

[0066] Further, in step 3, the hyperparameters of the change detection model are set as follows: training is performed at multiple scales, adjusting the input remote sensing images from the two periods to 256×256 and 512×512 sizes, and performing two different training sets respectively; training is performed using the Adam optimizer, with a 10-epoch warmup process; the model is trained for a total of 100 epochs, with the initial learning rate set to 0.001; and training is performed using 8 3090 graphics cards.

[0067] The high-resolution image change detection method based on the CIM-T architecture provided by this invention has at least the following technical advantages:

[0068] 1. The technical solution provided by this invention uses the CIM module as the first part of two-stage image feature extraction, and then uses the Transformer module to regenerate the features at each level, thereby improving the network's feature extraction capability for images and providing more refined extraction of changing boundaries.

[0069] 2. The technical solution provided by this invention performs a fusion process on features from different levels by employing a dense skip connection mechanism. Unlike the traditional PANet network, which simply sums multi-scale features, the dense skip connection mechanism integrates features from different levels as well as features from the same level, performing cross-scale and fast normalization fusion. This enhances the representational ability of input features at different resolutions, thereby improving the more refined extraction of changing boundaries.

[0070] 3. To address the semantic differences among feature pixels at each level of fusion, the technical solution provided by this invention employs an ECAM approach to process features at each level. ECAM is based on CAM and uses an attention mechanism to assimilate features. By applying mutual attention mechanisms within and between features, semantic biases at each feature pixel are coordinated, resulting in better detection results.

[0071] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0072] Figure 1 This is a flowchart illustrating a high-resolution image transformation method based on the CIM-T architecture provided in a preferred embodiment of the present invention.

[0073] Figure 2 for Figure 1 A schematic diagram of the overall network architecture of the provided embodiment;

[0074] Figure 3 for Figure 1 A schematic diagram of the backbone network encoder for extracting multi-scale features from two phases of images provided in the embodiment;

[0075] Figure 4 for Figure 1 A schematic diagram of the architecture of the dense skip connection mechanism that fuses multi-scale input features from the backbone network in the provided embodiment;

[0076] Figure 5 for Figure 1 A schematic diagram of the architecture of the ECAM network for predicting the final detection result in the provided embodiment. Detailed Implementation

[0077] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0078] This invention provides a high-resolution image change detection method based on the CIM-T architecture. On the one hand, it integrates self-attention and convolution to reduce the number of parameters while improving feature extraction capabilities and minimizing the loss of image details due to continuous downsampling. On the other hand, it reuses the Transformer to extract features from the feature maps extracted by the convolutional neural network, obtaining richer feature information. Compared to simply using the Transformer to extract features from the original image, this method reduces a significant number of parameters. Finally, the ECAM module is used to fuse the features extracted by the Transformer, further improving the final result.

[0079] Example 1

[0080] This invention provides a high-resolution image change detection method based on the CIM-T architecture, comprising the following steps:

[0081] Step 1: Collect two periods of remote sensing images to obtain a change detection dataset;

[0082] Step 2: Build a change detection model using the change detection dataset;

[0083] Step 3: Train the change detection model using the change detection dataset;

[0084] Step 4: Input the dataset of the image to be tested into the change detection model to obtain the change detection results.

[0085] like Figure 1 As shown, step 1 can be further divided into three sub-steps: image acquisition, image filtering and labeling, and data augmentation, ultimately obtaining the change detection dataset. Step 2 involves building the change detection model, step 3 involves training the change detection model, and step 4 involves predicting the change detection model, ultimately obtaining the change detection results.

[0086] Example 2

[0087] Based on Example 1, this invention specifically adopts a building change detection method in the field of remote sensing based on the CIM-T architecture (CIM-Transformer) and ECAM, wherein the three sub-steps of step 1 are as follows:

[0088] Step 1.1: Image acquisition. Use drone aerial photography technology to acquire remote sensing images of two different time periods in real life.

[0089] Step 1.2, Image Selection and Labeling: First, by selecting from numerous remote sensing images from two periods, images showing changes in buildings are chosen. Image labeling tools are then used to label the changes in the acquired remote sensing images from the two periods. The labels depict the outlines of the changed areas. The resulting labeled files and the original images are used to construct the building change detection dataset required for the model.

[0090] Step 1.3: Data augmentation. Perform data augmentation on the obtained building change detection dataset. This mainly includes random cropping, random horizontal flipping, random vertical flipping, random fixed rotation, random rotation, random Gaussian noise, random scale cropping, random fixed cropping, and scale jitter.

[0091] Steps 2 through 4 are as follows:

[0092] The change detection model construction in step 2 is as follows: First, multi-scale features are extracted using CIM. Then, features at each level are regenerated using the Transformer module to form the feature extraction module of the backbone network. Next, the features extracted by the backbone network are fused at multiple scales using a dense skip connection mechanism to enhance features at different resolutions. Finally, based on the fused and enhanced multi-scale features, the features are assimilated using the ECAM module to obtain the final detection head part.

[0093] Step 3, the change detection model training, specifically involves dividing the building change detection dataset into a training set and a test set according to a certain ratio. The data-augmented training set is then input into the surface defect detection model for training. During training, the division of positive and negative samples is based on the Optimal Transport Assignment (OTA) strategy.

[0094] Step 4, the change detection model prediction, specifically involves: using the surface defect detection model of the training number for inference, inputting product surface images from the test set, and outputting and displaying the detected change edges.

[0095] Example 3

[0096] Based on Examples 1 and 2, this invention provides a high-resolution image change detection method based on the CIM-T architecture, including: network construction, training and testing. Specifically, the network is a dual-channel twin deep change detection network with an encoder-decoder structure built using the PyTorch deep learning framework. After training the network based on a remote sensing image data sample set, two expected remote sensing images are used as the input to the network to obtain the change detection results of the two remote sensing images.

[0097] The three sub-steps of step 1 are as follows:

[0098] Step 1.1: Image acquisition. Two remote sensing images of a certain location were captured using an industrial camera on a drone. The locations of the changed areas were marked, and the marked images and the original images were used to construct the change detection dataset required by the model.

[0099] Specifically, firstly, high-quality regional remote sensing images are captured using a CCD industrial camera on a day with good lighting conditions. The image size is related to the region being captured, and the images are RGB three-channel color images;

[0100] Step 1.2: Image Selection and Annotation. First, the numerous remote sensing images from both periods are visually interpreted to select all images showing changes to buildings. Then, image annotation tools are used to annotate the changes in the acquired images from both periods.

[0101] Specifically, firstly, all collected datasets were filtered through visual comparison to select images suitable for building change detection. Then, Photoshop image processing software was used to manually annotate the edges and internal contours of the collected building change detection dataset at the pixel level. By comparing the changed areas between two images, the changed parts and their contours were labeled as 1, while the unchanged parts were labeled as 0. Finally, a single-channel black-and-white binary image containing only 0s and 1s was generated as the final training label.

[0102] Step 1.3: Data Augmentation. The dataset serves as the foundation for change detection. Considering the limited number of changed areas in the actual change detection dataset, data augmentation is necessary to achieve better results. This mainly includes: random cropping, random horizontal flipping, random vertical flipping, random fixed rotation, random rotation, random Gaussian noise, random scale cropping, random fixed cropping, and scale jittering. Data augmentation needs to be performed on both image phases. Simultaneously, if the input image for change detection shifts or transforms, the same data augmentation process needs to be applied to the tag data. The goal is to ensure that after image transformation, the changed areas still match the labeled areas.

[0103] Specifically, the following methods are employed: Random cropping: The image is randomly cropped while ensuring that the defective target is not removed, and the position of the original annotation boundary in the cropped image is calculated; Random horizontal flipping: The image and annotation boundary are horizontally flipped with a probability of 0.5; Random vertical flipping: The image and annotation boundary are vertically flipped with a probability of 0.5; Random fixed rotation: The image and annotation image are rotated by 90°, 180°, and 270° with a probability of 0.75; Random rotation: The image and annotation image are randomly rotated using angles conforming to a uniform distribution; Random Gaussian noise: A random Gaussian noise with a random radius is applied to the two input images; Random scale cropping: The two input images and the annotation image are cropped by a random size; Random fixed cropping: The two input images and the annotation image are cropped by a fixed-size crop at the center; Scale jittering: Before cropping, the image size is randomly adjusted to 0.5-1.5 times the original image size, and the annotation image and annotation boundary are adjusted accordingly.

[0104] Step 2 includes the following sub-steps:

[0105] Step 2.1: Extract multi-scale features from the change detection dataset through the backbone network feature extraction module and record them in the first feature map. Then, regenerate the multi-scale features through the label generator module.

[0106] Step 2.2: The extracted multi-scale features are fused and enhanced using a dense skip connection module;

[0107] Step 2.3: Use the ECAM module to perform differential assimilation on multi-scale features to obtain a change detection model.

[0108] The building remote sensing image change detection model proposed in step 2 is a typical encoder-decoder model, and the overall network structure diagram is as follows: Figure 2 As shown, the encoder module mainly consists of two parts: a backbone network feature extraction module and a label generator module. The encoder architecture is as follows: Figure 3 As shown:

[0109] The backbone network feature extraction module mainly consists of a dual-path weight-sharing Siamese network. Change detection datasets obtained from two periods of remote sensing images are input into the Siamese network, and then the CIM module is used to obtain multi-scale features from the change detection dataset. The CIM module includes a convolution module and an inner convolution module. The input change detection dataset is first processed through the convolution module to obtain intermediate results, which are then input into the inner convolution module to obtain multi-scale features.

[0110] Specifically, two high-resolution remote sensing images are input into a Siamese network. Since the weights of the two networks are identical, feature maps at the same locations are obtained. Unlike traditional methods using convolution modules, this invention employs a convolution-involution module to obtain feature maps from the two remote sensing images. The implementation mainly involves two steps: first, the input is processed by the convolution module to obtain the corresponding result; then, the input to the involution module is the output of the convolution module, which is then input into the involution module to obtain the final output. The convolution module includes a 3×3 convolutional layer, a batch normalization layer, and an activation layer. The involution module includes a 1×1 convolutional kernel, a batch normalization layer, an activation function, and an element-wise addition module. Finally, the result obtained from the involution is added to the original image via residual connections to obtain multi-scale features, which are recorded in a four-layer first feature map. Each layer of the first feature map has a different size and number of channels.

[0111] The CIM module is given by the following formula:

[0112] x = x + CIM(x)

[0113] Where x represents the change detection dataset, CIM represents the result of the convolution-involution operation, and the addition operation represents the residual connection method.

[0114] The above method yields four feature maps of different sizes and numbers of channels. The results from the inner convolution module are then added to the change detection dataset using residual connections to obtain multi-scale features, which are recorded in the first feature map, which has four layers, each with a different size and number of channels.

[0115] The label generator module mainly consists of two modules: a label encoding module (Token-Encoder) and a label decoding module (Token-Decoder). The label encoding module encodes the multi-scale features obtained by the backbone network feature extraction module through the spatial attention module and the transformer encoding module, generating label vectors with global context information.

[0116] The main purpose of the label encoding module is to encode global feature information through the spatial attention module and the transformer encoding module, thereby generating a label vector (token) of size b×l×c with global context information. The transformer encoding module includes: a 1*1 convolutional layer, a positional encoding structure, a layer normalization layer, a multi-head attention layer, and a bilinear layer.

[0117] The label decoding module uses multi-head inter-layer attention to re-obtain the label vector and the first feature. Figure 1 A second feature map of sample size with global information.

[0118] The specific implementation of the label encoder module is as follows: The spatial attention module first processes the input feature map through a 1*1 convolutional kernel, adjusting the number and size of the input features to obtain a vector that does not contain global context information. The calculation process is as follows: t blc =f b ' l(hw) f bc(hw) This vector is then input into the transformer encoder module, where positional encoding and multi-head self-attention mechanisms generate Q, K, and V vectors respectively. After softmax, the vector is passed through two linear layers to obtain the final label vector. Assume the input is F∈R. (b×c×h×w) A feature map is processed by a spatial attention module to obtain a vector of size b×l×c, which is then used by a transformer encoder to generate three vectors Q, K, and V. The three vectors generated by the multi-head self-attention module are given by the following formula: Q,K,V=t'W Q ,t'W K ,t'W V Here, the three vectors Q, K, and V are all given by the input feature map itself. The final label vector calculation formula obtained through the multi-head self-attention mechanism is given by the following formulas:

[0119]

[0120] head i =SDPA(t'W i Q ,t'W i K ,t'W i V ),

[0121] MHSA(Q,K,V)=Concat(head1,head2,...head n ).

[0122] Finally, a label vector of size b×l×c is obtained through a bilinear layer. The label decoding module uses multi-head inter-layer attention (MHCA(Q,K,V)=Concat(head1,head2,...head)) to process the labels. n ), regain and original features Figure 1A feature map of similar size with more global information. The specific structure of the label decoder is exactly the same as that of the label encoder, except that Q,K,V = t'W Q ,t'W K ,t'W V The three vectors originate from different sources. The encoder is implemented using a multi-head self-attention approach, while the decoder is implemented using a multi-head inter-layer attention approach.

[0123] The decoder module mainly consists of two parts: a dense skip connection module and an ECAM module, such as... Figure 4 and Figure 5 As shown.

[0124] In the skip-dense module, to obtain more representative features and reduce the loss of numerous details due to downsampling, a skip-dense connection mechanism is used to connect the encoder and decoder. The input is a series of feature maps obtained from the encoder. Specifically, for each upsampling unit, the skip-dense module uses double bilinear interpolation to fuse and enhance the second feature map, resulting in a higher-resolution second feature map. This second feature map is then used as the input for the next upsampling unit, resulting in each layer of the final second feature map having the same size. This upsampling method restores the size of the previous layer's feature map, thereby recovering the lost details and achieving better detection results.

[0125] The upsampling units used include: 1×1 convolutional layers, batch normalization layers, activation function layers, and double bilinear interpolation layers.

[0126] Since feature maps with the same resolution obtained by double bilinear interpolation have biases in the prediction of different pixels, the ECAM module is used to optimize the above biases to obtain better prediction results.

[0127] Specifically, the input to the ECAM module is the four-layer feature map of the second feature map.

[0128] Step 2.3 includes the following sub-steps:

[0129] Step 2.3.1: Directly add the four feature maps together, specifically as follows:

[0130] F intra =x (0,1) +x (0,2) +x (0,3) +x (0,4)

[0131] Where, x (0,1) This represents the first layer feature map, x.(0,2) This represents the second layer feature map, x. (0,3) This represents the third layer feature map, x. (0,4) F represents the fourth layer feature map. intra This represents the internal relationship of the four input feature maps. The addition operation represents the element-wise addition of the feature maps of a specific layer.

[0132] Step 2.3.2: Perform a channel-level concatenation operation to obtain the interrelationships between the four input feature maps, specifically:

[0133] F inter =[x (0,1) ,x (0,2) ,x (0,3) ,x (0,4) ]

[0134] Among them, F inter This indicates the relationship between the four input feature maps, with square brackets representing concatenation operations along the channel dimension.

[0135] Step 2.3.3, F intra The input channel attention module calculates the channel attention CAM, specifically as follows:

[0136] CAM(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))

[0137] Where F can take the value F intra or F inter F intra or F inter The channel attention calculation formula is the same. MLP is a multilayer perceptron, AvgPool is average pooling, MaxPool is max pooling, and σ represents the sigmoid operation.

[0138] M intra =CAM(F intra )

[0139] Where F takes F intra M intra This represents a relationship within the four-layer feature map.

[0140] Step 2.3.4, F inter Input channel attention module, and M after four repetitions intra Adding them together, we get:

[0141] F inter +repeat (4) (M (intra) )

[0142] Step 2.3.5: Multiply the two results pixel-by-pixel to calculate the overall attention mechanism ECAM, specifically:

[0143]

[0144] Among them, M (inter) This represents a relationship between the four feature maps, and its calculation method is the same as M. intra The calculation method is similar, except that F is... intra Replace with F inter ;

[0145] Step 2.3.6: Obtain the final prediction result through a 1×1 convolution operation, specifically as follows:

[0146] Y = (ECAM(F) ensemble ))

[0147] Among them, F ensemble represent Y represents the final prediction result, and h represents a 1×1 convolution operation.

[0148] The channel attention module mainly includes an average pooling layer, a max pooling layer, a fully connected layer, and an activation function layer. The ECAM module mainly includes the channel attention module, and the prediction module mainly includes a 1×1 convolutional module. By concatenating different features along the channel dimension and obtaining a relationship between different features through pixel-by-pixel addition, the semantic bias between features can be greatly reduced, resulting in better prediction results.

[0149] In step 3, the change detection model is trained. Specifically, the dataset is divided into a training set and a test set according to a certain ratio, and the data-augmented training set is input into the change detection model for training.

[0150] Specifically, the two sets of building remote sensing change detection datasets obtained in Step 1 are divided into training and test sets in a 4:1 ratio. The training set is augmented using the data augmentation strategy from Step 1 and then input into the change detection model constructed in Step 2 for training. In the change detection task, since the number of unchanged pixels is often much greater than the number of changed pixels, to reduce the serious impact of sample imbalance on the results, the total training loss of the change detection model is the sum of the cross-entropy loss and the similarity metric loss (dice loss):

[0151] Loss = L wce +L dice

[0152] Where Loss is the total training loss, Lwce For cross-entropy loss, L dice Loss is used to measure similarity.

[0153] L wce The calculation process is as follows:

[0154]

[0155] Where H represents the height of the two remote sensing images, W represents the width, wig t[class] represents the weight of each category, and class represents two lists: 0 or 1; L wce The overall meaning of the calculation process is an abbreviation for the calculation method of binary cross-entropy loss, and the calculation method of binary cross-entropy loss function is fixed.

[0156] L dice The calculation process is as follows:

[0157]

[0158] Where Y represents the true value of the label, Representing the prediction result, softmax is a normalization exponential function. The softmax function normalizes the original data from the previous layer, transforming it into a value between (0,1). Using softmax, the final prediction result is converted into a probability value between 0 and 1, used to determine whether a change has occurred or not. This hybrid approach of using two loss functions can better aid model training. The resulting value is treated as a probability distribution and used as the target prediction value for multi-class classification.

[0159] In step 3, the hyperparameters of the change detection model are set as follows: Training is conducted at multiple scales, adjusting the input remote sensing images from the two periods to 256×256 and 512×512 sizes, and performing two different training sets respectively; the Adam optimizer is used for training, along with a 10-epoch warmup process; the model is trained for a total of 100 epochs, with an initial learning rate set to 0.001; training is performed using eight 3090 GPUs. For input images of size 256, 16 images are allocated to each GPU, resulting in a total batch size of 128.

[0160] In the prediction of the change detection model in step 4, the trained network model is used for inference. The two remote sensing surface images from the test set are input, and the detected change boundaries are output and displayed.

[0161] Specifically, the two remote sensing images from the divided dataset are input into the change detection model trained in step five. First, multi-scale features are extracted using CIM, and then the Transformer module regenerates features at each level to form multi-scale features containing more feature information. Then, the features extracted from the backbone network are fused at multiple scales using a dense skip connection mechanism to enhance features at different resolutions. Based on the fused and enhanced multi-scale features, the ECAM module performs differential assimilation on the features, and the detection head is used to predict the change region using the above multi-scale features. Finally, a 1×1 convolutional kernel is used to obtain the final predicted change result.

[0162] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A high resolution image change detection method based on CIM-T architecture, characterized in that, The method comprises the following steps: Step 1, collecting two period remote sensing image images, obtaining change detection data set; Step 2, using the change detection data set to build a change detection model; Step 3, training the change detection model with the change detection data set; Step 4, inputting the image data set to be tested into the change detection model to obtain the change detection result; Wherein, The change detection model in step 2 is an encoder-decoder model; The encoder includes a backbone network feature extraction module and a label generator module; The decoder includes a dense skip connection module and an ECAM module; Specifically, step 2 comprises the following sub-steps: Step 2.1, extracting multi-scale features in the change detection data set through the backbone network feature extraction module and recording in the first feature map, and then regenerating the multi-scale features through the label generator module; Step 2.2, fusing and enhancing the extracted multi-scale features through the dense skip connection module; Step 2.3, using the ECAM module to difference assimilate the multi-scale features to obtain the change detection model; The backbone network feature extraction module is composed of a twin network sharing double weights, the change detection data set obtained from the two period remote sensing image images is input into the twin network, and then the CIM module is used to obtain the multi-scale features in the change detection data set; The CIM module includes a convolution module and an inner convolution module, the input change detection data set is first processed through the convolution module to obtain an intermediate result, and then the intermediate result is input into the inner convolution module to obtain the multi-scale features; The convolution module includes a 3x3 convolution layer, a first batch normalization layer and an activation layer; the inner convolution module includes a 1x1 convolution kernel, a second batch normalization layer, an activation function and an element-wise addition module; The result obtained by the inner convolution module is added to the change detection data set through a residual connection mode to obtain the multi-scale features, which are recorded in the first feature map with four layers, and each layer of the first feature map has different sizes and different channel numbers; The CIM module is given by the following formula: Wherein, x represents the change detection data set, CIM represents the result of convolution-inner convolution operation, and the addition operation represents the residual connection mode; The label generator module includes a label encoding module and a label decoding module; The label encoding module encodes the multi-scale features obtained by the backbone network feature extraction module through a spatial attention module and a transformer encoding module to generate a label vector with global context information; The label decoding module uses a multi-head inter-layer attention mode to reobtain a second feature map with the same size as the first feature map and with global information.

2. The high resolution image change detection method based on CIM-T architecture according to claim 1, wherein, For an upsampling unit, the dense skip connection module adopts a bilinear interpolation method to fuse and enhance the second feature map, to obtain a second feature map with improved resolution, and the second feature map is continuously taken as the input of the next upsampling unit, and finally the size of each layer of the second feature map obtained is the same; The upsampling unit comprises a 1x1 convolution layer, a batch normalization layer, an activation function layer and a bilinear interpolation layer.

3. The high resolution image change detection method based on CIM-T architecture according to claim 2, wherein, The input of the ECAM module is four layers of feature maps of the second feature map; The step 2.3 comprises the following sub-steps: Step 2.3.1, directly adding the four layers of feature maps, specifically: wherein, represents the first layer feature map, represents the second layer feature map, represents the third layer feature map, represents the fourth layer feature map, represents the internal relationship of the four input layer feature maps, and the addition operation represents element-wise addition of the feature maps of a certain layer; Step 2.3.2, performing a concatenation operation in the channel dimension to obtain the mutual relationship between the input four layers of feature maps, specifically: wherein, represents the interrelation between the input four-layer feature maps, and the square brackets represent the splicing operation in the channel dimension; Step 2.3.3, computing the channel attention input channel attention module, computing the channel attention , specifically: Wherein, F can take or , or The channel attention calculation formula is the same as represent operation; wherein F takes the value of , represents a relationship within the four-layer feature map; Step 2.3.4, add to the channel attention module and add the four times repeated together, resulting in: Step 2.3.5, the two results are multiplied by pixel by pixel to calculate the overall attention mechanism Specifically, wherein, represents a relationship between the four-layer feature maps, which is calculated in a manner similar to , except that is replaced by ; Step 2.3.6, the final prediction result is obtained by the convolution operation of , specifically: wherein, representing , representing the final prediction result, representing convolution operation.

4. The high resolution image change detection method based on CIM-T architecture according to claim 3, wherein, The channel attention module comprises an average pooling layer, a maximum pooling layer, a full connection layer and an activation function layer. 5.The high resolution image change detection method based on CIM-T architecture of claim 3, wherein, In the step 3, the total loss of the training of the change detection model is the sum of the cross-entropy loss and the similarity measurement loss: wherein, is the total loss for the training, is the cross-entropy loss, is the similarity measure loss; The calculation process is as follows: wherein, represents the height of the two-phase remote sensing image image, represents the width, represents the weight of each class, class represents two lists: 0 or 1; The calculation process is as follows: wherein, represents the true label result, represents the predicted result, softmax is a normalized exponential function, and the softmax function normalizes the original data of the previous layer into a value between (0, 1).

6. The high resolution image change detection method based on CIM-T architecture according to claim 3, wherein, In the step 3, the hyperparameter settings of the change detection model are as follows: training with multiple scales, adjusting the input two-period remote sensing image to and different sizes for two different groups of training respectively; using Adam optimizer for training, and using 10 epoch warmup process; the model is trained for a total of 100 epochs, the initial learning rate is set to 0.001; training is performed using 8 3090 graphics cards.

Images