Remote sensing image transformation detection method based on multi-scale global local feature fusion

Abstract

The invention discloses a remote sensing image transformation detection method based on multi-scale global and local feature fusion. According to the method, the CNN module and the Transform module are adaptively mixed, and the advantage of extracting the local feature information of the image by the CNN module and the advantage of extracting the global feature information of the image by the multi-scale Transform module are utilized, so that all the global and local information of the high-resolution remote sensing image is better fused in the feature extraction stage, and the feature extraction efficiency is improved. According to the method, feature information richness of a feature extraction layer is guaranteed, meanwhile, a multi-scale attention mechanism is used for performing feature extraction and fusion on different scales, complexity of attention calculation is reduced, pixel detail information of a high-resolution remote sensing image can be reserved, and the accuracy of feature extraction is improved. Expression of shallow layer and deep layer information of the high-resolution remote sensing image is guaranteed, the change content in the high-resolution remote sensing image at different moments in the same area can be accurately recognized, the detection precision is high, the calculation complexity is low, and the detection efficiency is high.

Description

Technical Field

[0001] The present invention specifically relates to a transformation detection method for remote sensing images based on multi-scale global and local feature fusion. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing images have been widely used in environmental monitoring, urban planning, disaster assessment and other fields. Among them, remote sensing image transformation detection technology aims to detect surface change information by comparing and analyzing remote sensing images of different periods, which is of great significance for understanding natural disasters, monitoring environmental changes, and conducting resource management.

[0003] However, the current remote sensing image transformation detection methods mainly rely on statistical methods and feature-based detection methods. These methods are computationally complex and have low detection efficiency when facing high-resolution remote sensing images. They are also prone to loss of pixel detail information, insufficient feature information in the feature extraction layer, and cannot accurately identify changes in remote sensing images of the same area at different times. The detection accuracy is poor and needs to be improved urgently.

[0004] Therefore, it is necessary to invent a transformation detection method for remote sensing images based on multi-scale global and local feature fusion to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a transformation detection method for remote sensing images based on multi-scale global and local feature fusion. By adaptively mixing a CNN module with a Transformer module, all global and local information of a high-resolution remote sensing image is better fused in the feature extraction stage, thereby ensuring the richness of feature information in the feature extraction layer. At the same time, a multi-scale attention mechanism is used to extract and fuse features at different scales, thereby reducing the complexity of attention calculation, and enabling pixel detail information of the high-resolution remote sensing image to be retained, thereby ensuring the expression of shallow and deep information of the high-resolution remote sensing image, thereby being able to accurately identify changes in high-resolution remote sensing images of the same area at different times, with high detection accuracy, low computational complexity, and fast detection efficiency, so as to solve the above-mentioned shortcomings in the technology.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a transformation detection method of remote sensing images based on multi-scale global and local feature fusion, comprising the following steps:

[0007] Step 1: Input two high-resolution remote sensing images of the same area at different times and group them by channel;

[0008] Step 2: extract and merge the grouped high-resolution remote sensing images to obtain global and local feature information at multiple scales;

[0009] Step 3: The output of the global and local feature information at different scales in the ConvTrans module is input into the CC fusion module for feature fusion and output to obtain f′, f″, f″′, f″″;

[0010] Step 4: Input f′, f″, f″′, and f″″ into the multi-scale attention decoder of the U-shaped structure to obtain global and local information at multiple scales and generate a predicted change graph.

[0011] Step 5: Use the mixed loss function L to predict the change graph Perform parameter optimization.

[0012] In the aforementioned remote sensing image transformation detection method based on multi-scale global and local feature fusion, in step 1, two high-resolution remote sensing images of the same area at different time periods are input and grouped by channel. The specific steps are as follows:

[0013] 1.1, let T 1 Represents the remote sensing image of the previous phase, T 2 represents the later phase remote sensing image, and T∈{T i ∈R C×H×W |i=1,2};

[0014] Where H is the height of the high-resolution remote sensing image,

[0015] W is the width of the high-resolution remote sensing image,

[0016] C represents the number of channels of the high-resolution remote sensing image, that is, there are C channels;

[0017] 1.2. T i Divide by channel into X 1 ,X 2 There are two groups, and the grouping results are:

[0018]

[0019] X 1 =ChannelSplit(T i )

[0020] X 2 =ChannelSplit(T i )

[0021] Among them, ChannelSplit() is a channel splitting operation.

[0022] In the aforementioned remote sensing image transformation detection method based on multi-scale global and local feature fusion, in step 2, feature extraction and feature merging are performed on the grouped high-resolution remote sensing images to obtain multi-scale global and local feature information, namely:

[0023] T 1 , T 2 X 1 ,X 2 The outputs are respectively input into the mixed modules of Transformer module and CNN module, and the outputs of the two modules are combined to obtain the global and local feature information at multiple scales.

[0024] Among them, the global and local feature information at multiple scales includes T 1 The global and local feature information F 11 , global and local feature information F 21 , global and local feature information F 31 , global and local feature information F 41 ;

[0025] and T 2 The global and local feature information F 12 , global and local feature information F 22 , global and local feature information F 32 , global and local feature information F 42 .

[0026] In the aforementioned remote sensing image transformation detection method based on multi-scale global and local feature fusion, in step 2, specifically, the global and local feature information F is obtained based on the first ConvTransBlock module. 11 The steps are as follows:

[0027] 2.1、X 1 The input is downsampled into the CNN module with a convolutional layer size of 3×3, and X is extracted 1 Local feature information The specific formula is as follows:

[0028]

[0029] Among them, Conv(·) is the convolution operation;

[0030] 2.2. X 2 After downsampling through the 1×1 convolutional layer, it is input into the Transformer module to extract X 2 Global feature information The specific formula is as follows:

[0031] F Z =Conv(X2 )

[0032]

[0033] Where MST(·) represents the Multiscale Transformer module;

[0034] 2.3 Local feature information and global feature information Perform channel splicing to obtain global and local feature information F 11 , the specific formula is as follows:

[0035]

[0036] in, Represents a channel concatenation operation.

[0037] In the aforementioned remote sensing image transformation detection method based on multi-scale global and local feature fusion, in step 2, the global and local feature information F is obtained. 21 , global and local feature information F 31 , global and local feature information F 41 The specific steps are as follows:

[0038] 2.4. Taking the second ConvTransBlock module as the benchmark, the global local feature information F 11 Input into the ConvTransBlock module, group by channel, and repeat steps 2.1-2.3 to obtain the global local feature information F 21 ;

[0039] 2.5. Based on the third ConvTransBlock module, the global local feature information F 21 Input into the ConvTransBlock module, group by channel, and repeat steps 2.1-2.3 to obtain the global local feature information F 31 ;

[0040] 2.6, based on the fourth ConvTransBlock module, the global local feature information F 31 Input into the ConvTransBlock module, group by channel, and repeat steps 2.1-2.3 to obtain the global local feature information F 41 .

[0041] In the aforementioned remote sensing image transformation detection method based on multi-scale global and local feature fusion, in step 2, T is obtained. 2 The global and local feature information F 12 , global and local feature information F22 , global and local feature information F 32 , global and local feature information F 42 The specific steps are:

[0042] Repeat steps 2.1-2.6 to obtain the global and local feature information F 12 , global and local feature information F 22 , global and local feature information F 32 , global and local feature information F 42 ;

[0043] Among them, T 1 , T 2 The outputs of the hybrid network of the Transformer module and the CNN module of the two branches at four different stages are

[0044] In the aforementioned remote sensing image transformation detection method based on multi-scale global-local feature fusion, in step 3, the output of the multi-scale global-local feature information at different scales of the ConvTrans module is input into the CC fusion module for feature fusion and output to obtain f′, f″, f″′, and f″″. The specific steps are as follows:

[0045] The outputs of the ConvTrans module at different scales are input into the CC fusion module for channel splicing, and a 1×1 convolution is performed to obtain f′, f″, f″′, f″″. The specific formula is as follows;

[0046]

[0047] In the aforementioned remote sensing image transformation detection method based on multi-scale global and local feature fusion, in step 4, f′, f″, f″′, and f″″ are input into the multi-scale attention decoder of the U-shaped structure to obtain the global and local information at multiple scales and generate a predicted change map. The specific steps are as follows:

[0048] 4.1. Input f' into the MSA module and downsample to obtain output D1. Add D1 and f'' to obtain D1' and then input it into the MSA module for downsampling to obtain D2. Add D2 and f'' to obtain D2' and then input it into the MSA module for downsampling to obtain D3. Add D3 and f'' to obtain D3' and then input it into the MSA module for downsampling to obtain D4. The specific formula is as follows:

[0049] D1=MSA(f′)

[0050]

[0051] D2=MSA(D1′)

[0052]

[0053] D3=MSA(D2′)

[0054]

[0055] D4=MSA(D3′)

[0056] Among them, MSA(·) represents the multi-scale attention module;

[0057] 4.2. Upsample and merge D1, D2, D3, and D4 and input them into the corresponding ECA module to restore the resolution of the two high-resolution remote sensing images and generate a predicted change map. The specific formula is as follows:

[0058] d4′=ECA(D4)

[0059]

[0060] Among them, d i i={1, 2, 3, 4} in ′,

[0061] Among them, ECA(·) is the operation of the ECA module,

[0062] is the matrix multiplication operation,

[0063] Up(·) is an upsampling operation.

[0064] In the aforementioned remote sensing image transformation detection method based on multi-scale global and local feature fusion, in step 5, a hybrid loss function L is used to predict the change map. Parameter optimization is performed, and the specific details are as follows:

[0065] L=L wce +L dice

[0066]

[0067] Among them, L is the total mixed loss function, that is, the weighted cross entropy loss L wce and Dice loss L dice of and;

[0068] It is the predicted change graph An element in represents the prediction result of a specific position in the graph.

[0069] Y is the true change graph,

[0070] L wce is the weighted cross entropy loss,

[0071] L dice Dice loss.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] The present invention adaptively mixes the CNN module with the Transformer module, utilizes the advantage of the CNN module in extracting local feature information of the image, and the advantage of the multi-scale Transformer module in extracting global feature information of the image, so as to better fuse all the global and local information of the high-resolution remote sensing image in the feature extraction stage, thereby ensuring the richness of the feature information of the feature extraction layer. At the same time, a multi-scale attention mechanism is used in the decoder of the U-shaped structure to perform dimensionality reduction operations on high-dimensional features by performing feature extraction and fusion at different scales, which not only reduces the complexity of attention calculation, but also can effectively solve the limitation that the feature extraction of a single scale is often difficult to fully capture the information in the high-resolution remote sensing image, ensures that the pixel detail information of the high-resolution remote sensing image can be retained, thereby accurately identifying the changes in the high-resolution remote sensing images of the same area at different times, ensuring the expression of shallow and deep information of the high-resolution remote sensing images, improving the accuracy of change detection of the high-resolution remote sensing images, and having low detection calculation complexity and high detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0075] Figure 1 is a flow chart of the present invention;

[0076] Figure 2 It is the front phase remote sensing image of the present invention;

[0077] Figure 3 is the post-phase remote sensing image of the present invention;

[0078] Figure 4 It is the target effect diagram of the present invention;

[0079] Figure 5 It is a real label schematic diagram of the present invention. DETAILED DESCRIPTION

[0080] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0081] The present invention provides Figure 1-5 A transformation detection method for remote sensing images based on multi-scale global and local feature fusion is shown, comprising the following steps:

[0082] Step 1: Input two high-resolution remote sensing images of the same area at different times and group them by channel. The specific steps are as follows:

[0083] 1.1, let T 1 Represents the remote sensing image of the previous phase, T 2 represents the later phase remote sensing image, and T∈{T i ∈R C×H×W |i=1,2};

[0084] Where H is the height of the high-resolution remote sensing image,

[0085] W is the width of the high-resolution remote sensing image,

[0086] C represents the number of channels of the high-resolution remote sensing image, that is, there are C channels;

[0087] 1.2. T i Divide by channel into X 1 ,X 2 There are two groups, and the grouping results are:

[0088]

[0089] X 1 =ChannelSplit(T i )

[0090] X 2 =ChannelSplit(T i )

[0091] Among them, ChannelSplit(·) is the channel splitting operation;

[0092] In this step, the computational complexity can be reduced by dividing the high-resolution remote sensing image into channels, and the X 1 ,X 2 The two groups of channels can reduce the computational burden and facilitate the subsequent convolution operation of CNN on one part of the channels and the attention operation of the Transformer module on the other part, respectively extracting features and facilitating better acquisition of global and local feature information at multiple scales.

[0093] Step 2: Extract and merge the features of the grouped high-resolution remote sensing images to obtain global and local feature information at multiple scales, which is:

[0094] T 1 , T 2 X 1 ,X 2 The outputs are respectively input into the mixed modules of Transformer module and CNN module, and the outputs of the two modules are combined to obtain the global and local feature information at multiple scales.

[0095] Among them, the global and local feature information at multiple scales includes T 1 The global and local feature information F 11 , global and local feature information F 21 , global and local feature information F 31 , global and local feature information F 41 ;

[0096] and T 2 The global and local feature information F 12 , global and local feature information F 22 , global and local feature information F 32 , global and local feature information F 42 ;

[0097] Specifically, taking the first ConvTransBlock module as the benchmark, we get the global local feature information F 11 The steps are as follows:

[0098] 2.1、X 1 The input is downsampled into the CNN module with a convolutional layer size of 3×3, and X is extracted 1 Local feature information The specific formula is as follows:

[0099]

[0100] Among them, Conv(·) is the convolution operation;

[0101] 2.2. X 2 After downsampling through the 1×1 convolutional layer, it is input into the Transformer module to extract X 2 Global feature information The specific formula is as follows:

[0102] F Z =Conv(X 2 )

[0103]

[0104] Where MST(·) represents the Multiscale Transformer module;

[0105] 2.3 Local feature information and global feature information Perform channel splicing to obtain global and local feature information F 11 , the specific formula is as follows:

[0106]

[0107] in, Indicates channel splicing operation;

[0108] At this time, the global local feature information F is obtained 21 , global and local feature information F 31 , global and local feature information F 41 The specific steps are as follows:

[0109] 2.4. Taking the second ConvTransBlock module as the benchmark, the global local feature information F 11 Input into the ConvTransBlock module, group by channel, and repeat steps 2.1-2.3 to obtain the global local feature information F 21 ;

[0110] 2.5. Based on the third ConvTransBlock module, the global local feature information F 21 Input into the ConvTransBlock module, group by channel, and repeat steps 2.1-2.3 to obtain the global local feature information F 31 ;

[0111] 2.6, based on the fourth ConvTransBlock module, the global local feature information F 31 Input into the ConvTransBlock module, group by channel, and repeat steps 2.1-2.3 to obtain the global local feature information F 41 ;

[0112] Among them, we get T 2 The global and local feature information F 12 , global and local feature information F 22 , global and local feature information F 32 , global and local feature information F 42 The specific steps are:

[0113] Repeat steps 2.1-2.6 to obtain the global and local feature information F 12, global and local feature information F 22 , global and local feature information F 32 , global and local feature information F 42 ;

[0114] At this time, T 1 、T 2 The outputs of the hybrid network of the Transformer module and the CNN module of the two branches at four different stages are

[0115] In this step, the ConvTransBlock module is a hybrid network structure that combines the Transformer module and the CNN module, which can enhance representation learning by fusing local features and global representations;

[0116] At the same time, because after each Block, T 1 、T 2 It is continuously down-sampled, and since there are four blocks in this method, there are four dimensions, corresponding to the outputs of different blocks, that is, the ConvTransBlock module in the first stage processes T 1 Block 1 obtains the global and local feature information F 11 Input, process T 2 The output of Block 1 is the global local feature information F 12 , and so on, we can get the global and local feature information at multiple scales;

[0117] In this step, by using a module that adaptively mixes the two methods, the feature weights of the CNN and Transformer modules are dynamically adjusted, making the fusion of global feature information and local feature information of the hybrid network more intelligent and efficient at different stages, ensuring the richness of feature information in the feature extraction layer, and better solving the problem of missing global and local information in the traditional single use of CNN modules or Transformer modules.

[0118] Step 3: The output of the global and local feature information at different scales in the ConvTrans module is input into the CC fusion module for feature fusion and output to obtain f′, f″, f″′, and f″″. The specific steps are as follows:

[0119] 3.1. The outputs of the ConvTrans module at different scales are input into the CC fusion module for channel splicing, and f′, f″, f″′, f″″ are obtained through a 1×1 convolution. The specific formula is as follows;

[0120]

[0121]

[0122] Among them, the role of the CC fusion module is to effectively integrate features of different scales, sources, and levels, and balance the contributions of different data sources through splicing operations;

[0123] In this step, the feature maps of different sources at the same level are effectively spliced ​​and merged through the CC fusion module, so that the information captured by the model complements each other. The fusion of multi-source data can improve the sensitivity and robustness of the model to changing information and improve the model's ability to detect diversified changes.

[0124] Step 4: Input f′, f″, f″′, and f″″ into the multi-scale attention decoder of the U-shaped structure to obtain global and local information at multiple scales and generate a predicted change graph. The specific steps are as follows:

[0125] 4.1. Input f' into the MSA module and downsample to obtain output D1. Add D1 and f'' to obtain D1' and then input it into the MSA module for downsampling to obtain D2. Add D2 and f'' to obtain D2' and then input it into the MSA module for downsampling to obtain D3. Add D3 and f'' to obtain D3' and then input it into the MSA module for downsampling to obtain D4. The specific formula is as follows:

[0126] D1=MSA(f′)

[0127]

[0128] D2=MSA(D1′)

[0129]

[0130] D3=MSA(D2′)

[0131]

[0132] D4=MSA(D3′)

[0133] Among them, MSA(·) represents the multi-scale attention module;

[0134] 4.2. Upsample and merge D1, D2, D3, and D4 and input them into the corresponding ECA module to restore the resolution of the two high-resolution remote sensing images and generate a predicted change map. The specific formula is as follows:

[0135] d4′=ECA(D4)

[0136]

[0137] Among them, d i 'i={1, 2, 3, 4};

[0138] ECA(·) is the operation of the ECA module,

[0139] is the matrix multiplication operation,

[0140] Up(·) is the upsampling operation;

[0141] In this step, through the combination of multi-scale attention mechanism and ECA module, downsampling and upsampling are performed layer by layer to achieve feature decoding and fusion. Among them, the MSA module can capture the change information of different scales and enhance the global context perception. At the same time, the gradual fusion of features in each layer helps to combine shallow details and deep semantics. The ECA module further strengthens the important channel features and improves the detection ability of this model for changed areas. The overall design can not only accurately capture the changed areas, but also restore the resolution during the upsampling process, realizing efficient and accurate remote sensing image change detection.

[0142] Step 5: Use the mixed loss function L to predict the change graph Optimize the parameters, the details are as follows;

[0143] L=L wce +L dice

[0144]

[0145] Among them, L is the total mixed loss function, that is, the weighted cross entropy loss L wce and Dice loss L dice of and;

[0146] It is the predicted change graph An element in represents the prediction result of a specific position in the graph.

[0147] Y is the true change graph,

[0148] L wce is the weighted cross entropy loss,

[0149] L dice For Dice loss;

[0150] In this step, the weighted cross entropy loss L is used wce and Dice loss L diceThe mixed loss function is optimized. Among them, the weighted cross entropy loss can deal with the problem of category imbalance and enhance the model's recognition ability for minority categories. The Dice loss focuses on the overlapping area between the predicted image and the real change image, improving the accuracy of the model in the change area. The design of the mixed loss ensures that the model can not only deal with category imbalance, but also accurately capture the change area, thereby improving the overall effect of remote sensing image change detection.

[0151] Among them, in the field of change detection, since the number of unchanged pixels is often much larger than the number of changed pixels, in order to weaken the impact of sample imbalance, a hybrid loss function is usually used, that is, a combination of weighted cross entropy loss and Dice loss. The cross entropy loss handles the sample imbalance problem by adjusting the category weights, calculates the loss of each pixel, and averages the loss values ​​of all pixels. The Dice loss measures the overlap between the prediction and the true annotation, optimizes the model to increase the overlap, and finally optimizes the parameters through the hybrid loss function, thereby improving the prediction performance and accuracy of the model.

[0152] Verification experiment

[0153] like Figure 2 , Figure 3 and Figure 4 As shown, the effect of hyperspectral change detection can be shown. Figure 2 and Figure 3 are hyperspectral images of the same area at two different time points. Figure 4 To use the change detection results of the method of the present invention, Figure 5 It is a true value map of actual ground changes; from the detection results and the true value map of actual ground changes, it can be seen that the method proposed in the present invention can effectively identify the changed area and improve the accuracy of remote sensing image change detection.

[0154] In summary, the present invention adaptively mixes the CNN module with the Transformer module, utilizes the advantage of the CNN module in extracting local feature information of the image, and the advantage of the multi-scale Transformer module in extracting global feature information of the image, so as to better fuse all the global and local information of the high-resolution remote sensing image in the feature extraction stage, thereby ensuring the richness of the feature information of the feature extraction layer. At the same time, a multi-scale attention mechanism is used in the decoder of the U-shaped structure to perform dimensionality reduction operations on high-dimensional features by performing feature extraction and fusion at different scales, which not only reduces the complexity of attention calculation, but also can effectively solve the limitation that feature extraction of a single scale is often difficult to fully capture the information in the high-resolution remote sensing image, and ensures that the pixel detail information of the high-resolution remote sensing image can be retained, thereby accurately identifying the changes in the high-resolution remote sensing images of the same area at different times, ensuring the expression of shallow and deep information of the high-resolution remote sensing images, and improving the accuracy of change detection of high-resolution remote sensing images. The detection calculation complexity is low and the detection efficiency is fast.

[0155] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A transformation detection method for remote sensing images based on multi-scale global and local feature fusion, characterized in that: The following steps are involved: Step 1: Input two high-resolution remote sensing images of the same area at different times and group them by channel; Step 2: extract and merge the grouped high-resolution remote sensing images to obtain global and local feature information at multiple scales; Step 3: The output of the global and local feature information at different scales in the ConvTrans module is input into the CC fusion module for feature fusion and output to obtain f′, f″, f″′, f″″; Step 4: Input f′, f", f″′, and f″″ into the multi-scale attention decoder of the U-shaped structure to obtain global and local information at multiple scales and generate a predicted change graph. Step 5: Use the mixed loss function L to predict the change graph Perform parameter optimization.

2. The method for detecting transformation of remote sensing images based on multi-scale global and local feature fusion according to claim 1, characterized in that: In step 1, two high-resolution remote sensing images of the same area at different times are input and grouped by channel. The specific steps are as follows: 1.1, let T 1 represents the remote sensing image of the previous phase, T 2 represents the later phase remote sensing image, and T∈{T i ∈R C×H×W |i=1,2}; Where H is the height of the high-resolution remote sensing image, W is the width of the high-resolution remote sensing image, C represents the number of channels of the high-resolution remote sensing image, that is, there are C channels; 1.

2. Divide the channels into X 1 , X 2 There are two groups, and the grouping results are: X 1 =ChannelSplit(T i ) X 2 =ChannelSplit(T i ) Among them, ChannelSplit() is a channel splitting operation.

3. The method for detecting transformation of remote sensing images based on multi-scale global and local feature fusion according to claim 1, characterized in that: In step 2, feature extraction and feature merging are performed on the grouped high-resolution remote sensing images to obtain global and local feature information at multiple scales, namely: T 1 , T 2 X 1 , X 2 The outputs are respectively input into the mixed modules of Transformer module and CNN module, and the outputs of the two modules are combined to obtain the global and local feature information at multiple scales. Among them, the global and local feature information at multiple scales includes T 1 The global and local feature information F 11 , global and local feature information F 21 , global and local feature information F 31 , global and local feature information F 41 ; and T 2 The global and local feature information F 12 , global and local feature information F 22 , global and local feature information F 32 , global and local feature information F 42 .

4. The method for detecting transformation of remote sensing images based on multi-scale global and local feature fusion according to claim 3, characterized in that: Specifically, taking the first ConvTransBlock module as the benchmark, we get the global local feature information F 11 The steps are as follows: 2.1、X 1 The input is downsampled into the CNN module with a convolutional layer size of 3×3, and X is extracted 1 Local feature information The specific formula is as follows: Among them, Conv(·) is the convolution operation; 2.

2. X 2 After downsampling through the 1×1 convolutional layer, it is input into the Transformer module to extract X 2 Global feature information The specific formula is as follows: F Z =Conv(X 2 ) Where MST(·) represents the Multiscale Transformer module; 2.3 Local feature information and global feature information Perform channel splicing to obtain global and local feature information F 11 , the specific formula is as follows: in, Represents a channel concatenation operation.

5. The method for detecting transformation of remote sensing images based on multi-scale global and local feature fusion according to claim 4, characterized in that: In step 2, the global local feature information F is obtained 21 , global and local feature information F 31 , global and local feature information F 41 The specific steps are as follows: 2.

4. Taking the second ConvTransBlock module as the benchmark, the global local feature information F 11 Input into the ConvTransBlock module, group by channel, and repeat steps 2.1-2.3 to obtain the global local feature information F 21 ; 2.

5. Based on the third ConvTransBlock module, the global local feature information F 21 Input into the ConvTransBlock module, group by channel, and repeat steps 2.1-2.3 to obtain the global local feature information F 31 ; 2.6, based on the fourth ConvTransBlock module, the global local feature information F 31 Input into the ConvTransBlock module, group by channel, and repeat steps 2.1-2.3 to obtain the global local feature information F 41 .

6. The method for detecting transformation of remote sensing images based on multi-scale global and local feature fusion according to claim 5, characterized in that: In step 2, we get T 2 The global and local feature information F 12 , global and local feature information F 22 , global and local feature information F 32 , global and local feature information F 42 The specific steps are: Repeat steps 2.1-2.6 to obtain the global and local feature information F 12 , global and local feature information F 22 , global and local feature information F 32 , global and local feature information F 42 ; Among them, T 1 、T 2 The outputs of the hybrid network of the Transformer module and the CNN module of the two branches at four different stages are 7. The method for detecting transformation of remote sensing images based on multi-scale global and local feature fusion according to claim 1, characterized in that: In step 3, the output of the global and local feature information at different scales in the ConvTrans module is input into the CC fusion module for feature fusion and output to obtain f′, f″, f″′, and f″″. The specific steps are as follows: The outputs of the ConvTra ns module at different scales are input into the CC fusion module for channel splicing, and a 1×1 convolution is performed to obtain f′, f″, f″′, f″″. The specific formula is as follows; 8. The method for detecting transformation of remote sensing images based on multi-scale global and local feature fusion according to claim 1, characterized in that: In step 4, f′, f″, f″′, and f″″ are input into the multi-scale attention decoder of the U-shaped structure to obtain the global and local information at multiple scales and generate a predicted change graph The specific steps are as follows: 4.

1. Input f' into the MSA module and downsample to obtain output D1. Add D1 and f'' to obtain D1' and then input it into the MSA module for downsampling to obtain D2. Add D2 and f'' to obtain D2' and then input it into the MSA module for downsampling to obtain D3. Add D3 and f'' to obtain D3' and then input it into the MSA module for downsampling to obtain D4. The specific formula is as follows: D1=MSA(f′) D2=MSA(D1′) D3=MSA(D2′) D4=MSA(D3′) Among them, MSA(·) represents the multi-scale attention module; 4.

2. Upsample and merge D1, D2, D3, and D4 and input them into the corresponding ECA module to generate a predicted change graph. The specific formula is as follows: d4′=ECA(D4) Among them, d i i={1, 2, 3, 4} in ′, Among them, ECA(·) is the operation of the ECA module, is the matrix multiplication operation, Up(·) is an upsampling operation.

9. The method for detecting transformation of remote sensing images based on multi-scale global and local feature fusion according to claim 8, characterized in that: In step 5, the hybrid loss function L is used to predict the change map Parameter optimization is performed, and the specific details are as follows: L=L wce +L dice Among them, L is the total mixed loss function, that is, the weighted cross entropy loss L wce and Dice loss L dice of and; It is the predicted change graph An element in represents the prediction result of a specific position in the graph. Y is the true change graph, L wce is the weighted cross entropy loss, L dice For Dice loss.

Images