A dual-temporal remote sensing change detection method combining local representation and global modeling

Abstract

The present invention discloses a dual-time remote sensing change detection method combining local representation and global modeling, including: obtaining a dual-time remote sensing image pair and dividing the image into sub-images of the same size; establishing a dual backbone network model and pre-training; using the pre-trained dual backbone network model to extract features from the sub-images, and correspondingly obtaining multi-scale local feature maps and global feature maps; performing a dual-branch information cross-fusion operation on the local feature maps and the global feature maps at the same spatial resolution to obtain a joint representation; performing a multi-scale feature map fusion operation on the joint representation of different spatial resolutions to obtain a multi-scale fused feature map; performing difference, upsampling and convolution transformations on the multi-scale fused feature maps obtained from the corresponding sub-images in the image pair in sequence to obtain a change detection map. The method can achieve better target positioning, edge refinement and semantic understanding, ensure detection accuracy and generalization performance, and has a fast data processing speed.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a dual-time remote sensing change detection method combining local representation and global modeling. Background Art

[0002] Change detection in dual-temporal remote sensing images is crucial for understanding land surface changes and has been widely studied in applications such as disaster loss assessment, land cover mapping, and urban expansion investigation. With the development of high-resolution optical sensors, the availability of high-resolution remote sensing images has been increasing, thus broadening the application scope of change detection in high-resolution dual-temporal images.

[0003] The change detection model CNN based on convolutional network has achieved good detection performance due to the characteristics of local connection and weight sharing. However, it is limited by the size of receptive field and local feature extraction and does not have the ability to model global context information. Once the Transformer network was applied from the field of natural language processing to computer vision, it caused a huge sensation due to its long-distance dependency characteristics in the self-attention mechanism. However, the global features captured by different network stages in the Transformer are extremely similar and redundant. Therefore, a dual-temporal remote sensing image change detection method with better detection accuracy and generalization performance is proposed. Summary of the invention

[0004] The purpose of the present invention is to propose a dual-time remote sensing change detection method that combines local representation and global modeling in response to the above problems, which has the advantages of both CNN model and Transformer model, and establishes long-distance dependency while introducing local inductive bias, thereby ensuring the feature complementarity and effective representation of the model, and can achieve better target positioning, edge refinement and semantic understanding, further ensuring the detection accuracy and generalization performance of dual-time remote sensing images for change detection, and fast data processing speed.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] The present invention proposes a dual-time remote sensing change detection method combining local representation and global modeling, comprising the following steps:

[0007] S1, obtaining a dual-time remote sensing image pair, and dividing two images in the image pair into a number of sub-images of the same size;

[0008] S2. Establish a dual backbone network model and perform pre-training. The dual backbone network model includes a parallel CNN model and a Transformer model.

[0009] S3. Use the pre-trained dual backbone network model to extract features from the sub-graph, and obtain the multi-scale local feature map F output by the CNN model.local And the multi-scale global feature map F output by the Transformer model global ;

[0010] S4, for the local feature map F at the same spatial resolution local and the global feature map F global Perform a double-branch information cross-fusion operation, local feature map F local The size is W×H×L local , global feature map F global The size is W×H×L global , where W is the local feature map F at the current spatial resolution local Or the global feature map F global The width of H is the local feature map F at the current spatial resolution. local Or the global feature map F global Height, L local is the local feature map F at the current spatial resolution local The channel depth, L global is the global feature map F at the current spatial resolution global The channel depth of the two branches is as follows:

[0011] S41, local feature map F local and the global feature map F global The convolution operations are unified into a unified size of W×H×d, where d is the local feature map F local Or the global feature map F global The unified channel depth;

[0012] S42, linear expansion of local feature map F local and the global feature map F global Get the corresponding vector group representation, which is recorded as (Q local ,K local ,V local ) and (Q global ,K global ,V global ), where Q local is the local feature map F local The query matrix, K local is the local feature map F local The bond matrix, V local is the local feature map F local The value matrix, Q global is the global feature map F global The query matrix, K global is the global feature map F global The bond matrix, V global is the global feature map Fglobal The value matrix of is N×d, N=W×H;

[0013] S43, calculate local feature map F local The cross attention output F cross-local and the global feature map F global The cross attention output F cross-global , the dimensions are W×H×d;

[0014] S44, F cross-local and F cross-global Stacked along the channel direction, and the joint representation F of size W×H×d is obtained through the residual module cross ;

[0015] S5. Joint representation of different spatial resolutions F cross Perform multi-scale feature map fusion operation to obtain multi-scale fusion feature map f fuse , multi-scale fusion feature map f fuse The size of the corresponding subgraph joint representation F cross The maximum size is W1×H1×d1;

[0016] S6. Obtain the multi-scale fusion feature map f from the corresponding sub-images in the image pair fuse The difference, upsampling and convolution transformation are performed in sequence to obtain a change detection map F with a size of nW1×nH1×2 final , where n is the upsampling multiple.

[0017] Preferably, in step S43, the local feature map F local The cross attention output F cross-local and the global feature map F global The cross attention output F cross-global , the calculation formula is as follows:

[0018]

[0019]

[0020] Among them, softmax(·) is a nonlinear mapping function and T is a transposition operation.

[0021] Preferably, in step S5, the multi-scale feature map fusion operation is specifically as follows:

[0022] S51. Joint representation at different spatial resolutions F cross In the example, two sizes are selected in order from small to large, and the large size is the low-level feature map f low , the smaller size is the high-level feature map f high ;

[0023] S52, for the low-level feature map f low After performing convolution operations and upsampling in sequence, the first mask map mask is generated low , the first mask map mask low With the high-level feature map f high Perform Hadamard product to generate high-level enhanced feature map f en-high ;

[0024] S53, for high-level feature map f high After the convolution operation, the second mask map mask is generated high , the second mask map mask high With the low-level feature map f low Perform Hadamard product to generate low-level enhanced feature map f en-low ;

[0025] S54, for high level enhanced feature map f en-high and the low-level enhanced feature map f en-low Perform stacking and convolution operations in the channel direction in sequence to form an initial fusion feature map;

[0026] S55, determine whether the size of the initial fusion feature map is a joint representation F cross If the maximum size is, the initial fusion feature map is used as the multi-scale fusion feature map f fuse Otherwise, the initial fusion feature map is classified as the joint representation F cross The size of the initial fused feature map is taken as the minimum size, and the process returns to step S51.

[0027] Preferably, in step S52, the first mask image mask low By low Convolution operation, normalization, convolution operation and upsampling generation are performed in sequence.

[0028] Preferably, in step S53, the low-level feature map f low With the second mask high Before performing the Hadamard product, it is also preprocessed, and the preprocessing includes convolution operation, normalization and upsampling in sequence.

[0029] Preferably, in step S52 or S53, after performing the Hadamard product, a corresponding enhanced feature map is generated by an activation function.

[0030] Preferably, in step S54, after performing the convolution operation, the initial fused feature map is obtained by normalization and activation function in sequence.

[0031] Preferably, in step S6, upsampling is 4 times interpolation upsampling.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] This application uses a dual backbone network model composed of a CNN model and a Transformer model in parallel to extract a dual-branch multi-scale feature map of a dual-time remote sensing image pair, including a multi-scale local feature map and a multi-scale global feature map. The local and global features are combined through a dual-branch information cross-fusion operation, and a local inductive bias is introduced while establishing long-distance dependency, thereby ensuring feature complementarity and effective representation of the model; and a multi-scale feature map fusion operation is used to fuse low-level feature maps that are helpful for target positioning and boundary estimation and high-level feature maps that are helpful for pixel-level semantic understanding, thereby achieving better target positioning, edge refinement and semantic understanding, further ensuring the detection accuracy and generalization performance of change detection of dual-time remote sensing images, and fast data processing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the structure of the dual-time remote sensing change detection method combining local representation and global modeling of the present invention;

[0035] Figure 2 It is a structural schematic diagram of the double-branch information cross-fusion operation of the present invention;

[0036] Figure 3 It is a structural schematic diagram of the multi-scale feature fusion operation of the present invention;

[0037] Figure 4 2 is a diagram showing the detection results of an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0040] This application combines the advantages of CNN and Transformer to construct a dual backbone network model to extract dual-branch multi-scale feature maps, and obtains local and global joint feature representations through dual-branch information cross-fusion, thereby ensuring the feature complementarity and effective characterization of the model; and constructs multi-scale feature fusion to further fuse feature maps at different levels to achieve better target positioning, edge refinement and semantic understanding, thereby fully ensuring the accuracy and generalization performance of dual-time remote sensing image change detection.

[0041] like Figure 1-4 As shown, a dual-temporal remote sensing change detection method combining local representation and global modeling includes the following steps:

[0042] S1. Obtain a dual-time remote sensing image pair, and divide the two images in the image pair into a number of sub-images of the same size.

[0043] like Figure 1 As shown, the two images in the dual-time remote sensing image pair are denoted as X1 and X2, respectively. As input data, in order to improve data processing speed and accuracy, the acquired dual-time remote sensing image pair is first preprocessed, and each image is cut into sub-images of uniform spatial size (such as 256×256).

[0044] S2. Establish a dual backbone network model and perform pre-training. The dual backbone network model includes a parallel CNN model and a Transformer model.

[0045] S3. Use the pre-trained dual backbone network model to extract features from the sub-graph, and obtain the multi-scale local feature map F output by the CNN model. local And the multi-scale global feature map F output by the Transformer model global .

[0046] Given the multi-scale local feature map F output by the CNN model local In the early stage of the backbone network, the receptive field is relatively small, and the extracted features are more focused on local inductive bias. As the CNN network deepens, its receptive field also expands slightly, but long-distance dependency modeling is still missing. Transformer outputs a multi-scale global feature map F global , by considering the self-attention mechanism of the similarity or affinity between each position, the long-distance dependency is captured with a receptive field that is approximately global from the shallow network, resulting in more similarities between the representations obtained by the shallow and deep layers of the Transformer, leading to excessive redundancy. This application combines the advantages of the CNN model and the Transformer model to construct a dual backbone network model for multi-scale feature extraction of dual-time remote sensing images.

[0047] S4, for the local feature map F at the same spatial resolutionlocal and the global feature map F global Perform a double-branch information cross-fusion operation, local feature map F local The size is W×H×L local , global feature map F global The size is W×H×L global , where W is the local feature map F at the current spatial resolution local Or the global feature map F global The width of H is the local feature map F at the current spatial resolution. local Or the global feature map F global Height, L local is the local feature map F at the current spatial resolution local The channel depth, L global is the global feature map F at the current spatial resolution global The channel depth of the two branches is as follows:

[0048] S41, local feature map F local and the global feature map F global The convolution operations are unified into a unified size of W×H×d, where d is the local feature map F local Or the global feature map F global The unified channel depth;

[0049] S42, linear expansion of local feature map F local and the global feature map F global Get the corresponding vector group representation, which is recorded as (Q local ,K local ,V local ) and (Q global ,K global ,V global ), where Q local is the local feature map F local The query matrix, K local is the local feature map F local The bond matrix, V local is the local feature map F local The value matrix, Q global is the global feature map F global The query matrix, K global is the global feature map F global The bond matrix, V global is the global feature map F global The value matrix of is N×d, N=W×H;

[0050] S43, calculate local feature map F local The cross attention output F cross-localand the global feature map F global The cross attention output F cross-global , the dimensions are W×H×d.

[0051] In one embodiment, in step S43, the local feature map F local The cross attention output F cross-local and the global feature map F global The cross attention output F cross-global , the calculation formula is as follows:

[0052]

[0053]

[0054] Among them, softmax(·) is a nonlinear mapping function and T is a transposition operation.

[0055] S44, F cross-local and F cross-global Stacked along the channel direction, and the joint representation F of size W×H×d is obtained through the residual module cross The calculation formula is as follows:

[0056] F cross =Residual(Concat(F cross-local ,F cross-global ))

[0057] Among them, Residual(·) is the residual module, and Concat(·) means stacking along the channel direction.

[0058] To address the limited receptive field in convolution operations and the high redundancy in the self-attention mechanism, a cross-attention mechanism is introduced to simultaneously introduce local inductive bias and global dependency modeling in the data representation stage to achieve efficient cross-fusion of dual-branch information.

[0059] S5. Joint representation of different spatial resolutions F cross Perform multi-scale feature map fusion operation to obtain multi-scale fusion feature map f fuse , multi-scale fusion feature map f fuse The size of the corresponding subgraph joint representation F cross The maximum size is W1×H1×d1.

[0060] In one embodiment, in step S5, the multi-scale feature map fusion operation is specifically as follows:

[0061] S51. Joint representation at different spatial resolutions F cross In the example, two sizes are selected in order from small to large, and the large size is the low-level feature map f low, the smaller size is the high-level feature map f high ;

[0062] S52, for the low-level feature map f low After performing convolution operations and upsampling in sequence, the first mask map mask is generated low , the first mask map mask low With the high-level feature map f high Perform Hadamard product to generate high-level enhanced feature map f en-high ;

[0063] S53, for high-level feature map f high After the convolution operation, the second mask map mask is generated high , the second mask map mask high With the low-level feature map f low Perform Hadamard product to generate low-level enhanced feature map f en-low ;

[0064] S54, for high level enhanced feature map f en-high and the low-level enhanced feature map f en-low Perform stacking and convolution operations in the channel direction in sequence to form an initial fusion feature map;

[0065] S55, determine whether the size of the initial fusion feature map is the joint representation F cross If the maximum size is, the initial fusion feature map is used as the multi-scale fusion feature map f fuse Otherwise, the initial fusion feature map is classified as the joint representation F cross The size of the initial fused feature map is taken as the minimum size, and the process returns to step S51.

[0066] In one embodiment, in step S52, the first mask image mask low By low Convolution operation, normalization, convolution operation and upsampling generation are performed in sequence.

[0067] In one embodiment, in step S53, the low-level feature map f low With the second mask high Before performing the Hadamard product, the data is preprocessed, and the preprocessing includes sequentially performing convolution operation, normalization, and upsampling. It should be noted that the convolution operation and normalization in the preprocessing can be combined with the generation of the first mask map mask. low The convolution operations and normalization performed sequentially can be shared or performed separately.

[0068] In one embodiment, in step S52 or S53, after performing the Hadamard product, a corresponding enhanced feature map is generated by an activation function.

[0069] In one embodiment, in step S54, after performing the convolution operation, an initial fused feature map is obtained through normalization and activation functions in sequence.

[0070] In this embodiment, Figure 3 As shown in Figure 2, the multi-scale feature map fusion operation is as follows:

[0071] 1) High-level enhanced feature map f en-high The calculation formula is as follows:

[0072] stmask low =UP(Conv(BN(Conv(f low ))))

[0073] 2) Low-level enhanced feature map f en-low The calculation formula is as follows:

[0074] stmask high =Conv(f high )

[0075] 3) The calculation formula of the initial fusion feature map is as follows:

[0076] f fuse =ReLU(BN(Conv(Concat(ReLU(f en-low ),ReLU(f en-high )))))

[0077] Among them, is the Hadamard product, UP(·) is the upsampling operation, Conv(·) is the convolution operation, ReLU(·) is the nonlinear mapping function, BN(·) is the normalization operation, and Concat(·) is the connection function.

[0078] In the remote sensing image change detection task, the sizes of the changed objects are different. Introducing feature maps with different resolution levels will further improve the generalization performance and detection accuracy of the model. cross Perform a multi-scale feature map fusion operation to merge the low-level feature maps f that are helpful for target positioning and boundary estimation low and a high-level feature map f that facilitates pixel-level semantic understanding high Perform inter-scale fusion to obtain a multi-scale fusion feature map f fuse , further improve the model's representation capabilities, achieve better target positioning, edge refinement and semantic understanding, and further enhance the model's detection accuracy.

[0079] S6. Obtain the multi-scale fusion feature map f from the corresponding sub-images in the image pair fuseThe difference, upsampling and convolution transformation are performed in sequence to obtain a change detection map F with a size of nW1×nH1×2 final , where n is the upsampling multiple.

[0080] In one embodiment, in step S6, upsampling is 4 times interpolation upsampling.

[0081] As a preferred embodiment, taking the high-resolution remote sensing images of Guangzhou City, Guangdong Province in 2006 and 2019 as an example, Figure 4 X1 is the image collected in 2006, X2 is the image collected in 2019, (a)-(f) are six groups of images, and the dual-time remote sensing change detection method of combined local representation and global modeling provided by this application is used for experiments. The images of the detection dataset contain three bands of R, G, and B, with a resolution of 0.55m. The specific steps are as follows:

[0082] Step 1: Load high-resolution remote sensing images of Guangzhou City, Guangdong Province in 2006 and 2019, select the three displayed bands (default R, G, B), perform a 256*256 cutting operation on the dual-time remote sensing image pairs and their corresponding detection label maps, and divide the data into three parts: training set, validation set, and test set.

[0083] Step 2: Pre-train the CNN model (specifically using the Resnet-18 network) and the Transformer model (specifically using the PVTv2-B1 network) in the dual backbone network model on the ImageNet classification dataset. The CNN model and the Transformer model may also use other applicable networks in the prior art.

[0084] Step 3: Load the training set and validation set data cut in step 1, and feed them into the parallel CNN model and Transformer model to capture multi-scale local feature maps F with sizes of (64×64×64, 32×32×128, 16×16×256) respectively. local and (64×64×64, 32×32×128, 16×16×320) multi-scale global feature maps F global .

[0085] Step 4: Perform a dual-branch information cross-fusion operation to combine the local feature maps F of the CNN model and the Transformer model with the same spatial resolution. local and the global feature map F global Fusion, obtain the multi-scale joint representation F with the size of (64×64×64, 32×32×64, 16×16×64) cross .

[0086] Step 5: Perform a multi-scale feature map fusion operation. The high-level feature map of size 16×16×64 after the cross-fusion of the dual-branch information is fused with the lower-level feature map of size 32×32×64 to output a feature map of size 32×32×64 (initial fused feature map). The feature map of size 32×32×64 is then fused with the lowest-level feature map of size 64×64×64 to obtain a multi-scale fused feature map f of size 64×64×64. fuse .

[0087] Step 6: Output the multi-scale fusion feature map f of size 64×64×64 for both dual-time images fuse The difference, 4x interpolation upsampling and convolution operations are performed in sequence to obtain a change detection map F with a size of 256×256×2 final The cross entropy loss function and the stochastic gradient descent algorithm can be used to minimize the loss between the detection prediction graph and the label graph to complete the model training, which is a technique well known to those skilled in the art and will not be described in detail here.

[0088] Step 7: Load the trained parameters of this method, obtain the predicted output on the test set, and compare the change detection results with the test set labels to calculate the precision, recall, F1 score, intersection-over-union ratio, and overall accuracy, as shown in Figure 4 As shown, there are six groups of image pairs (a) to (f), which are the comparison results of the change detection diagrams of the proposed method (the method of the present application) and the reference method (dual time converter). The detection evaluation results are shown in Table 1.

[0089] Table 1

[0090]

[0091] Among them, the dual time transformer is a method proposed in the prior art, reference: H.Chen, Z.Qi and Z.Shi,"Remote Sensing Image Change Detection With Transformers,"in IEEE Transactions on Geoscience and Remote Sensing,vol.60,pp.1-14,2022,Art no.5607514,doi:10.1109 / TGRS.2021.3095166. The proposed method is the method proposed in this application. The detection results of the proposed method in Table 1 exceed the results of the latest dual time transformer method in all aspects. Figure 4 It has fewer missed detections (light gray pixels) and false detections (dark gray pixels), as well as higher accuracy (white pixels), which further verifies the effectiveness of our method in the task of change detection in dual-temporal remote sensing images.

[0092] This application uses a dual backbone network model composed of a CNN model and a Transformer model in parallel to extract a dual-branch multi-scale feature map of a dual-time remote sensing image pair, including a multi-scale local feature map and a multi-scale global feature map. The local and global features are combined through a dual-branch information cross-fusion operation, and a local inductive bias is introduced while establishing long-distance dependency, thereby ensuring feature complementarity and effective representation of the model; and a multi-scale feature map fusion operation is used to fuse low-level feature maps that are helpful for target positioning and boundary estimation and high-level feature maps that are helpful for pixel-level semantic understanding, thereby achieving better target positioning, edge refinement and semantic understanding, further ensuring the detection accuracy and generalization performance of change detection of dual-time remote sensing images, and fast data processing speed.

[0093] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] The above-described embodiments only express the more specific and detailed embodiments described in this application, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of this application, which all belong to the protection scope of this application. Therefore, the protection scope of the patent application shall be based on the attached claims.

Claims

1. A dual-temporal remote sensing change detection method combining local representation and global modeling, characterized by: The dual-time remote sensing change detection method combining local representation and global modeling comprises the following steps: S1, obtaining a dual-time remote sensing image pair, and dividing two images in the image pair into a number of sub-images of the same size; S2. Establish a dual backbone network model and perform pre-training, wherein the dual backbone network model includes a parallel CNN model and a Transformer model; S3. Use the pre-trained dual backbone network model to extract features from the sub-graph, and obtain the multi-scale local feature map F output by the CNN model. local And the multi-scale global feature map F output by the Transformer model global ; S4, for the local feature map F at the same spatial resolution local and the global feature map F global Perform a double-branch information cross-fusion operation, the local feature map F local The size is W×H×L local , the global feature map F global The size is W×H×L global , where W is the local feature map F at the current spatial resolution local Or the global feature map F global The width of H is the local feature map F at the current spatial resolution. local Or the global feature map F global Height, L local is the local feature map F at the current spatial resolution local The channel depth, L global is the global feature map F at the current spatial resolution global The channel depth of the dual-branch information cross-fusion operation is as follows: S41, local feature map F local and the global feature map F global The convolution operations are unified into a unified size of W×H×d, where d is the local feature map F local Or the global feature map F global The unified channel depth; S42, linear expansion of local feature map F local and the global feature map F global Get the corresponding vector group representation, which is recorded as (Q local ,K local ,V local ) and (Q global ,K global ,V global ), where Q local is the local feature map F local The query matrix, K local is the local feature map F local The bond matrix, V local is the local feature map F local The value matrix, Q global is the global feature map F global The query matrix, K global is the global feature map F global The bond matrix, V global is the global feature map F global The value matrix of is N×d, N=W×H; S43, calculate local feature map F local The cross attention output F cross-local and the global feature map F global The cross attention output F cross-global , the dimensions are W×H×d; S44, F cross-local and F cross-global Stacked along the channel direction, and the joint representation F of size W×H×d is obtained through the residual module cross ; S5. Joint representation of different spatial resolutions F cross Perform multi-scale feature map fusion operation to obtain multi-scale fusion feature map f fuse , the multi-scale fusion feature map f fuse The size of the corresponding subgraph joint representation F cross The maximum size is W1×H1×d1; S6. Obtain the multi-scale fusion feature map f from the corresponding sub-images in the image pair fuse The difference, upsampling and convolution transformation are performed in sequence to obtain a change detection map F with a size of nW1×nH1×2 final , where n is the upsampling multiple.

2. The dual-temporal remote sensing change detection method combining local representation and global modeling as claimed in claim 1, characterized in that: In step S43, the local feature map F local The cross attention output F cross-local and the global feature map F global The cross attention output F cross-global , the calculation formula is as follows: Among them, softmax(·) is a nonlinear mapping function, and T is a transposition operation.

3. The dual-temporal remote sensing change detection method combining local representation and global modeling as claimed in claim 1, characterized in that: In step S5, the multi-scale feature map fusion operation is specifically as follows: S51. Joint representation at different spatial resolutions F cross In the example, two sizes are selected in order from small to large, and the large size is the low-level feature map f low , the smaller size is the high-level feature map f high ; S52, for the low-level feature map f low After performing convolution operations and upsampling in sequence, the first mask map mask is generated low , the first mask map mask low With the high-level feature map f high Perform Hadamard product to generate high-level enhanced feature map f en-high ; S53, for high-level feature map f high After the convolution operation, the second mask map mask is generated high , the second mask map mask high With the low-level feature map f low Perform Hadamard product to generate low-level enhanced feature map f en-low ; S54, for high level enhanced feature map f en-high and the low-level enhanced feature map f en-low Perform stacking and convolution operations in the channel direction in sequence to form an initial fusion feature map; S55, determine whether the size of the initial fusion feature map is the joint representation F cross If the maximum size is, the initial fusion feature map is used as the multi-scale fusion feature map f fuse Otherwise, the initial fusion feature map is classified as the joint representation F cross The size of the initial fused feature map is taken as the minimum size, and the process returns to step S51.

4. The dual-temporal remote sensing change detection method combining local representation and global modeling as claimed in claim 3, characterized in that: In step S52, the first mask image mask low By low Convolution operation, normalization, convolution operation and upsampling generation are performed in sequence.

5. The dual-temporal remote sensing change detection method combining local representation and global modeling as claimed in claim 3, characterized in that: In step S53, the low-level feature map f low With the second mask high Before performing the Hadamard product, the data is preprocessed, wherein the preprocessing includes sequentially performing convolution operations, normalization, and upsampling.

6. The dual-temporal remote sensing change detection method combining local representation and global modeling as claimed in claim 3, characterized in that: In step S52 or S53, after executing the Hadamard product, a corresponding enhanced feature map is generated through an activation function.

7. The dual-temporal remote sensing change detection method combining local representation and global modeling as claimed in claim 3, characterized in that: In step S54, after performing the convolution operation, an initial fused feature map is obtained through normalization and activation functions in sequence.

8. The dual-temporal remote sensing change detection method combining local representation and global modeling as claimed in claim 1, characterized in that: In step S6, the upsampling is 4 times interpolation upsampling.

Images