Double-time remote sensing image semantic change detection method based on Fourier transform

Abstract

The invention provides a two-time remote sensing image semantic change detection method based on Fourier transform. The method comprises the steps of selecting original remote sensing image data and performing preprocessing; performing feature extraction on the original remote sensing image data to obtain shallow features and deep features; generating binary change features according to the deep features; performing multi-scale feature fusion and decoding on the binary change features to obtain decoded binary change features; performing multi-scale feature fusion and decoding on the deep features and the shallow features to obtain dual-temporal semantic features; semantic change modeling is carried out in the frequency domain and the space domain, and binary change features and dual-temporal semantic features after semantic change modeling are obtained; generating a dual-temporal semantic segmentation graph and a binary change detection graph through classification and up-sampling; obtaining a semantic change graph through mask multiplication; according to the invention, the change position of the dual-time-phase remote sensing image can be accurately detected, and the detail capturing capability of the change area is improved.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing images, and in particular to a dual-time remote sensing image semantic change detection method based on Fourier transform. Background Art

[0002] As the main spatial information technology, remote sensing has gradually become a cutting-edge tool for exploring the earth's environment, detecting local ecological changes, and even studying the earth's historical changes. Among them, semantic change detection of remote sensing images, as a key technology, aims to monitor the changes of surface elements in the same area at different times, and has been widely used in environmental monitoring, urban expansion and reconstruction, and disaster assessment. With the rapid development of aerospace and satellite earth observation technology, remote sensing images have the characteristics of high resolution, high spectrum, and complex details. Therefore, they can provide high-precision, large-scale, and periodic surface information, and are widely used in environmental monitoring, urban expansion and reconstruction, and disaster assessment. At present, detecting and identifying the changes of typical surface elements in the same area at different times, that is, semantic change detection of remote sensing images, is a difficult and hot spot in the field of remote sensing interpretation. Semantic change detection of remote sensing images refers to clearly marking the transition between specific land cover types in the change area while identifying pixel changes or locating the change area.

[0003] The semantic change detection technology of remote sensing images can be roughly divided into change detection based on statistics and expert knowledge and change detection based on deep learning. One of the most intuitive methods of the former is to use the method of first classification and then comparison.

[0004] Post-classification comparison (PCC) directly compares the statistics of land cover / land use (LCLU) classification maps to generate changes, but because the LCLU maps of the two remote sensing images are generated separately, the temporal correlation between the two temporal remote sensing images is ignored. In addition, the detection results of the classification method often depend on the classification accuracy, resulting in error accumulation and inaccurate boundaries.

[0005] In the change detection based on deep learning, early researchers proposed a semantic change detection method combining CNN-RNN, which extracts semantic features through CNN and models the temporal correlation of bi-phase images through RNN module; Daudt et al. proposed a three-branch convolutional neural network to deconstruct the two subtasks under semantic change detection, modeling semantic information and change information respectively. Because the semantic features of multiple phases are embedded in the network separately, the learning of temporal dependency is ignored. Ding et al. proposed to use the temporal and spatial attention module to model the bi-phase semantic association, which is essentially a self-attention mechanism. The above models have improved the feature representation to a certain extent, but the network's modeling ability of spatiotemporal domain dependency and long-range dependency is insufficient, and there is a lack of mining and utilization of relevant information in the frequency domain. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a Fourier transform dual-temporal remote sensing image semantic change detection method, which can accurately detect the change position of the dual-temporal remote sensing image and improve the ability to capture the details of the changed area.

[0007] The present invention provides a dual-time remote sensing image semantic change detection method based on Fourier transform, comprising the following steps:

[0008] S1. Select the public dual-temporal remote sensing image semantic change detection dataset as the original remote sensing image data, and preprocess the original remote sensing image data;

[0009] S2, extracting features from the pre-processed original remote sensing image data to obtain shallow features and deep features;

[0010] S3, generating a binary change feature indicating the change area according to the deep feature;

[0011] S4, perform multi-scale feature fusion and decoding on the binary change features to obtain decoded binary change features; perform multi-scale feature fusion and decoding on the deep features and shallow features to obtain dual-phase semantic features;

[0012] S5, performing semantic change modeling on the decoded binary change features and bi-temporal semantic features in the frequency domain and the spatial domain to obtain the binary change features and bi-temporal semantic features after semantic change modeling;

[0013] S6, generating a bi-temporal semantic segmentation map and a binary change detection map by classifying and upsampling the binary change features and the bi-temporal semantic features after semantic change modeling;

[0014] S7. Mask multiply the bi-temporal semantic segmentation map and the binary change detection map to obtain a semantic change map.

[0015] In a possible implementation, step S1 includes the following steps:

[0016] S11. Select a public dual-temporal remote sensing image semantic change detection dataset as original remote sensing image data, where the original remote sensing image data includes multiple groups of high-resolution optical remote sensing images, each group of high-resolution optical remote sensing images includes multiple high-resolution optical images and corresponding semantic change labels; divide the original remote sensing image data into training samples and test samples;

[0017] S12. In the training samples and test samples, normalize the uint type samples whose pixel values ​​are in the interval [0, 255] to the double type training samples and test samples whose pixel values ​​are in the interval [0, 1];

[0018] S13. Enhance the double type training samples whose pixel values ​​are in the interval [0,1] by random rotation, translation, scaling, cropping or flipping.

[0019] In a possible implementation, step S2 includes the following steps:

[0020] S21. Build an improved twin Resnet34 network with weight sharing. The improved twin Resnet34 network includes five layers: Layer0, Layer1, Layer2, Layer3 and Layer4. Layer0 includes a 7×7 convolution layer, a BN layer, a Relu nonlinear activation layer and a maximum pooling layer. The structures of Layer1, Layer2, Layer3 and Layer4 are consistent with those of the corresponding layers in the public Resnet34 network. The convolution step size of the downsampling layer in Layer3 and Layer4 is 1.

[0021] S22. Perform feature extraction and obtain shallow features and deep features through the following operations:

[0022] F1=maxpool(Relu(BN(Conv(x1))))),

[0023] F1 low_feature =Layer 1(F1),

[0024] F1 deep_feature =Layer4(Layer3(Layer2(F1 low_feaure ))),

[0025] F2=maxpool(Relu(BN(Conv(x2)))),

[0026] F2 low_feature=Layer 1(F2),

[0027] F2 deep_feature =Layer4(Layer3(Layer2(F2 low_feature )));

[0028] Among them, x1 represents the remote sensing image of the previous phase in the training sample, x2 represents the remote sensing image of the later phase in the training sample, Conv represents a 7×7 convolution operation, BN represents a batch normalization operation, Relu represents a nonlinear activation operation, maxpool represents a maximum pooling operation, and F1 low_feature Indicates the shallow features of the remote sensing image in the previous phase, and F1 deep_feature Indicates the deep features of the remote sensing image in the previous phase, F2 low_feature Indicates the shallow features of the remote sensing image in the later phase, F2 deep_feature Represents the deep features of remote sensing images in the later phase.

[0029] In a possible implementation, step S3 includes the following steps:

[0030] S31, construct a binary change detection module, the binary change detection module includes five residual blocks connected in series, each residual block includes a 3×3 convolution layer, a BN layer and a ReLU layer;

[0031] S32, generating a binary change feature indicating a change region through the following operation:

[0032] F cd =cat(F1 deep_feature , F2 deep_feature ),

[0033] F change_map =Res_CD(F cd );

[0034] Among them, cat represents the concatenation operation; Res_CD represents the binary change detection module, and F change_map Represents binary change features.

[0035] In a possible implementation, step S4 includes the following steps:

[0036] S41. Construct a multi-scale feature fusion network and output bi-temporal semantic features through the following operation:

[0037] F upi =ConvTrans(Fi deep_feature ),

[0038] F low_feat =Relu(BN(Conv11(Fi low_feature))),

[0039] F outi =Relu(BN(Conv(cat(F upi : F lpw_feat )))), i=1, 2;

[0040] Among them, ConvTrans represents the transposed convolution operation, Conv11 represents the 1×1 convolution operation, and Fi deep_feature Represents the deep features of the i-th phase, Fi low_feature represents the shallow features of the i-th phase;

[0041] S42, and output the decoded binary change feature through the following operation:

[0042] F up =ConvTrans(F change_map ),

[0043] F low_feat_change =Relu(BN(Conv11(cat(F1 low_feature , F2 low_feature )))),

[0044] F out_change =Relu(BN(Conv(cat(F up , F low_feat_change )))).

[0045] In a possible implementation, step S5 includes the following steps:

[0046] S51. Construct a neural network and model the semantic changes in the frequency domain through the following operations:

[0047] X=cat(F out1 , F out2 , F out_change ),

[0048]

[0049]

[0050] Where FFT represents two-dimensional discrete Fourier transform, ⊙ represents element-by-element multiplication, W represents weight parameter, IFFT represents two-dimensional inverse Fourier transform, LN represents normalization operation, MLP represents multilayer perceptron, X n Represents the output of the nth frequency domain;

[0051] S52, perform semantic change modeling in the spatial domain and obtain binary change features and bi-temporal semantic features after modeling through the following operation:

[0052]

[0053]

[0054]

[0055] Where SA represents the self-attention layer in the visual Transformer, x m represents the output of the mth attention block; sep represents the separation operation along the channel, represents the semantic features of the pre-temporal phase after modeling, represents the post-temporal semantic features after modeling, Represents the binary change features after modeling.

[0056] In a possible implementation, step S6 includes the following steps:

[0057] S61, generate a bi-temporal semantic segmentation map and a binary change detection map through the following operation:

[0058]

[0059]

[0060]

[0061]

[0062]

[0063]

[0064] Among them, Upsample represents the bilinear interpolation operation, SMP1 represents the semantic segmentation map of the previous phase, SMP2 represents the semantic segmentation map of the later phase, and change_map represents the binary change detection map.

[0065] In a possible implementation, in step S7, mask multiplication is performed by the following operation to generate a semantic change map:

[0066] semantic_change_map1=SMP1⊙change_map,

[0067] semantic_change_map2=SMP2⊙change_map,

[0068] Among them, semantic_change_map1 represents the semantic change map of the previous phase, and semantic_change_map2 represents the semantic change map of the subsequent phase.

[0069] Compared with the prior art, the present application has the following advantages: ResNet34 with weight sharing is used to extract multi-scale features of remote sensing images. Through feature extraction, the model can more comprehensively capture the detail information and global information in the image, thereby enhancing the robustness of the model, which enables the model to maintain excellent performance when facing changes in different scales, angles and lighting conditions. The present invention combines Fourier transform and visual Transformer, so that the model can fully model the frequency domain while paying attention to long-range dependencies in the spatiotemporal domain. Therefore, the detection method provided by the present invention has reasonable steps and realizes the fusion of multi-scale deep features and shallow features, so that the changing position of dual-phase remote sensing images can be accurately detected, and the ability to capture details of the changing area can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a schematic diagram of the process of the present invention;

[0071] Figure 2 This is a network structure diagram of the present invention;

[0072] Figure 3 It is a schematic diagram of a binary change detection module of the present invention;

[0073] Figure 4 It is a schematic diagram of feature extraction of the present invention;

[0074] Figure 5 A schematic diagram of semantic change modeling of the present invention;

[0075] Figure 6 It is a schematic diagram of the dual-temporal remote sensing image and the label before enhancement of the present invention;

[0076] Figure 7 It is a schematic diagram of the dual-temporal remote sensing image and label enhancement of the present invention;

[0077] Figure 8 This is a comparison diagram of the present invention with other methods. DETAILED DESCRIPTION

[0078] First, those skilled in the art should understand that these implementations are only used to explain the technical principles of the embodiments of the present application, and are not intended to limit the protection scope of the embodiments of the present application. Those skilled in the art can make adjustments to them as needed to adapt to specific application scenarios.

[0079] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Figure 2The CD Module in the figure represents the binary change detection module, SpectFormer represents the semantic change modeling part; Decoder represents the multi-scale feature fusion and decoding part; ClassfierSeg represents the classification part; Upsample represents the upsampling part.

[0080] This embodiment discloses a method for detecting semantic changes in dual-time remote sensing images based on Fourier transform, comprising the following steps:

[0081] S1. Select the public dual-temporal remote sensing image semantic change detection dataset as the original remote sensing image data, and preprocess the original remote sensing image data;

[0082] S2, extracting features from the pre-processed original remote sensing image data to obtain shallow features and deep features;

[0083] S3, generating a binary change feature indicating the change area according to the deep feature;

[0084] S4, perform multi-scale feature fusion and decoding on the binary change features to obtain decoded binary change features; perform multi-scale feature fusion and decoding on the deep features and shallow features to obtain dual-phase semantic features;

[0085] S5, performing semantic change modeling on the decoded binary change features and bi-temporal semantic features in the frequency domain and the spatial domain to obtain the binary change features and bi-temporal semantic features after semantic change modeling;

[0086] S6, generating a bi-temporal semantic segmentation map and a binary change detection map by classifying and upsampling the binary change features and the bi-temporal semantic features after semantic change modeling;

[0087] S7. Mask multiply the bi-temporal semantic segmentation map and the binary change detection map to obtain a semantic change map.

[0088] Step S1 includes the following steps:

[0089] S11. Select a public dual-temporal remote sensing image semantic change detection dataset as original remote sensing image data, where the original remote sensing image data includes multiple groups of high-resolution optical remote sensing images, each group of high-resolution optical remote sensing images includes multiple high-resolution optical images and corresponding semantic change labels; divide the original remote sensing image data into training samples and test samples;

[0090] S12. In the training samples and test samples, normalize the uint type samples whose pixel values ​​are in the interval [0, 255] to the double type training samples and test samples whose pixel values ​​are in the interval [0, 1];

[0091] S13. Enhance the double type training samples whose pixel values ​​are in the interval [0, 1] by random rotation, translation, scaling, cropping or flipping.

[0092] Specifically, the binary temporal remote sensing image semantic change detection dataset is the SECOND dataset, which contains 2968 pairs of public binary temporal remote sensing images and their corresponding semantic change maps. In this embodiment, images are randomly selected and the original remote sensing image data is divided into training samples and test samples at a ratio of 8:2, that is, 2376 pairs of images are used as training samples, and the remaining 592 pairs of images are used as test samples. Figure 6 and attached Figure 7 As shown, the resolution of the remote sensing image in the enhanced training sample is 512 × 512. The enhancement method uses random rotation, translation, scaling, cropping or flipping, that is, at least one or more of the above methods.

[0093] As attached Figure 4 As shown, step S2 includes the following steps:

[0094] S21. Build an improved twin Resnet34 network with weight sharing. The improved twin Resnet34 network includes five layers: Layer0, Layer1, Layer2, Layer3 and Layer4. Layer0 includes a 7×7 convolution layer, a BN layer, a Relu nonlinear activation layer and a maximum pooling layer. The structures of Layer1, Layer2, Layer3 and Layer4 are consistent with those of the corresponding layers in the public Resnet34 network. The convolution step size of the downsampling layer in Layer3 and Layer4 is 1.

[0095] S22. Perform feature extraction and obtain shallow features and deep features through the following operations:

[0096] F1=maxpool(Relu(BN(Conv(x1)))),

[0097] F1 low_feature =Layer 1(F1),

[0098] F1 deep_feature =Layer4(Layer3(Layer2(F1 low_feature ))),

[0099] F2=maxpool(Relu(BN(Conv(x2)))),

[0100] F2 low_feature =Layer 1(F2),

[0101] F2deep_feature =Layer4(Layer3(Layer2(F2 low_feature )));

[0102] Among them, x1 represents the remote sensing image of the previous phase in the training sample, x2 represents the remote sensing image of the later phase in the training sample, Conv represents a 7×7 convolution operation, BN represents a batch normalization operation, Relu represents a nonlinear activation operation, maxpool represents a maximum pooling operation, and Fl represents a 2×2 convolution operation. low_ f eatura Indicates the shallow features of the remote sensing image in the previous phase, and F1 deep_feature Indicates the deep features of the remote sensing image in the previous phase, F2 low_feature Indicates the shallow features of the remote sensing image in the later phase, F2 deep_feature Represents the deep features of remote sensing images in the later phase.

[0103] Specifically, the improved twin Resnet34 network is divided into two paths, and the weights are shared between the two paths. Both paths include five layers: Layer0, Layer1, Layer2, Layer3 and Layer4. Layer0 includes a 7×7 convolution layer, a BN layer, a Relu nonlinear activation layer and a maximum pooling layer. The structures of Layer1, Layer2, Layer3 and Layer4 are consistent with the structures of the corresponding layers in the public Resnet34 network. The step size of all convolution layers in Layer3 and Layer4 is 1; in the two paths of the improved twin Resnet34 network, one is used to process remote sensing images of the previous phase, and the other is used to process remote sensing images of the later phase; the step size of all convolution layers in Layer3 and Layer4 is 1 to keep the deep features in the original Figure 1 / 8 spatial resolution. The above formula means that in the improved twin Resnet34 network, the input remote sensing image is sequentially subjected to convolution operation, batch normalization operation and nonlinear activation operation in Layer0, and then subjected to a maximum pooling operation to obtain features with a size of 1 / 4 of the original image, and then input into Layer1, Layer2, Layer3 and Layer4. In the public Resnet34 network, Layer1, Layer2, Layer3 and Layer4 are all stacked by multiple residual blocks, and deep features are obtained by Layer1, Layer2, Layer3 and Layer4 in turn; shallow features are directly obtained in Layer1.

[0104] As attached Figure 2 and attached Figure 3 As shown, step S3 includes the following steps:

[0105] S31, construct a binary change detection module, which includes five residual blocks connected in series, each of which includes a 3×3 convolutional layer, a BN layer and a Rel u layer;

[0106] S32, generating a binary change feature indicating a change region through the following operation:

[0107] F cd =cat(F1 deep_feature , F2 deep_faature ),

[0108] F change_map =Res_CD(F cd );

[0109] Among them, cat represents the concatenation operation; Res_CD represents the binary change detection module, and F change_map Represents binary change features.

[0110] Step S4 includes the following steps:

[0111] S41. Construct a multi-scale feature fusion network and output bi-temporal semantic features through the following operation:

[0112] F upi =ConvTrans(Fi deep _ feature ),

[0113] F low_feat =Relu(BN(Conv11(Fi low_feature ))),

[0114] F outi =Relu(BN(Conv(cat(F upi , F low_feat )))), i=1, 2;

[0115] Among them, ConvTrans represents the transposed convolution operation, Conv11 represents the 1×1 convolution operation, and Fi deep_feature Represents the deep features of the i-th phase, Fi low_featue represents the shallow features of the i-th phase;

[0116] S42, and output the decoded binary change feature through the following operation:

[0117] F up =ConvTrans( F c bange_map ),

[0118] F low_feat_change=Relu(BN(Conv11(cat(F1 low_feature , F2 low_feature )))),

[0119] F out_channge =Relu(BN(Conv(cat(F up , F low_feat_change )))).

[0120] Specifically, as attached Figure 2 As shown, the binary change detection module needs to be connected to the output of the improved twin Resnet34 network to generate the deep features of the output of the improved twin Resnet34 network into binary change features; the shallow features of the two-phase are jump-connected to the decoder in the multi-scale feature fusion network, and the two-phase semantic features are obtained after processing by the multi-scale feature fusion network; the binary change detection module is also connected to the decoder in the multi-scale feature fusion network, so that the output binary change features and the shallow features enter the multi-scale feature fusion network together, and the decoded binary change features are obtained after processing by the multi-scale feature fusion network.

[0121] As attached Figure 5 As shown, step S5 includes the following steps:

[0122] S51. Construct a neural network and model the semantic changes in the frequency domain through the following operations:

[0123] X=cat(F outl , F out2 , F out_change ),

[0124]

[0125]

[0126] Where FFT represents two-dimensional discrete Fourier transform, ⊙ represents element-by-element multiplication, W represents weight parameter, IFFT represents two-dimensional inverse Fourier transform, LN represents normalization operation, MLP represents multilayer perceptron, X n Represents the output of the nth frequency domain;

[0127] S52, perform semantic change modeling in the spatial domain and obtain binary change features and bi-temporal semantic features after modeling through the following operation:

[0128]

[0129]

[0130]

[0131] Where SA represents the self-attention layer in the visual Transformer, x m represents the output of the mth attention block; sep represents the separation operation along the channel, represents the semantic features of the pre-temporal phase after modeling, represents the post-temporal semantic features after modeling, Represents the binary change features after modeling.

[0132] Specifically, the constructed neural network model needs to include the layers described in the steps, so that the data can be processed in the neural network as described in the steps; in step S51, first, F out1 、F out2 and F out_change It is spliced ​​along the channel dimension and then converted into a two-dimensional discrete Fourier transform in the frequency domain. A learnable weight parameter W is introduced for element-by-element multiplication, and then the feature is mapped back to the spatial domain through a two-dimensional inverse Fourier transform, replacing the self-attention sublayer in the traditional visual Transformer. The entire frequency domain module consists of n frequency domain blocks, namely Spectral Block. In step S52, after modeling the semantic changes in the frequency domain, it is necessary to continue to pay attention to the correlation in the embedded semantic space, and then use the visual Transformer to deeply model the semantic change dependencies in the entire spatial domain. The entire spatial domain module consists of m attention blocks, namely Attention Block.

[0133] Step S6 includes the following steps:

[0134] S61, generate a bi-temporal semantic segmentation map and a binary change detection map through the following operation:

[0135]

[0136]

[0137]

[0138]

[0139]

[0140]

[0141] Among them, Upsample represents the bilinear interpolation operation, SMP1 represents the semantic segmentation map of the previous phase, SMP2 represents the semantic segmentation map of the later phase, and change_map represents the binary change detection map.

[0142] Specifically, the bi-temporal semantic features and binary change features are projected to the corresponding dimensions through a 1×1 convolution operation, which is also classification. Finally, bilinear interpolation is used to upsample the image to restore the original image size.

[0143] In step S7, the semantic change map is generated by performing mask multiplication through the following operation:

[0144] semantic_change_map1=SMP1⊙change_map,

[0145] semantic_change_map2=SMP2⊙change_map,

[0146] Among them, semantic_change_map1 represents the semantic change map of the previous phase, and semantic_change_map2 represents the semantic change map of the subsequent phase.

[0147] As attached Figure 8 As shown, attached Figure 8 The partial detection results of this application and six popular semantic change detection methods on the preprocessed SECOND dataset are shown, and different colors represent different land cover types. By comparing the visualization results, it can be found that the present invention can deeply associate the features of different levels of dual-phase remote sensing images and improve the model's ability to express semantic features. It combines the information of the frequency domain and the spatial domain to enhance the expression of features, thereby more effectively capturing the complex change patterns in remote sensing images and improving the network's ability to capture details of the changing area. In order to more comprehensively evaluate the accuracy of the proposed method.

[0148] Table 1 below shows the evaluation indicators of this application and other semantic change detection methods on the SECOND dataset through the test samples. It can be seen that the indicators of this application's method basically exceed those of other methods, especially in the Sek and mIoU indicators, which reach 23.72 and 73.45% respectively. This shows that this method can well model the semantic information of long-distance contexts, and detect the change categories of interest by extracting and identifying complex features, which is more advantageous for semantic change detection of remote sensing images.

[0149]

[0150] Table 1

[0151] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "in the present embodiment", "specific example", or "some examples" etc. means that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0152] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A dual-temporal remote sensing image semantic change detection method based on Fourier transform, characterized in that: The following steps are involved: S1. Select a public dual-temporal remote sensing image semantic change detection dataset as original remote sensing image data, and preprocess the original remote sensing image data; S2, extracting features from the pre-processed original remote sensing image data to obtain shallow features and deep features; S3, generating a binary change feature indicating a change area according to the deep feature; S4, performing multi-scale feature fusion and decoding on the binary change feature to obtain the decoded binary change feature; performing multi-scale feature fusion and decoding on the deep feature and the shallow feature to obtain a dual-phase semantic feature; S5, performing semantic change modeling on the decoded binary change features and bi-phase semantic features in the frequency domain and the spatial domain to obtain the binary change features and bi-phase semantic features after semantic change modeling; S6, generating a bi-phase semantic segmentation map and a binary change detection map by classifying and upsampling the binary change features and the bi-phase semantic features after semantic change modeling; S7. Mask multiplication of the bi-temporal semantic segmentation map and the binary change detection map is performed to obtain a semantic change map.

2. The method for detecting semantic changes in dual-temporal remote sensing images based on Fourier transform according to claim 1, characterized in that: Step S1 includes the following steps: S11, selecting a public dual-temporal remote sensing image semantic change detection dataset as original remote sensing image data, wherein the original remote sensing image data includes multiple groups of high-resolution optical remote sensing images, each group of the high-resolution optical remote sensing images includes multiple high-resolution optical images and corresponding semantic change labels; dividing the original remote sensing image data into training samples and test samples; S12. In the training samples and test samples, normalize the uint type samples whose pixel values ​​are in the interval [0, 255] to the double type training samples and test samples whose pixel values ​​are in the interval [0, 1]; S13. Enhance the double type training samples whose pixel values ​​are in the interval [0,1] by random rotation, translation, scaling, cropping or flipping.

3. The method for detecting semantic changes in dual-temporal remote sensing images based on Fourier transform according to claim 2, characterized in that: Step S2 includes the following steps: S21. Build an improved twin Resnet34 network with weight sharing, wherein the improved twin Resnet34 network includes five layers: Layer0, Layer1, Layer2, Layer3 and Layer4. The Layer0 includes a 7×7 convolution layer, a BN layer, a Relu nonlinear activation layer and a maximum pooling layer. The structures of the Layer1, Layer2, Layer3 and Layer4 are consistent with the structures of the corresponding layers in the public Resnet34 network. The convolution step size of the downsampling layer in the Layer3 and Layer4 is 1. S22. Perform feature extraction and obtain shallow features and deep features through the following operations: F1=maxpool(Relu(BN(Conv(x l )))), F1 low_feature =Layer1(F1), F1 deep_feature =Layer4(Layer3(Layer2(F1 low_feature ))), F2=maxpool(Relu(BN(Conv(x2)))), <h2 style=";text-align:left;direction:ltr">F2<h2 style=";text-align:left;direction:ltr"> low_feature <h2 style=";text-align:left;direction:ltr"> Layer 1(F2) <h2 style=";text-align:left;direction:ltr">F2<h2 style=";text-align:left;direction:ltr"> deep_feature <h2 style=";text-align:left;direction:ltr"> =Layer4(Layer3(Layer2(F2<h2 style=";text-align:left;direction:ltr"> low_feature <h2 style=";text-align:left;direction:ltr"> ))); Among them, x1 represents the remote sensing image of the previous phase in the training sample, x2 represents the remote sensing image of the later phase in the training sample, Conv represents a 7×7 convolution operation, BN represents a batch normalization operation, Relu represents a nonlinear activation operation, maxpool represents a maximum pooling operation, and F1 low_feature Indicates the shallow features of the remote sensing image in the previous phase, and F1 deep_feature Indicates the deep features of the remote sensing image in the previous phase, F2 low_feature Indicates the shallow features of the remote sensing image in the later phase, F2 deep_feature Represents the deep features of remote sensing images in the later phase.

4. The method for detecting semantic changes in dual-time remote sensing images based on Fourier transform according to claim 3, characterized in that: Step S3 includes the following steps: S31, constructing a binary change detection module, wherein the binary change detection module includes five residual blocks connected in series, each of which includes a 3×3 convolutional layer, a BN layer, and a ReLU layer; S32, generating a binary change feature indicating a change region through the following operation: F cd =cat(F1 deep_feature ,F2 deep_feature ), F change_map =Res_CD(F cd ); Wherein, cat represents the concatenation operation; Res_CD represents the binary change detection module, F change_map Represents binary change features.

5. The method for detecting semantic changes in dual-temporal remote sensing images based on Fourier transform according to claim 4, characterized in that: Step S4 includes the following steps: S41, construct a multi-scale feature fusion network, and output the bi-temporal semantic features through the following operation: F upi =ConvTrans(Fi deep_feature ), F low_feat =Relu(BN(Conv11(Fi low_feature ))), F outi =Relu(BN(Conv(cat(F upi ,F low_feat )))),i=1,2; Among them, ConvTrans represents the transposed convolution operation, Conv11 represents the 1×1 convolution operation, and F ideep_feature Represents the deep features of the i-th phase, Fi low_feature represents the shallow features of the i-th phase; S42, and output the decoded binary change feature through the following operation: F up =ConvTrans(F change_map ), F low_fest_change =Relu(BN(Conv11(cat(F1) low_feature ,F2 low_feature )))), F out_change =Relu(BN(Conv(cat(F up ,F low_feat_change ))))。 6. The method for detecting semantic changes in dual-temporal remote sensing images based on Fourier transform according to claim 5, characterized in that: Step S5 includes the following steps: S51. Construct a neural network and model the semantic changes in the frequency domain through the following operations: X=cat(F out1 ,F out2 ,F out_change ), Where FFT represents two-dimensional discrete Fourier transform, ⊙ represents element-by-element multiplication, W represents weight parameter, IFFT represents two-dimensional inverse Fourier transform, LN represents normalization operation, MLP represents multilayer perceptron, X n Represents the output of the nth frequency domain; S52, perform semantic change modeling in the spatial domain and obtain the binary change features and bi-temporal semantic features after modeling through the following operation: Where SA represents the self-attention layer in the visual Transformer, x m represents the output of the mth attention block; sep represents the separation operation along the channel, represents the semantic features of the pre-temporal phase after modeling, represents the post-temporal semantic features after modeling, Represents the binary change features after modeling.

7. The method for detecting semantic changes in dual-temporal remote sensing images based on Fourier transform according to claim 6, characterized in that: Step S6 includes the following steps: S61, generating the bi-temporal semantic segmentation map and the binary change detection map through the following operations: Among them, Upsample represents the bilinear interpolation operation, SMP1 represents the semantic segmentation map of the previous phase, SMP2 represents the semantic segmentation map of the later phase, and change_map represents the binary change detection map.

8. The method for detecting semantic changes in dual-temporal remote sensing images based on Fourier transform according to claim 7, characterized in that: In step S7, the semantic change map is generated by performing mask multiplication through the following operation: semantic_change_map1=SMP1⊙change_map, semantic_change_map2=SMP2⊙change_map, Among them, semantic_change_map1 represents the semantic change map of the previous phase, and semantic_change_map2 represents the semantic change map of the subsequent phase.

Images