Remote sensing image change detection method and system based on prior knowledge and frequency domain learning

Abstract

The invention relates to the technical field of remote sensing images, in particular to a remote sensing image change detection method and system based on priori knowledge and frequency domain learning, and the method comprises the steps: inputting a first time phase remote sensing image and a second time phase remote sensing image to be detected into a trained remote sensing image change detection model, and obtaining change prediction results of the two images; the model performs feature extraction processing on the first time-phase remote sensing image to obtain a preprocessing feature of the first time-phase remote sensing image; performing feature extraction processing on the second time-phase remote sensing image to obtain a preprocessing feature of the second time-phase remote sensing image; performing feature filtering on the preprocessing features of the first time phase remote sensing image and the preprocessing features of the second time phase remote sensing image to obtain filtered frequency features; further processing the filtered frequency features to generate final difference fusion features; and based on the final difference fusion features, generating a change prediction result of the two images. According to the invention, an accurate change prediction result can be provided.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing images, and in particular to a remote sensing image change detection method and system based on prior knowledge and frequency domain learning. Background Art

[0002] In recent years, with the rapid development of aerospace technology, a large number of high-performance remote sensing satellites have been launched. These satellites are equipped with advanced sensors and imaging devices, capable of observing the Earth's surface with unprecedented accuracy and frequency, which has greatly improved the ability to obtain high-resolution, multi-spectral remote sensing images. As a result, many large and diverse remote sensing image datasets have been established. These datasets cover various geographical information of vast regions across the globe, providing a solid data foundation for remote sensing image analysis methods based on deep learning.

[0003] Remote sensing image change detection is an important surface observation task. By using dual-temporal images collected by remote sensing satellites to monitor surface object changes, remarkable progress has been made in many practical applications. Specifically, by comparing remote sensing images of different time phases, researchers can accurately monitor urban construction, land use changes, and environmental changes; in the field of natural disaster prevention and control, forest fires, flood-prone areas, and post-earthquake damage can be monitored in real time, providing effective support for emergency rescue and post-disaster reconstruction; in agricultural damage assessment, the growth status of crops and the occurrence of pests and diseases can be monitored to help farmers take targeted measures to reduce economic losses. With the rapid development of deep learning, many deep learning-based methods have been used for remote sensing image change detection. For example, convolutional neural networks, due to their significant advantages in extracting spatial detail features, many change detection methods use them to extract the features of dual-temporal images respectively, and further fuse the obtained dual-temporal features to identify change regions. However, affected by environmental factors such as illumination and atmosphere, there are two objective factors in remote sensing image change detection: some attributes of the same object may be different in different time phases, and some attributes of different objects may be similar. For example, the same building has different colors under different illuminations, and different types of crops may all be green, which may lead to false changes or missed detections, posing challenges to deep learning-based methods. Summary of the Invention

[0004] To solve the deficiencies of the prior art, the present invention provides a remote sensing image change detection method and system based on prior knowledge and frequency domain learning; On the one hand, a remote sensing image change detection method based on prior knowledge and frequency domain learning is provided, including: Obtain the first-temporal remote sensing image and the second-temporal remote sensing image to be detected; Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images. Among them, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-phase remote sensing image by using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; perform feature extraction processing on the second-phase remote sensing image by using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image by using an adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features by using a decoder to obtain multi-scale dual-phase fusion features; generate a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features; generate a final difference fusion feature based on the multi-scale difference fusion feature map; and generate the change prediction results of the two images based on the final difference fusion feature.

[0005] On the other hand, a remote sensing image change detection system based on prior knowledge and frequency domain learning is provided, including: An acquisition module configured to: acquire the first-phase remote sensing image and the second-phase remote sensing image to be detected; A detection module configured to: input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images. Among them, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-phase remote sensing image by using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; perform feature extraction processing on the second-phase remote sensing image by using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image by using an adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features by using a decoder to obtain multi-scale dual-phase fusion features; generate a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features; generate a final difference fusion feature based on the multi-scale difference fusion feature map; and generate the change prediction results of the two images based on the final difference fusion feature.

[0006] The above technical solution has the following advantages or beneficial effects: First, the present invention analyzes and summarizes the method for humans to accurately judge changing objects, that is, to judge changing objects based on the comparison results of discriminative attributes (color, boundary, texture) between objects, rather than the overall comparison method of spatial features that is currently popular in deep learning networks. For example, humans can identify the same building by using the discriminative boundary attribute rather than the color attribute, and identify different crops based on the differences in texture attributes. Guided by this prior knowledge of attribute comparison, the present invention proposes to decompose an RGB image into frequency features of different bands by using a sliding discrete cosine transform. These features of different bands can be considered to represent different attributes. For example, the low-frequency band features are generally related to the color attribute, while the high-frequency band features are related to the texture attribute. Subsequently, these frequency features are fed into an encoder based on a convolutional neural network to further extract and enhance these features, and an adaptive gate unit is used to filter out interfering band (attribute) features and retain discriminative band features. Finally, an accurate change prediction is given based on these discriminative band features, so as to effectively cope with the above objective factors. Compared with the attribute comparison method proposed by the present invention, the currently popular overall comparison method of spatial features is restricted by the inseparability of attributes in the spatial domain and is inconsistent with the prior knowledge of humans for change detection, thus resulting in more false changes or missed detected changes.

[0007] Different from the currently popular deep learning change detection method based on the overall comparison of spatial features, the present invention is inspired by human prior knowledge and embeds the attribute comparison method based on frequency learning into the deep learning network. Specifically, the present invention proposes to decompose the bi-temporal RGB image into frequency features of different bands by using a sliding discrete cosine transform. These features of different bands represent different attributes. At the same time, the present invention reorganizes, sorts, and balances the energy of these features for subsequent feature extraction processes and enhancing the robustness of the network. Subsequently, these frequency features are fed into an encoder based on a convolutional neural network, and an adaptive gate unit is used to filter out interfering band features and retain discriminative band features. Finally, these discriminative band features are progressively decoded in the decoder and an accurate change prediction is given. The present invention effectively reduces the situation of false changes and missed detected change regions, and finally obtains a remote sensing change detection model with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0009] Figure 1 It is a flowchart of the method for the first embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0010] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0011] Embodiment 1 This embodiment provides a remote sensing image change detection method based on prior knowledge and frequency domain learning; As Figure 1 shown, the remote sensing image change detection method based on prior knowledge and frequency domain learning includes: S101: Obtain the first-phase remote sensing image and the second-phase remote sensing image to be detected; S102: Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images; wherein, the trained remote sensing image change detection model is used for: performing feature extraction processing on the first-phase remote sensing image by using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; performing feature extraction processing on the second-phase remote sensing image by using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; performing feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image by using an adaptive gate unit to obtain the filtered frequency features; further processing the filtered frequency features by using a decoder to obtain the multi-scale dual-phase fusion features; generating a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features; generating the final difference fusion feature based on the multi-scale difference fusion feature map; and generating the change prediction results of the two images based on the final difference fusion feature.

[0012] Furthermore, the S101: Obtain the first-phase remote sensing image and the second-phase remote sensing image to be detected includes: obtaining by means of satellite remote sensing.

[0013] Furthermore, for the S102: Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images, the training process of the trained remote sensing image change detection model includes: Construct a data set, where the data set is a number of pairs of dual-phase remote sensing images with known remote sensing image change labels; each pair of dual-phase remote sensing images includes: a remote sensing image taken in the first phase and a remote sensing image taken in the second phase of the same geographical area; the first phase is earlier than the second phase; Divide the data set into: a training set, a validation set, and a test set according to a set ratio; Input the training set into the remote sensing image change detection model to train the model; stop training when the total loss function value of the model stops decreasing or the number of iterations exceeds the set number of times; Input the validation set into the trained remote sensing image change detection model to validate the model; Input the test set into the remote sensing image change detection model that has passed the validation to test the model; Take the model that meets the set test conditions as the trained remote sensing image change detection model.

[0014] It should be understood that the original remote sensing image dataset is divided into a training set, a validation set, and a test set in a ratio of 6:2:2. The model is trained on the training set, and the model with the best results on the validation set is used as the final convergent model, and the model effect is evaluated on the test set.

[0015] It should be understood that high-resolution remote sensing images have a large size. Limited by computing resources, the original dual-temporal images need to be cropped into image patches and then fed into the model. Among them, the dual-temporal image patches , correspond to each other in pairs. Subsequently, the present invention performs data augmentation. First, calculate the mean and standard deviation of the RGB three channels of all image patches, and standardize the pixel values of the image patches. Secondly, perform the same horizontal and vertical random flipping on the corresponding pair of dual-temporal image patches and exchange the time order of this pair of dual-temporal image patches and other data augmentation operations.

[0016] The present invention believes that the positions with values greater than 0.5 in the final prediction map belong to the changed area, and the positions less than 0.5 belong to the unchanged area. At the same time, the present invention generates a boundary label using the Canny operator based on the image label, and uses this boundary label to supervise the prediction map to further ensure the boundary accuracy of the prediction map. By comparing the prediction result with the image label and the boundary label, using the binary cross-entropy function as the loss function, calculating the loss value, and performing gradient backpropagation to update the model parameters.

[0017] The expression of the cross-entropy loss function of the trained remote sensing image change detection model is as follows: ; Among them, is the number of pixels of each image patch; is the number of categories, and the value here is 2. represents the th pixel point in the image patch, and its value range is [1, N]; represents the th category, and its value range is [1, M]; is the sign function. If the true class of a pixel point is equal to then take 1, otherwise take 0; is the probability value that the pixel point belongs to the class .

[0018] The present invention is trained within the framework of Pytorch using an NVIDIA GeForce RTX 3090 GPU. The stochastic gradient descent method is used and the initial learning rate is set to 0.01. The learning rate will linearly decay until the last training cycle. The total number of training cycles is 300, and the batch size is set to 32. The present invention performs validation during the training process and finally selects the model weights with the best performance on the validation set.

[0019] Further, in S102: The first-phase remote sensing image and the second-phase remote sensing image to be detected are input into the trained remote sensing image change detection model to obtain the change prediction results of the two images. The trained remote sensing image change detection model includes: A first encoder, a second encoder, an adaptive gate unit, and a decoder; The input end of the first encoder is used to input the first-phase remote sensing image to be detected; The input end of the second encoder is used to input the second-phase remote sensing image to be detected; The output ends of the first encoder and the second encoder are both connected to the input end of the adaptive gate unit; the output end of the adaptive gate unit is connected to the input end of the decoder; The output end of the decoder outputs the change prediction results of the two images.

[0020] Further, the first encoder includes: a first sliding discrete cosine transform module, a first batch normalization module, a first frequency feature extraction module of the first encoder, a second frequency feature extraction module of the first encoder, and a third frequency feature extraction module of the first encoder, which are connected in sequence.

[0021] Further, the second encoder includes: a second sliding discrete cosine transform module, a second batch normalization module, a first frequency feature extraction module of the second encoder, a second frequency feature extraction module of the second encoder, and a third frequency feature extraction module of the second encoder, which are connected in sequence.

[0022] Further, in S102: The first encoder is used to perform feature extraction processing on the first-phase remote sensing image to obtain the preprocessed features of the first-phase remote sensing image, which specifically includes: ; Among them, Indicates the conversion of an image from space to space. Indicates that the image is segmented in a sliding window manner. Indicates the discrete cosine transform. Indicates collecting and reorganizing the same band frequency features of different windows according to the spatial position of the original image. Indicates the channel sorting operation; Indicates the preprocessing features of the first-phase remote sensing image.

[0023] It should be understood that The obtained features have the following characteristics: the positional relationship is consistent with that of the original image pixels, but the size is reduced. Each channel of the obtained features corresponds to the features of different bands. These channels are sorted in ascending order according to their corresponding bands. The subsequent division criterion is based on the channel sequence number. Exemplarily, if the feature has 147 channels, then the 1st - 49th channels are low-frequency features, the 50th - 98th channels are medium-frequency features, and the 99th - 147th channels are high-frequency features.

[0024] ; Among them, represents batch normalization; Indicates the preprocessing features of the first-phase remote sensing image with energy balance. ; ; ; ; ; ; ; ; ; ; Among them, Indicates a two-dimensional convolution with a convolution kernel of and a stride of 2. Indicates splitting the feature along the channel dimension. Indicates concatenating the features along the channels; Indicates the th input low-frequency feature of the first-phase remote sensing image. Represents the th input intermediate-frequency feature of the first-phase remote sensing image, Represents the th input high-frequency feature of the first-phase remote sensing image, Represents the th low-frequency feature of the first-phase remote sensing image, Represents the th intermediate-frequency feature of the first-phase remote sensing image, Represents the th high-frequency feature of the first-phase remote sensing image; Represents the first half of the th low-frequency feature of the first-phase remote sensing image, Represents the second half of the th low-frequency feature of the first-phase remote sensing image, Represents the first half of the th intermediate-frequency feature of the first-phase remote sensing image, Represents the second half of the th intermediate-frequency feature of the first-phase remote sensing image, Represents the first half of the th high-frequency feature of the first-phase remote sensing image, Represents the second half of the th high-frequency feature of the first-phase remote sensing image; Represents the th low-intermediate-frequency feature of the first-phase remote sensing image, Represents the th mid-high-frequency feature of the first-phase remote sensing image, Represents the first-phase output feature of the th frequency feature extraction module of the first encoder; The value range of

[0025] should be understood that this method not only reduces the mixing degree of different bands but also avoids the closure of band features, enhancing the robustness and performance of the network.

[0026] It should be understood that, first, a method called sliding discrete cosine transform is proposed to extract the dual-phase frequency features with a size of and and , this method first converts the RGB image to the YCbCr image because the YCbCr space decomposes the luminance and chrominance, which is more in line with the perception characteristics of the human visual system.

[0027] Subsequently, a sliding window-based discrete cosine transform is performed on each channel of the YCbCr image, where the size and stride of the sliding window are customizable. The size of the window is proportional to the number of bands of the output frequency features, and the stride of the window determines the spatial dimension of the output frequency features. Subsequently, the present invention collects the same band frequency features located in different windows and reorganizes them into the same channel according to the positional relationship of the original image , of the original image. In this way, the present invention converts the RGB image into frequency features, where different channels of these frequency features represent different band features and the spatial positions are consistent with the original image. Next, the present invention sorts these frequency features in the order from low frequency to high frequency.

[0028] Compared with the traditional discrete cosine transform applied to image blocks, the sliding discrete cosine transform can simultaneously extract frequency features of rich bands and customize the output spatial dimension. In addition, it solves the problem of spatial discontinuity caused by the hard segmentation image block strategy. In the present invention, the size of the sliding window is set to 7 7, and the stride is set to 4. However, the frequency features extracted using the discrete cosine transform have more energy in the low-frequency band, which results in a greater impact of the low-frequency band features on the change prediction results. To solve this problem, the present invention introduces batch normalization to balance the energy and influence of features in different bands (channels) and enhances the robustness of the entire network.

[0029] Therefore, the present invention is fair to the frequency features in different bands and does not favor the low-frequency band features. To further extract deep frequency features, the present invention designs a frequency feature extraction module to generate multi-scale frequency features , , , , , , with sizes respectively: 64 64 147,64 64 147,32 32 294,32 32 294,16 16 588,16 16 588.

[0030] Taking the th phase remote sensing image in the network as an example, the th frequency feature extraction module, ( , ), whose input frequency characteristics are first divided into three parts: low-frequency characteristics , medium-frequency characteristics and high-frequency characteristics . Subsequently, the present invention further extracts and enhances them through different convolutional layers respectively. This band separation and enhancement strategy is to avoid the mixing between the discriminative band characteristics and the interfering band characteristics after decoupling as much as possible, so as to better extract the frequency characteristics of the discriminative band.

[0031] To solve the problems brought by hard band separation, the present invention connects the low-frequency characteristics of the latter half channels with the medium-frequency characteristics of the first half channels and calls it low-medium frequency characteristics ; connects the medium-frequency characteristics of the latter half channels with the high-frequency characteristics of the first half channels and calls it medium-high frequency characteristics . Then, the low-frequency characteristics of the first half channels , , and the high-frequency characteristics of the latter half channels are respectively sent into the convolutional layer for enhancement. Finally, they are concatenated together in the original channel order as the output .

[0032] Furthermore, the S102: using the second encoder to perform feature extraction processing on the second-phase remote sensing image to obtain the features of the second-phase remote sensing image specifically includes: ; ; ; ; ; ; ; ; ; ; ; ; represents the th input low-frequency characteristic of the second-phase remote sensing image, Denote the th input intermediate frequency feature of the second - phase remote - sensing image, Denote the th input high - frequency feature of the second - phase remote - sensing image, Denote the th low - frequency feature of the second - phase remote - sensing image, Denote the th intermediate frequency feature of the second - phase remote - sensing image, Denote the th high - frequency feature of the second - phase remote - sensing image; Denote the first half of the th low - frequency feature of the second - phase remote - sensing image, Denote the second half of the th low - frequency feature of the second - phase remote - sensing image, Denote the first half of the th intermediate frequency feature of the second - phase remote - sensing image, Denote the second half of the th intermediate frequency feature of the second - phase remote - sensing image, Denote the first half of the th high - frequency feature of the second - phase remote - sensing image, Denote the second half of the th high - frequency feature of the second - phase remote - sensing image; Denote the th low - intermediate frequency feature of the second - phase image, Denote the th middle - high frequency feature of the second - phase image, Denote the second - phase output feature of the th frequency - feature extraction module of the second encoder; The value range of is 1 to 3.

[0033] Furthermore, in step S102: An adaptive gate unit is used to perform feature filtering on the pre - processed features of the first - phase remote - sensing image and the pre - processed features of the second - phase remote - sensing image to obtain filtered frequency features, and a decoder is used to further process the filtered frequency features to obtain multi - scale dual - phase fusion features, including: ; ; ; ; ; ; ; , ; ; Among them, represents element-wise subtraction, represents the absolute value operation, represents the Sigmoid function, represents element-wise addition, represents element-wise multiplication.

[0034] represents the first-phase output feature of the th frequency feature extraction module of the first encoder, represents the second-phase output feature of the th frequency feature extraction module of the second encoder, represents the th frequency difference feature, represents the th low-frequency difference feature, represents the th mid-frequency difference feature, represents the th high-frequency difference feature, represents the th low-frequency difference attention, represents the th mid-frequency difference attention, represents the th high-frequency difference attention, represents the th global inter-band attention, represents the th gate attention, represents the th first-phase filtered frequency feature, represents th second-phase filtered frequency feature, represents the th auxiliary prediction.

[0035] ; ; Among them, represents the first multi-scale fusion feature of the th-phase remote sensing image, represents the second multi-scale fusion feature of the th-phase remote sensing image, represents concatenating the features along the channel dimension, Denotes an upsampling operation, Denotes the third filtered frequency feature of the temporal remote sensing image, Denotes the second filtered frequency feature of the temporal remote sensing image.

[0036] In order to retain the frequency features of discriminative bands and filter out the frequency features of interfering bands, the present invention designs an adaptive gate unit. Taking the i-th adaptive gate unit in the network (i = 1, 2, 3) as an example, the present invention first makes the input dual-temporal frequency features , perform pixel-by-pixel subtraction and absolute value operation to obtain the frequency difference feature . Next, the present invention divides the frequency difference feature into three parts according to the bands: the low-frequency difference feature , the middle-frequency difference feature and the high-frequency difference feature , which are respectively used to generate three local intra-band attentions , , . At the same time, from the global perspective of the bands, the present invention uses to obtain a global inter-band attention . Subsequently, the present invention connects along the channels , , , and the connected feature is added to and sent to function to obtain the gate attention , whose size is the same as that of the input and the value is limited to [0, 1]. Subsequently, the present invention multiplies and , element-wise to retain the frequency features of discriminative bands in the dual-temporal frequency features and filter out the frequency features of interfering bands. For example, when the value of a certain point in the feature corresponding to the gate attention is 1, it means to retain this point, and when it is 0, this point is filtered out. In addition, the present invention uses a change label to supervise the auxiliary prediction to further ensure the accuracy of the attention, which is generated by the difference feature obtained by element-wise multiplying and .

[0037] Finally, the present invention obtains the dual-temporal features processed by these gate units , , , , , and uses them for subsequent decoding. The sizes of these features are respectively: 64 64 147, 64 64 147,32 32 294,32 32 294,16 16 588,16 16 588。

[0038] During the decoding process, the present invention selects a dense decoding method. Specifically, the present invention first uses , , , , , and gradually generates multi-scale dual-temporal fusion features through upsampling operations, channel concatenation, and convolutional layers , .

[0039] Furthermore, the S102: generating a multi-scale difference fusion feature map based on the multi-scale dual-temporal fusion feature, and generating a final difference fusion feature based on the multi-scale difference fusion feature map, includes: ; ; ;

[0040] Wherein, represents the final difference fusion feature, represents the upsampling operation. represents the first multi-scale difference fusion feature, represents the first multi-scale fusion feature of the first-temporal remote sensing image, represents the first multi-scale fusion feature of the second-temporal remote sensing image, represents the second multi-scale difference fusion feature, represents the second multi-scale fusion feature of the first-temporal remote sensing image, represents the second multi-scale fusion feature of the second-temporal remote sensing image, represents the third multi-scale difference fusion feature, represents the third filtered frequency feature of the first-temporal remote sensing image, represents the 3rd filtered frequency feature of the second-temporal remote sensing image.

[0041] The multi-scale dual-temporal fusion feature map is used to generate a multi-scale difference fusion feature map , , , with sizes of 64 64 128, 32 32 128, 16 16 588. A final difference fusion feature DF with a size of 64 64 128 is obtained and used to generate the final prediction

[0042] Furthermore, in S102: Based on the final difference fusion feature, generate the change prediction results of the two images, specifically including: processing the final difference fusion feature based on the convolutional layer, and then processing the processing result through the activation function, and finally obtaining the change prediction results of the two images

[0043] Embodiment 2 This embodiment provides a remote sensing image change detection system based on prior knowledge and frequency domain learning The remote sensing image change detection system based on prior knowledge and frequency domain learning includes An acquisition module configured to: acquire the first-temporal remote sensing image and the second-temporal remote sensing image to be detected A detection module configured to: input the first-temporal remote sensing image and the second-temporal remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images; wherein, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-temporal remote sensing image using the first encoder to obtain the preprocessed features of the first-temporal remote sensing image; perform feature extraction processing on the second-temporal remote sensing image using the second encoder to obtain the preprocessed features of the second-temporal remote sensing image; perform feature filtering on the preprocessed features of the first-temporal remote sensing image and the preprocessed features of the second-temporal remote sensing image using the adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features using the decoder to obtain the multi-scale dual-temporal fusion features; generate a multi-scale difference fusion feature map based on the multi-scale dual-temporal fusion features; generate the final difference fusion feature based on the multi-scale difference fusion feature map; generate the change prediction results of the two images based on the final difference fusion feature

[0044] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A remote sensing image change detection method based on prior knowledge and frequency domain learning, characterized in that Including: Obtain the first-phase remote sensing image and the second-phase remote sensing image to be detected; Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images; wherein, the trained remote sensing image change detection model is used for: performing feature extraction processing on the first-phase remote sensing image by using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; performing feature extraction processing on the second-phase remote sensing image by using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; performing feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image by using an adaptive gate unit to obtain the filtered frequency features; further processing the filtered frequency features by using a decoder to obtain the multi-scale dual-phase fusion features; generating a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features; generating the final difference fusion feature based on the multi-scale difference fusion feature map; generating the change prediction results of the two images based on the final difference fusion feature.

2. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 1, characterized in that Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images. The trained remote sensing image change detection model includes: A first encoder, a second encoder, an adaptive gate unit and a decoder; The input end of the first encoder is used to input the first-phase remote sensing image to be detected; The input end of the second encoder is used to input the second-phase remote sensing image to be detected; The output ends of the first encoder and the second encoder are both connected to the input end of the adaptive gate unit; the output end of the adaptive gate unit is connected to the input end of the decoder; The output end of the decoder outputs the change prediction results of the two images; The first encoder includes: a first sliding discrete cosine transform module, a first batch normalization module, a first frequency feature extraction module of the first encoder, a second frequency feature extraction module of the first encoder, and a third frequency feature extraction module of the first encoder that are connected in sequence; The second encoder includes: a second sliding discrete cosine transform module, a second batch normalization module, a first frequency feature extraction module of the second encoder, a second frequency feature extraction module of the second encoder, and a third frequency feature extraction module of the second encoder that are connected in sequence.

3. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 2, characterized in that Performing feature extraction processing on the first-phase remote sensing image by using the first encoder to obtain the preprocessed features of the first-phase remote sensing image specifically includes: ； Among them, represents the conversion of the image from space to space, represents splitting the image in the form of a sliding window, represents the discrete cosine transform, represents collecting and reorganizing the same band frequency features of different windows according to the spatial positions of the original image, represents the channel sorting operation; represents the preprocessing features of the first-phase remote sensing image; ； Among them, represents batch normalization; represents the preprocessing features of the first-phase remote sensing image for energy balancing; ； ； ； ； ； ； ； ； ； ； Among them, represents a two-dimensional convolution with a convolution kernel of and a stride of 2, represents splitting the feature along the channel dimension, represents concatenating the features along the channels; representing the th input low-frequency feature of the first-phase remote sensing image, representing the th input mid-frequency feature of the first-phase remote sensing image, representing the th input high-frequency feature of the first-phase remote sensing image, representing the th low-frequency feature of the first-phase remote sensing image, representing the th mid-frequency feature of the first-phase remote sensing image, representing the th high-frequency feature of the first-phase remote sensing image; The first half of the n-th low-frequency feature of the first-phase remote sensing image, The second half of the n-th low-frequency feature of the first-phase remote sensing image, The first half of the n-th medium-frequency feature of the first-phase remote sensing image, The second half of the n-th medium-frequency feature of the first-phase remote sensing image, The first half of the n-th high-frequency feature of the first-phase remote sensing image, The second half of the n-th high-frequency feature of the first-phase remote sensing image; representing the th low- to medium-frequency feature of the first-phase remote sensing image, representing the th medium- to high-frequency feature of the first-phase remote sensing image, representing the first-phase output feature of the th frequency feature extraction module of the first encoder; The value range of is 1 to 3.

4. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 3, characterized in that, Performing feature extraction processing on the second-phase remote sensing image by using the second encoder to obtain the features of the second-phase remote sensing image specifically includes: ； ； ； ； ； ； ； ； ； ； ； ； represents the th input low-frequency feature of the second-phase remote sensing image, represents the th input mid-frequency feature of the second-phase remote sensing image, represents the th input high-frequency feature of the second-phase remote sensing image, represents the th low-frequency feature of the second-phase remote sensing image, represents the th mid-frequency feature of the second-phase remote sensing image, represents the th high-frequency feature of the second-phase remote sensing image; The first half of the nth low-frequency feature of the second-phase remote sensing image, The second half of the nth low-frequency feature of the second-phase remote sensing image, The first half of the nth medium-frequency feature of the second-phase remote sensing image, The second half of the nth medium-frequency feature of the second-phase remote sensing image, The first half of the nth high-frequency feature of the second-phase remote sensing image, The second half of the nth high-frequency feature of the second-phase remote sensing image; representing the th low-to-medium frequency feature of the second-phase image, representing the th medium-to-high frequency feature of the second-phase image, representing the second-phase output feature of the th frequency feature extraction module of the second encoder; The value range of is 1 to 3.

5. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 4, characterized in that Performing feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image by using the adaptive gate unit to obtain the filtered frequency features, and further processing the filtered frequency features by using the decoder to obtain the multi-scale dual-phase fusion features, including: ； ； ； ； ； ； ； ， ； ； Among them, represents element-wise subtraction, represents the absolute value operation, represents the Sigmoid function, represents element-wise addition, represents element-wise multiplication; Represents the first-phase output feature of the first encoder's nth frequency feature extraction module, Represents the second-phase output feature of the second encoder's nth frequency feature extraction module, Represents the nth frequency difference feature, Represents the nth low-frequency difference feature, Represents the nth mid-frequency difference feature, Represents the nth high-frequency difference feature, Represents the nth low-frequency difference attention, Represents the nth mid-frequency difference attention, Represents the nth high-frequency difference attention, Represents the nth global inter-band attention, Represents the nth gate attention, Represents the nth first-phase filtered frequency feature, Represents nth second-phase filtered frequency feature, Represents the nth auxiliary prediction; ； ； Among them, represents the first multi-scale fusion feature of the temporal remote sensing image, represents the second multi-scale fusion feature of the temporal remote sensing image, represents concatenating the features along the channel dimension, represents the upsampling operation, represents the third filtered frequency feature of the temporal remote sensing image, represents the second filtered frequency feature of the 6. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 5, characterized in that Generating a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features, and generating the final difference fusion feature based on the multi-scale difference fusion feature map, including: ； ； ； ； Among them, represents the final differential fusion feature, represents the upsampling operation; represents the first multi-scale differential fusion feature, represents the first multi-scale fusion feature of the first-phase remote sensing image, represents the first multi-scale fusion feature of the second-phase remote sensing image, represents the second multi-scale differential fusion feature, represents the second multi-scale fusion feature of the first-phase remote sensing image, represents the second multi-scale fusion feature of the second-phase remote sensing image, represents the third multi-scale differential fusion feature, represents the third filtered frequency feature of the first-phase remote sensing image, represents the third filtered frequency feature of the second-phase remote sensing image.

7. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 6, characterized in that, Based on the final difference fusion feature, generate the change prediction results of the two images, specifically including: processing the final difference fusion feature through a convolutional layer, and then processing the processing result through an activation function, and finally obtaining the change prediction results of the two images.

8. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 1, characterized in that, Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images. The training process of the trained remote sensing image change detection model includes: Construct a data set, which is several pairs of double-phase remote sensing images with known remote sensing image change labels; each pair of double-phase remote sensing images includes: a remote sensing image taken in the first phase and a remote sensing image taken in the second phase of the same geographical area; the first phase is earlier than the second phase. Divide the data set into a training set, a validation set and a test set according to a set ratio. Input the training set into the remote sensing image change detection model to train the model; when the total loss function value of the model stops decreasing, or the number of iterations exceeds the set number of times, stop training. Input the validation set into the trained remote sensing image change detection model to validate the model. Input the test set into the remote sensing image change detection model that has passed the validation to test the model. Use the model that meets the set test conditions as the trained remote sensing image change detection model.

9. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 1, characterized in that, Obtain the first-phase remote sensing image and the second-phase remote sensing image to be detected, including: obtaining them by satellite remote sensing.

10. A remote sensing image change detection system based on prior knowledge and frequency domain learning, characterized in that, Include: An acquisition module, which is configured to: acquire the first-phase remote sensing image and the second-phase remote sensing image to be detected. A detection module, which is configured to: input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images; wherein, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-phase remote sensing image using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; perform feature extraction processing on the second-phase remote sensing image using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image using an adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features using a decoder to obtain the multi-scale double-phase fusion features; generate a multi-scale difference fusion feature map based on the multi-scale double-phase fusion features; generate the final difference fusion feature based on the multi-scale difference fusion feature map; generate the change prediction results of the two images based on the final difference fusion feature.

Images