Remote Sensing Image Change Detection Method and System Based on Prior Knowledge and Frequency Domain Learning

Abstract

The present invention relates to the technical field of remote sensing images, and particularly to a remote sensing image change detection method and system based on prior knowledge and frequency domain learning. The method includes: inputting the first and second phase remote sensing images to be detected into a trained remote sensing image change detection model to obtain the change prediction results of the two images; the model performs feature extraction processing on the first phase remote sensing image to obtain the preprocessed features of the first phase remote sensing image; performs feature extraction processing on the second phase remote sensing image to obtain the preprocessed features of the second phase remote sensing image; filters the preprocessed features of the first phase remote sensing image and the preprocessed features of the second phase remote sensing image to obtain the filtered frequency features; further processes the filtered frequency features to generate the final difference fusion features; and generates the change prediction results of the two images based on the final difference fusion features. The present invention can give an accurate change prediction result.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing images, and particularly to a remote sensing image change detection method and system based on prior knowledge and frequency domain learning. Background Art

[0002] In recent years, with the rapid development of aerospace technology, a large number of high-performance remote sensing satellites have been launched. These satellites are equipped with advanced sensors and imaging devices, capable of observing the Earth's surface with unprecedented accuracy and frequency, which greatly improves the ability to obtain high-resolution, multi-spectral remote sensing images. As a result, many large and diverse remote sensing image datasets have been established. These datasets cover various geographical information of vast regions across the globe, providing a solid data foundation for remote sensing image analysis methods based on deep learning.

[0003] Remote sensing image change detection is an important surface observation task. By using dual-temporal images collected by remote sensing satellites to monitor changes in surface objects, remarkable progress has been made in many practical applications. Specifically, by comparing remote sensing images of different time phases, researchers can accurately monitor urban construction, land use changes, and environmental changes; in the field of natural disaster prevention and control, forest fires, flood-prone areas, and post-earthquake damage can be monitored in real time, providing effective support for emergency rescue and post-disaster reconstruction; in agricultural damage assessment, the growth status of crops and the occurrence of pests and diseases can be monitored to help farmers take targeted measures to reduce economic losses. With the rapid development of deep learning, many deep learning-based methods have been used for remote sensing image change detection. For example, convolutional neural networks, due to their significant advantages in extracting spatial detail features, many change detection methods use them to extract the features of dual-temporal images respectively, and further fuse the obtained dual-temporal features to identify change regions. However, affected by environmental factors such as illumination and atmosphere, there are two objective factors in remote sensing image change detection: some attributes of the same object may be different in different time phases, and some attributes of different objects may be similar. For example, the same building has different colors under different illuminations, and different types of crops may all be green, which may lead to false changes or missed detections of changes, posing challenges to deep learning-based methods. Summary of the Invention

[0004] To solve the deficiencies of the prior art, the present invention provides a remote sensing image change detection method and system based on prior knowledge and frequency domain learning;

[0005] On the one hand, a remote sensing image change detection method based on prior knowledge and frequency domain learning is provided, including:

[0006] Obtain a first-temporal remote sensing image and a second-temporal remote sensing image to be detected;

[0007] Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images. Among them, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-phase remote sensing image using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; perform feature extraction processing on the second-phase remote sensing image using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image using an adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features using a decoder to obtain multi-scale dual-phase fusion features; generate a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features; generate the final difference fusion feature based on the multi-scale difference fusion feature map; generate the change prediction results of the two images based on the final difference fusion feature.

[0008] On the other hand, a remote sensing image change detection system based on prior knowledge and frequency domain learning is provided, including:

[0009] An acquisition module configured to: acquire the first-phase remote sensing image and the second-phase remote sensing image to be detected;

[0010] A detection module configured to: input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images. Among them, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-phase remote sensing image using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; perform feature extraction processing on the second-phase remote sensing image using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image using an adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features using a decoder to obtain multi-scale dual-phase fusion features; generate a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features; generate the final difference fusion feature based on the multi-scale difference fusion feature map; generate the change prediction results of the two images based on the final difference fusion feature.

[0011] The above technical solution has the following advantages or beneficial effects:

[0012] First, the present invention analyzes and summarizes the method for humans to accurately judge changing objects, that is, to judge changing objects based on the comparison results of discriminative attributes (color, boundary, texture) between objects, rather than the overall comparison method of spatial features that is currently popular in deep learning networks. For example, humans can identify the same building by using the discriminative boundary attribute instead of the color attribute and identify different crops based on the differences in texture attributes. Guided by this prior knowledge of attribute comparison, the present invention proposes to use a sliding discrete cosine transform to decompose an RGB image into frequency features of different bands, and these frequency features of different bands can be considered to represent different attributes. For example, the low-frequency band features are generally related to the color attribute, while the high-frequency band features are related to the texture attribute. Subsequently, these frequency features are fed into an encoder based on a convolutional neural network to further extract and enhance these features, and an adaptive gate unit is used to filter out interfering band (attribute) features and retain discriminative band features. Finally, an accurate change prediction is given based on these discriminative band features, so as to effectively cope with the above objective factors. Compared with the attribute comparison method proposed by the present invention, the currently popular overall comparison method of spatial features is restricted by the inseparability of attributes in the spatial domain and is inconsistent with the prior knowledge of humans for change detection, thus resulting in more false changes or missed detected changes.

[0013] Different from the currently popular deep learning change detection method based on the overall comparison of spatial features, the present invention is inspired by human prior knowledge and embeds an attribute comparison method based on frequency learning into the deep learning network. Specifically, the present invention proposes to use a sliding discrete cosine transform to decompose a pair of multi-temporal RGB images into frequency features of different bands, and these frequency features of different bands represent different attributes. At the same time, the present invention reorganizes, sorts, and balances the energy of these features for subsequent feature extraction processes and enhancing network robustness. Subsequently, these frequency features are fed into an encoder based on a convolutional neural network, and an adaptive gate unit is used to filter out interfering band features and retain discriminative band features. Finally, these discriminative band features are progressively decoded in the decoder and an accurate change prediction is given. The present invention effectively reduces the situation of false changes and missed detected change regions, and finally obtains a remote sensing change detection model with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings forming a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0015] Figure 1 It is a flowchart of the method for the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0017] Embodiment 1

[0018] This embodiment provides a remote sensing image change detection method based on prior knowledge and frequency domain learning;

[0019] As Figure 1 shown, the remote sensing image change detection method based on prior knowledge and frequency domain learning includes:

[0020] S101: Obtain the first-phase remote sensing image and the second-phase remote sensing image to be detected;

[0021] S102: Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images; wherein, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-phase remote sensing image by using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; perform feature extraction processing on the second-phase remote sensing image by using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image by using an adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features by using a decoder to obtain the multi-scale dual-phase fusion features; generate a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features; generate the final difference fusion feature based on the multi-scale difference fusion feature map; and generate the change prediction results of the two images based on the final difference fusion feature.

[0022] Furthermore, the S101: Obtain the first-phase remote sensing image and the second-phase remote sensing image to be detected, includes: Obtaining by satellite remote sensing method.

[0023] Furthermore, for the S102: Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images, the training process of the trained remote sensing image change detection model includes:

[0024] Construct a data set, the data set being several pairs of dual-phase remote sensing images with known remote sensing image change labels; each pair of dual-phase remote sensing images includes: the remote sensing image taken in the first phase and the remote sensing image taken in the second phase of the same geographical area; the first phase is earlier than the second phase;

[0025] Divide the dataset into a training set, a validation set, and a test set according to a set ratio;

[0026] Input the training set into the remote sensing image change detection model to train the model; stop training when the total loss function value of the model stops decreasing or the number of iterations exceeds the set number of times;

[0027] Input the validation set into the trained remote sensing image change detection model to validate the model;

[0028] Input the test set into the remote sensing image change detection model that has passed the validation to test the model;

[0029] Use the model that meets the set test conditions as the trained remote sensing image change detection model.

[0030] It should be understood that the original remote sensing image dataset is divided into a training set, a validation set, and a test set according to a ratio of 6:2:2. The model is trained on the training set, and the model that achieves the best results on the validation set is used as the final converged model, and the model effect is evaluated on the test set.

[0031] It should be understood that high-resolution remote sensing images have a large size. Limited by computing resources, the original dual-temporal images need to be cropped into image patches and then fed into the model, where the dual-temporal image patches , correspond to each other in pairs. Subsequently, the present invention performs data augmentation. First, calculate the mean and standard deviation of the RGB three channels of all image patches, and standardize the pixel values of the image patches. Secondly, perform the same horizontal and vertical random flips on the corresponding pair of dual-temporal image patches and data augmentation operations such as swapping the time order of this pair of dual-temporal image patches.

[0032] The present invention believes that the positions with values greater than 0.5 in the final prediction map belong to the changed area, and the positions less than 0.5 belong to the unchanged area. At the same time, the present invention generates a boundary label using the Canny operator based on the image label, and uses this boundary label to supervise the prediction map to further ensure the boundary accuracy of the prediction map. By comparing the prediction result with the image label and the boundary label, using the binary cross-entropy function as the loss function, calculating the loss value, and performing gradient backpropagation to update the model parameters.

[0033] The expression of the cross-entropy loss function of the trained remote sensing image change detection model is as follows:

[0034] ;

[0035] Among them, is the number of pixels of each image patch; is the number of categories, with a value of 2 here, represents the th pixel point in the image block, and its value range is [1, N]; represents the th category, and its value range is [1, M]; is the sign function. If the true category of the pixel point is equal to then take 1, otherwise take 0; is the probability value that the pixel point belongs to the category .

[0036] The present invention is trained within the framework of Pytorch using an NVIDIA GeForce RTX 3090 GPU. The stochastic gradient descent method is used and the initial learning rate is set to 0.01. The learning rate will decay linearly until the last training epoch. The total number of training epochs is 300, and the batch size is set to 32. The present invention performs validation during the training process and finally selects the model weights with the best performance on the validation set.

[0037] Further, in S102: The first-phase remote sensing image and the second-phase remote sensing image to be detected are input into the trained remote sensing image change detection model to obtain the change prediction results of the two images. The trained remote sensing image change detection model includes:

[0038] A first encoder, a second encoder, an adaptive gate unit, and a decoder;

[0039] The input end of the first encoder is used to input the first-phase remote sensing image to be detected;

[0040] The input end of the second encoder is used to input the second-phase remote sensing image to be detected;

[0041] The output ends of the first encoder and the second encoder are both connected to the input end of the adaptive gate unit; the output end of the adaptive gate unit is connected to the input end of the decoder;

[0042] The output end of the decoder outputs the change prediction results of the two images.

[0043] Further, the first encoder includes: a first sliding discrete cosine transform module, a first batch normalization module, a first frequency feature extraction module of the first encoder, a second frequency feature extraction module of the first encoder, and a third frequency feature extraction module of the first encoder, which are connected in sequence.

[0044] Further, the second encoder includes: a second sliding discrete cosine transform module, a second batch normalization module, a first frequency feature extraction module of the second encoder, a second frequency feature extraction module of the second encoder, and a third frequency feature extraction module of the second encoder, which are connected in sequence.

[0045] Further, for S102: using a first encoder to perform feature extraction processing on the first-phase remote sensing image to obtain the preprocessed features of the first-phase remote sensing image, specifically including:

[0046] ;

[0047] Among them, represents converting the image from space to space, represents dividing the image in a sliding window manner, represents discrete cosine transform, represents collecting and reorganizing the same-band frequency features of different windows according to the spatial position of the original image, represents channel sorting operation; represents the preprocessed features of the first-phase remote sensing image.

[0048] It should be understood that the obtained features have the following characteristics: the positional relationship is consistent with the pixel positional relationship of the original image, but the size is reduced. Each channel of the obtained features corresponds to the features of different bands. These channels are sorted from low to high according to their corresponding bands. The subsequent division criterion is based on the sequence number of the channels. Exemplarily, if the feature has 147 channels, then the 1st - 49th channels are low-frequency features, the 50th - 98th channels are medium-frequency features, and the 99th - 147th channels are high-frequency features.

[0049] ;

[0050] Among them, represents batch normalization; represents the preprocessed features of the first-phase remote sensing image with energy balance;

[0051] ;

[0052] ;

[0053] ;

[0054] ;

[0055] ;

[0056] ;

[0057] ;

[0058] ;

[0059] ;

[0060] ;

[0061] Among them, represents a two-dimensional convolution with a convolution kernel of and a stride of 2, represents splitting the feature along the channel dimension, represents concatenating the features along the channel;

[0062] represents the th input low-frequency feature of the first-phase remote sensing image, represents the th input mid-frequency feature of the first-phase remote sensing image, represents the th input high-frequency feature of the first-phase remote sensing image, represents the th low-frequency feature of the first-phase remote sensing image, represents the th mid-frequency feature of the first-phase remote sensing image, represents the th high-frequency feature of the first-phase remote sensing image;

[0063] represents the first half of the th low-frequency feature of the first-phase remote sensing image, represents the second half of the th low-frequency feature of the first-phase remote sensing image, represents the first half of the th mid-frequency feature of the first-phase remote sensing image, represents the second half of the th mid-frequency feature of the first-phase remote sensing image, represents the first half of the th high-frequency feature of the first-phase remote sensing image, represents the second half of the th high-frequency feature of the first-phase remote sensing image;

[0064] represents the a low - intermediate frequency feature, representing the th intermediate - high frequency feature of the first - phase remote sensing image, representing the first - phase output feature of the th frequency feature extraction module of the first encoder; The value range of is 1 to 3.

[0065] It should be understood that this method not only reduces the mixing degree of different bands but also avoids the closure of band features, enhancing the robustness and performance of the network.

[0066] It should be understood that first, a method called sliding discrete cosine transform is proposed to extract the dual - phase frequency features with a size of and and , and this method first converts the RGB image to the YCbCr image because the YCbCr space decomposes luminance and chrominance, which is more in line with the perception characteristics of the human visual system.

[0067] Subsequently, a sliding - window - based discrete cosine transform is performed on each channel of the YCbCr image, where the size and stride of the sliding window are customizable. The size of the window is proportional to the number of bands of the output frequency features, and the stride of the window determines the spatial size of the output frequency features. Subsequently, the present invention collects the same - band frequency features located in different windows and reorganizes them into the same channel according to the , position relationship of the original image. In this way, the present invention converts the RGB image into frequency features, and different channels of this frequency feature represent different band features and the spatial position is consistent with the original image. Next, the present invention sorts this frequency feature in the order from low frequency to high frequency.

[0068] Compared with the traditional discrete cosine transform applied to image blocks, the sliding discrete cosine transform can simultaneously extract the frequency features of rich bands and customize the output spatial size. In addition, it solves the problem of spatial discontinuity caused by the hard - segmentation image - block strategy. In the present invention, the size of the sliding window is set to 7 7, and the stride is set to 4. However, the frequency features extracted using the discrete cosine transform have more energy in the low - frequency band, which results in a greater impact of the low - frequency band features on the change prediction results. To solve this problem, the present invention introduces batch normalization to balance the energy and influence of different - band (channel) features and enhance the robustness of the entire network.

[0069] Therefore, the present invention is fair to the frequency features in different bands rather than being biased towards the low - frequency band features. To further extract the deep - frequency features, the present invention designs a frequency feature extraction module to generate multi - scale frequency features , , , , , , with dimensions:

[0070] 64 64 147, 64 64 147, 32 32 294, 32 32 294, 16 16 588, 16 16 588.

[0071] Taking the th temporal remote sensing image in the network as an example, ( ), its input frequency feature , ), is first divided into three parts: low-frequency feature , intermediate-frequency feature , and high-frequency feature . Subsequently, the present invention further extracts and enhances them through different convolutional layers respectively. This band separation and enhancement strategy is to avoid the mixing between the discriminative band features and the interfering band features after decoupling as much as possible, so as to better extract the frequency features of the discriminative bands. .

[0072] To solve the problems brought by hard band separation, the present invention connects the low-frequency feature of the latter half channels with the intermediate-frequency feature of the former half channels, and calls it the low-intermediate frequency feature ; connects the intermediate-frequency feature of the latter half channels with the high-frequency feature of the former half channels, and calls it the intermediate-high frequency feature . Then, the low-frequency feature , , of the former half channels and the high-frequency feature of the latter half channels are respectively sent into the convolutional layer for enhancement. Finally, they are concatenated together in the original channel order as the output .

[0073] Further, the step S102: using a second encoder to perform feature extraction processing on the second-phase remote sensing image to obtain the features of the second-phase remote sensing image, specifically including:

[0074] ;

[0075] ;

[0076] ;

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] ;

[0086] represents the th input low-frequency feature of the second-phase remote sensing image, represents the th input medium-frequency feature of the second-phase remote sensing image, represents the th input high-frequency feature of the second-phase remote sensing image, represents the th low-frequency feature of the second-phase remote sensing image, represents the th medium-frequency feature of the second-phase remote sensing image, represents the th high-frequency feature of the second-phase remote sensing image;

[0087] represents the first half of the th low-frequency feature of the second-phase remote sensing image, represents the second half of the th low-frequency feature of the second-phase remote sensing image, represents the The first half of the intermediate frequency features, represents the second half of the intermediate frequency features of the second-phase remote sensing image, the first half of the high-frequency features of the second-phase remote sensing image, represents the second half of the high-frequency features of the second-phase remote sensing image;

[0088] represents the low-intermediate frequency features of the second-phase image, represents the mid-high frequency features of the second-phase image, represents the second-phase output features of the frequency feature extraction module of the second encoder; The value range of [[ID=]] is 1 to 3.

[0089] Furthermore, in S102: an adaptive gate unit is used to perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image to obtain filtered frequency features, and a decoder is used to further process the filtered frequency features to obtain multi-scale dual-phase fusion features, including:

[0090] ;

[0091] ;

[0092] ;

[0093] ;

[0094] ;

[0095] ;

[0096] ;

[0097] , ;

[0098] ;

[0099] wherein, represents element-wise subtraction, represents the absolute value operation, represents the Sigmoid function, represents element-wise addition, represents element-wise multiplication.

[0100] Represents the first-phase output feature of the first frequency feature extraction module of the first encoder, Represents the second-phase output feature of the first frequency feature extraction module of the second encoder, Represents the first frequency difference feature, Represents the first low-frequency difference feature, Represents the first mid-frequency difference feature, Represents the first high-frequency difference feature, Represents the first low-frequency difference attention, Represents the first mid-frequency difference attention, Represents the first high-frequency difference attention, Represents the first global inter-band attention, Represents the first gate attention, Represents the first first-phase filtered frequency feature, Represents first second-phase filtered frequency feature, Represents the first auxiliary prediction.

[0101] ;

[0102] ;

[0103] Among them, Represents the first multi-scale fusion feature of the first-phase remote sensing image, Represents the second multi-scale fusion feature of the first-phase remote sensing image, Represents concatenating features along the channel dimension, Represents the upsampling operation, Represents the first third-phase filtered frequency feature of the remote sensing image, Represents the first second-phase filtered frequency feature of the remote sensing image.

[0104] To retain the frequency features of discriminative bands and filter out the frequency features of interfering bands, the present invention designs an adaptive gate unit. Taking the i-th adaptive gate unit in the network as an example (i = 1, 2, 3), the present invention first makes the input dual-temporal frequency features , perform pixel-wise subtraction and absolute value operation to obtain the frequency difference features . Next, the present invention divides the frequency difference features into three parts according to the bands: low-frequency difference features , intermediate-frequency difference features and high-frequency difference features , which are respectively used to generate three local intra-band attentions , , . At the same time, from the global perspective of the bands, the present invention uses to obtain a global inter-band attention . Subsequently, the present invention connects along the channels , , , and the connected features are added to and sent into function to obtain the gate attention , whose size is the same as that of the input, and the value is limited to [0, 1]. Subsequently, the present invention multiplies and , element-wise to retain the frequency features of discriminative bands in the dual-temporal frequency features and filter out the frequency features of interfering bands. For example, when the value of a certain point in the feature corresponding to the gate attention is 1, it means to retain this point, and when it is 0, this point is filtered out. In addition, the present invention uses the change label to supervise the auxiliary prediction to further ensure the accuracy of the attention, which is generated by the difference features obtained by element-wise multiplying and .

[0105] Finally, the present invention obtains the dual-temporal features processed by these gate units , , , , , and uses them for subsequent decoding. The sizes of these features are respectively:

[0106] 64 64 147, 64 64 147,32 32 294,32 32 294,16 16 588,16 16 588。

[0107] During the decoding process, the present invention selects a dense decoding method. Specifically, the present invention first uses , , , , , and progressively generates multi-scale dual-temporal fusion features through upsampling operations, channel concatenation, and convolutional layers , 。

[0108] Furthermore, the S102: generating a multi-scale differential fusion feature map based on the multi-scale dual-temporal fusion feature, and generating a final differential fusion feature based on the multi-scale differential fusion feature map, includes:

[0109] ;

[0110] ;

[0111] ;

[0112]

[0113] Wherein, represents the final differential fusion feature, represents the upsampling operation. represents the first multi-scale differential fusion feature, represents the first multi-scale fusion feature of the first-temporal remote sensing image, represents the first multi-scale fusion feature of the second-temporal remote sensing image, represents the second multi-scale differential fusion feature, represents the second multi-scale fusion feature of the first-temporal remote sensing image, represents the second multi-scale fusion feature of the second-temporal remote sensing image, represents the third multi-scale differential fusion feature, represents the third filtered frequency feature of the first-temporal remote sensing image, represents the 3rd filtered frequency feature of the second-temporal remote sensing image.

[0114] The multi-scale dual-temporal fusion feature map is used to generate the multi-scale differential fusion feature map , , , with sizes of 64 64 128, 32 32 128, 16 16 588. A final difference fusion feature DF with a size of 64 64 128 is obtained and used to generate the final prediction.

[0115] Further, in S102: based on the final difference fusion feature, generate the change prediction results of the two images, specifically including: processing the final difference fusion feature based on the convolutional layer, and then processing the processing result through the activation function, and finally obtaining the change prediction results of the two images.

[0116] Embodiment 2

[0117] This embodiment provides a remote sensing image change detection system based on prior knowledge and frequency domain learning;

[0118] A remote sensing image change detection system based on prior knowledge and frequency domain learning includes:

[0119] An acquisition module configured to: acquire a first-phase remote sensing image and a second-phase remote sensing image to be detected;

[0120] A detection module configured to: input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images; wherein, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-phase remote sensing image using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; perform feature extraction processing on the second-phase remote sensing image using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image using an adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features using a decoder to obtain the multi-scale dual-phase fusion features; generate a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features; generate a final difference fusion feature based on the multi-scale difference fusion feature map; generate the change prediction results of the two images based on the final difference fusion feature.

[0121] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A remote sensing image change detection method based on prior knowledge and frequency domain learning, characterized in that, Including: Obtain the first-phase remote sensing image and the second-phase remote sensing image to be detected; Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images; wherein, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-phase remote sensing image by using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; perform feature extraction processing on the second-phase remote sensing image by using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image by using an adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features by using a decoder to obtain the multi-scale two-phase fusion features; generate a multi-scale difference fusion feature map based on the multi-scale two-phase fusion features; generate the final difference fusion feature based on the multi-scale difference fusion feature map; generate the change prediction results of the two images based on the final difference fusion feature.

2. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 1, characterized in that Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images. The trained remote sensing image change detection model includes: A first encoder, a second encoder, an adaptive gate unit and a decoder; The input end of the first encoder is used to input the first-phase remote sensing image to be detected; The input end of the second encoder is used to input the second-phase remote sensing image to be detected; The output end of the first encoder and the output end of the second encoder are both connected to the input end of the adaptive gate unit; the output end of the adaptive gate unit is connected to the input end of the decoder; The output end of the decoder outputs the change prediction results of the two images; The first encoder includes: a first sliding discrete cosine transform module, a first batch normalization module, a first frequency feature extraction module of the first encoder, a second frequency feature extraction module of the first encoder and a third frequency feature extraction module of the first encoder connected in sequence; The second encoder includes: a second sliding discrete cosine transform module, a second batch normalization module, a first frequency feature extraction module of the second encoder, a second frequency feature extraction module of the second encoder and a third frequency feature extraction module of the second encoder connected in sequence.

3. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 2, characterized in that, Performing feature extraction processing on the first-phase remote sensing image by using a first encoder to obtain the preprocessed features of the first-phase remote sensing image specifically includes: ； Among them, represents converting the image from space to space, represents splitting the image in a sliding window manner, represents discrete cosine transform, represents collecting and reorganizing the same band frequency features of different windows according to the spatial position of the original image, represents channel sorting operation; represents the preprocessing features of the first-phase remote sensing image; ； Among them, represents batch normalization; represents the preprocessing feature of the first-phase remote sensing image of energy balancing; ； ； ； ； ； ； ； ； ； ； Among them, represents a two-dimensional convolution with a convolution kernel of and a stride of 2, represents splitting the feature along the channel dimension, represents concatenating the features along the channels; representing the th input low-frequency feature of the first-phase remote sensing image, representing the th input medium-frequency feature of the first-phase remote sensing image, representing the th input high-frequency feature of the first-phase remote sensing image, representing the th low-frequency feature of the first-phase remote sensing image, representing the th medium-frequency feature of the first-phase remote sensing image, representing the th high-frequency feature of the first-phase remote sensing image; The first half of the nth low-frequency feature of the first-phase remote sensing image, The second half of the nth low-frequency feature of the first-phase remote sensing image, The first half of the nth medium-frequency feature of the first-phase remote sensing image, The second half of the nth medium-frequency feature of the first-phase remote sensing image, The first half of the nth high-frequency feature of the first-phase remote sensing image, The second half of the nth high-frequency feature of the first-phase remote sensing image; representing the th low-to-medium frequency feature of the first-phase remote sensing image, representing the th medium-to-high frequency feature of the first-phase remote sensing image, representing the first-phase output feature of the th frequency feature extraction module of the first encoder; The value range of is 1 to 3.

4. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 3, characterized in that Performing feature extraction processing on the second-phase remote sensing image by using a second encoder to obtain the features of the second-phase remote sensing image specifically includes: ； ； ； ； ； ； ； ； ； ； ； ； representing the th input low-frequency feature of the second-phase remote sensing image, representing the th input mid-frequency feature of the second-phase remote sensing image, representing the th input high-frequency feature of the second-phase remote sensing image, representing the th low-frequency feature of the second-phase remote sensing image, representing the th mid-frequency feature of the second-phase remote sensing image, representing the th high-frequency feature of the second-phase remote sensing image; The first half of the nth low-frequency feature of the second-phase remote sensing image, The second half of the nth low-frequency feature of the second-phase remote sensing image, The first half of the nth medium-frequency feature of the second-phase remote sensing image, The second half of the nth medium-frequency feature of the second-phase remote sensing image, The first half of the nth high-frequency feature of the second-phase remote sensing image, The second half of the nth high-frequency feature of the second-phase remote sensing image; indicating the th low-to-medium frequency feature of the second-phase image, indicating the th medium-to-high frequency feature of the second-phase image, indicating the second-phase output feature of the th frequency feature extraction module of the second encoder; The value range of is 1 to 3.

5. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 4, characterized in that Performing feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image by using an adaptive gate unit to obtain the filtered frequency features, and further processing the filtered frequency features by using a decoder to obtain the multi-scale two-phase fusion features, including: ； ； ； ； ； ； ； ， ； ； Among them, represents element-wise subtraction, represents the absolute value operation, represents the Sigmoid function, represents element-wise addition, represents element-wise multiplication; Represents the first-phase output feature of the first encoder's first frequency feature extraction module, Represents the second-phase output feature of the second encoder's second frequency feature extraction module, Represents the nth frequency difference feature, Represents the nth low-frequency difference feature, Represents the nth mid-frequency difference feature, Represents the nth high-frequency difference feature, Represents the nth low-frequency difference attention, Represents the nth mid-frequency difference attention, Represents the nth high-frequency difference attention, Represents the nth global inter-band attention, Represents the nth gate attention, Represents the nth first-phase filtered frequency feature, Represents nth second-phase filtered frequency feature, Represents the nth auxiliary prediction; ； ； Among them, represents the first multi-scale fusion feature of the temporal remote sensing image, represents the second multi-scale fusion feature of the temporal remote sensing image, represents concatenating the features along the channel dimension, represents the upsampling operation, represents the third filtered frequency feature of the temporal remote sensing image, represents the second filtered frequency feature of the 6. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 5, characterized in that, Generating a multi-scale difference fusion feature map based on the multi-scale two-phase fusion features, and generating the final difference fusion feature based on the multi-scale difference fusion feature map, including: ； ； ； ； Among them, represents the final differential fusion feature, represents the upsampling operation; represents the first multi-scale differential fusion feature, represents the first multi-scale fusion feature of the first-phase remote sensing image, represents the first multi-scale fusion feature of the second-phase remote sensing image, represents the second multi-scale differential fusion feature, represents the second multi-scale fusion feature of the first-phase remote sensing image, represents the second multi-scale fusion feature of the second-phase remote sensing image, represents the third multi-scale differential fusion feature, represents the third filtered frequency feature of the first-phase remote sensing image, represents the third filtered frequency feature of the second-phase remote sensing image.

7. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 6, characterized in that, Based on the final difference fusion feature, generate the change prediction results of the two images, specifically including: processing the final difference fusion feature through a convolutional layer, and then processing the processing result through an activation function, and finally obtaining the change prediction results of the two images.

8. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 1, characterized in that, Input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images. The training process of the trained remote sensing image change detection model includes: Construct a data set, which is several pairs of dual-phase remote sensing images with known remote sensing image change labels; each pair of dual-phase remote sensing images includes: a remote sensing image taken at the first phase and a remote sensing image taken at the second phase of the same geographical area; the first phase is earlier than the second phase. Divide the data set into a training set, a validation set and a test set according to a set ratio. Input the training set into the remote sensing image change detection model to train the model; when the total loss function value of the model stops decreasing, or the number of iterations exceeds the set number of times, stop training. Input the validation set into the trained remote sensing image change detection model to validate the model. Input the test set into the remote sensing image change detection model that has passed the validation to test the model. Use the model that meets the set test conditions as the trained remote sensing image change detection model.

9. The remote sensing image change detection method based on prior knowledge and frequency domain learning according to claim 1, wherein Obtain the first-phase remote sensing image and the second-phase remote sensing image to be detected, including: obtaining them by satellite remote sensing.

10. A remote sensing image change detection system based on prior knowledge and frequency domain learning, characterized in that, Include: An acquisition module configured to: acquire the first-phase remote sensing image and the second-phase remote sensing image to be detected. A detection module configured to: input the first-phase remote sensing image and the second-phase remote sensing image to be detected into the trained remote sensing image change detection model to obtain the change prediction results of the two images; wherein, the trained remote sensing image change detection model is used to: perform feature extraction processing on the first-phase remote sensing image using a first encoder to obtain the preprocessed features of the first-phase remote sensing image; perform feature extraction processing on the second-phase remote sensing image using a second encoder to obtain the preprocessed features of the second-phase remote sensing image; perform feature filtering on the preprocessed features of the first-phase remote sensing image and the preprocessed features of the second-phase remote sensing image using an adaptive gate unit to obtain the filtered frequency features; further process the filtered frequency features using a decoder to obtain the multi-scale dual-phase fusion features; generate a multi-scale difference fusion feature map based on the multi-scale dual-phase fusion features; generate the final difference fusion feature based on the multi-scale difference fusion feature map; generate the change prediction results of the two images based on the final difference fusion feature.

Images