A model and method for finding changed areas in remote sensing images

Abstract

The present invention relates to the field of satellite remote sensing image technology, and in particular to a model and method for finding changed areas in remote sensing images. The model and method propose a receptive field and adaptive convolution kernel size mechanism for a small-size object feature extraction module guided by spectral information, and collaborate the mechanism with a twin structure change detection network to extract input image features. The "space-channel-frequency domain" triple attention fusion module enables the features in the twin structure change detection network to automatically learn and assign weights to changed and unchanged areas after being enhanced by frequency domain information. The frequency domain feature fusion module integrates low-resolution images into high-resolution images through frequency domain information transformation to enhance image details and quality. It fuses adjacent size features layer by layer, thereby preserving the details of images of different resolutions while improving the resolution of the final output. Finally, the change area prediction module outputs a prediction map that conforms to the size and number of channels after convolution processing.

Description

Technical Field

[0001] The invention belongs to the technical field of satellite remote sensing images, and in particular relates to a model and method for searching for change areas in remote sensing images. Background Art

[0002] Finding areas of change in remote sensing images is an important technology for observing dynamic changes in the earth's surface. It uses advanced platforms such as satellites and drones to obtain high-precision images of the earth's surface at different times for comparison. The coverage is wide and can provide effective decision-making support for tasks such as urban planning, agricultural and forestry surveys, and disaster assessments.

[0003] Satellite remote sensing imagery features wide coverage, multispectral imaging, and rich information. While high-resolution satellites currently offer sub-meter resolution, the representation of small-scale search objects is limited due to their small proportion in remote sensing images and the small number of valid features they contain. This leads to insufficient representation capabilities for these small-scale search objects. When extracting image features, these features become blurred after multiple downsampling operations, making them difficult to identify and extract. Depending on factors such as the season and imaging perspective of the satellite image, deep learning algorithms remain inefficient in finding regions of change in remote sensing images. This can lead to unstable feature extraction, increased false positives, and missed detections, impacting the accuracy and reliability of the task.

[0004] In their paper "Fully Convolutional Siamese Networks for Change Detection" presented at the 2018 25th IEEE International Conference on Image Processing (ICIP), Daudt et al. applied end-to-end fully convolutional networks to change region detection in remote sensing imagery and proposed three innovative network architectures. However, these CNN-based change region detection methods are still limited by the inherent constraints of convolutional operations used to model contextual information in remote sensing images. In their paper "Remote Sensing Image Change Detection With Transformers" published in the journal IEEE Transactions on Geoscience and Remote Sensing, Chen et al. introduced a bitemporal Siamese network (BIT), combining a CNN with a Transformer to form a new paradigm for the change region detection task. This framework combines the advantages of CNN's efficient feature extraction with the Transformer's effective capture of spatiotemporal context. However, BIT can overlook high-resolution shallow-level details during processing and may also lose deep, fine features in small-scale change regions during its upsampling phase. Summary of the Invention

[0005] In view of this, the present invention aims to provide a model and method for finding changed areas in remote sensing images to solve the problem in the prior art that the features of search objects with small change sizes are fuzzy, difficult to identify and extract, and affect the accuracy and reliability of the task.

[0006] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0007] A model for finding changed regions in remote sensing images, comprising: a spectrum information-guided small-size object feature extraction module, a triple-attention fusion module, a frequency domain feature fusion module, and a changed region prediction module;

[0008] The spectral information-guided small-size object feature extraction module is connected by a spectral information-guided module and a SwinTransformer module residual structure. The input image is converted from the spatial domain to the frequency domain through a fast Fourier transform, the spectrum of the input image is decomposed into frequency components contributed by different spatial frequencies, and the low-frequency and high-frequency regions of the input image are distinguished through spectral analysis. The appropriate convolution kernel size and receptive field are dynamically selected according to different frequency regions. The spectral information-guided small-size object feature extraction module enhances the ability to capture the edge, texture and structural features of the image through the SwinTransformer module to adapt to different frequency regions of the multi-scale input image and output feature maps of multiple scales.

[0009] The three-attention fusion module acts on the feature maps of multiple scales, gradually enhancing the features of multiple scales, so that the model pays attention to the channels in the features of multiple scales and gives greater weight to the pixels in the changing area of ​​the feature map;

[0010] The frequency domain feature fusion module performs frequency domain feature fusion on the feature map output by the three attention fusion modules, upsamples the low-resolution features in the feature map to the high-resolution space, and fuses the feature information of the high-resolution and low-resolution feature maps; the fused feature map is used as the low-resolution feature map and iteratively fused with the higher-resolution high-resolution feature map;

[0011] The change region prediction module performs convolution processing on the final high-resolution feature map output by the frequency domain feature fusion module and outputs a final prediction result that meets the predicted number of channels. Furthermore, the three-attention fusion module includes a channel attention module, a spatial attention module, and a frequency domain enhanced attention module.

[0012] Furthermore, the model is a twin structural change detection network.

[0013] Furthermore, the spectrum information guidance module performs shallow extraction on the features of the input image through the maximum pooling module and depthwise separable convolution, and sends the extracted features to the frequency domain feature fusion module for frequency domain feature fusion.

[0014] A method for finding a changed area in a remote sensing image, the method comprising the following steps:

[0015] Obtaining a data set, performing data augmentation on the data set, and feeding the augmented data into a model for training;

[0016] The spectral information-guided small-size object feature extraction module converts the input image from the spatial domain to the frequency domain through fast Fourier transform, decomposes the input image spectrum into frequency components contributed by different spatial frequencies, and distinguishes the low-frequency and high-frequency regions of the input image through spectral analysis. It dynamically selects the appropriate convolution kernel size and receptive field according to different frequency regions and performs convolution calculations. It then adjusts the extraction of frequency components in the image through the learning weights of the SwinTransformer module embedded in the form of a residual structure, so that the model pays more attention to high-frequency information. The output multi-scale feature map is sent to the three-attention fusion module;

[0017] The channel attention module, spatial attention module, and frequency domain enhanced attention module in the three-attention fusion module act on the feature map in sequence, gradually enhancing the feature map so that the model pays attention to important channels in the feature map and gives greater weight to pixels in the changing area of ​​the feature map, thereby improving the accuracy of the feature representation of the feature map and outputting features;

[0018] The frequency domain feature fusion module performs frequency domain feature fusion on the shallow features output by the spectrum information guidance module and the features of the feature map output by the triple attention fusion module, upsamples the low-resolution features to the high-resolution space, and fuses the feature information of the high-resolution and low-resolution feature maps; the fused feature map is used as the low-resolution feature map and iteratively fused with the higher-resolution high-resolution feature map;

[0019] The change area prediction module performs convolution processing on the final high-resolution feature map output by the frequency domain feature fusion module, and outputs a final prediction result that meets the prediction channel number, thereby realizing a method for searching for change areas in remote sensing images based on spectrum information and frequency domain features.

[0020] Furthermore, the model distinguishes the low-frequency and high-frequency areas of the input image through spectrum analysis, including the following steps: calculating the spectrum and frequency intensity based on the frequency components contributed by different spatial frequencies in the input image, and setting a threshold value according to the frequency intensity. f threshold , the frequency is higher than f threshold The area below is considered as high frequency area, and the area below is considered as high frequency area. f threshold The area is the low-frequency area.

[0021] Furthermore, the receptive field is dynamically selected according to different frequency regions by calculating the frequency spectrum of the input feature map and dynamically adjusting the expansion rate, thereby achieving adaptive adjustment of the regional receptive field.

[0022] Furthermore, the steps in which the channel attention module, the spatial attention module and the frequency domain enhanced attention module act on the feature map in sequence are: the feature map first passes through a channel attention module and then through the frequency domain enhanced attention module, and then passes through a spatial attention module and then through the frequency domain enhanced attention module.

[0023] Furthermore, it also includes combining high-resolution image features and low-resolution image features through internally integrated low-pass filters and high-pass filters to improve the image reconstruction quality and detail recovery ability. The specific steps are: first, channel compression is performed on high-resolution image features and low-resolution image features through convolution operations, so that the high-resolution image features and low-resolution image features are compressed to the same number of channels and compressed features are generated; frequency domain encoding is performed on the compressed features to capture low-frequency information and high-frequency information and a frequency domain filter mask is generated through local area similarity calculation and adaptive convolution, and combined with the image reconstruction technology CARAFE, the information of high-resolution image features and low-resolution image features is integrated.

[0024] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0025] The present invention creates a model and method for finding changed areas in remote sensing images by proposing a receptive field and adaptive convolution kernel size mechanism for a small-scale object feature extraction module guided by spectral information, and synergizing this mechanism with the SwinTransformer module of the twin structure change detection network in a residual connection structure to extract input image features to adapt to different frequency regions of multi-scale input images, thereby helping the model to better extract multi-scale deep features of the image. The "spatial-channel-frequency domain" triple attention fusion module enables the twin structure change detection network to automatically learn and assign weights to the changed and unchanged areas of the feature map after frequency domain information enhancement, so that the twin structure change detection network can give greater weight to pixels in the changed area and suppress interference noise in the unchanged area, thereby better capturing change information. The frequency domain feature fusion module integrates the low-resolution image into the high-resolution image by transforming the high-resolution and low-resolution images and frequency domain information to improve image details and quality. It fuses adjacent size features layer by layer, while retaining the details of images of different resolutions, while improving the resolution of the final output. Finally, after convolution processing in the change area prediction module, the prediction result of this algorithm and the binary mask image (white represents the change area and black represents the unchanged area) are output. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0027] Figure 1 Schematic diagram of the twin structure change detection network described in the embodiment of the present invention.

[0028] Figure 2 This is a diagram of the "space-channel-frequency domain" three-attention fusion structure described in the embodiment of the present invention.

[0029] Figure 3 This is a diagram of the multi-scale frequency domain feature fusion structure described in the embodiment of the present invention.

[0030] Figure 4 This is a flow chart of a method for searching for change areas in remote sensing images based on spectrum information and frequency domain features as described in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention. Similar elements in different embodiments use associated similar element numbers. In the following embodiments, many detailed descriptions are intended to enable the present invention to be better understood. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, or methods. In some cases, some operations related to the present invention are not shown or described in the specification. This is to avoid the core part of the present invention being overwhelmed by too much description. For those skilled in the art, it is not necessary to describe these related operations in detail. They can fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0032] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other to form various implementation methods. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various orders in the description and the drawings are only for the purpose of clearly describing a certain embodiment and are not intended to be a required order, unless otherwise specified that a certain order must be followed.

[0033] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0034] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0035] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0036] like Figures 1 to 4 As shown, a model for finding changed areas in remote sensing images is shown. In this embodiment, the model is a twin structural change detection network, including: a small-size object feature extraction module guided by spectral information, a triple attention fusion module, a frequency domain feature fusion module, and a changed area prediction module;

[0037] The dual-branch structure of the twin structural change detection network is better used to simultaneously extract long-range dependent global features in bi-temporal images through weight sharing.

[0038] The spectral information guided small-size object feature extraction module consists of a spectral information guided module and a SwinTransformer module. It converts the input image from the spatial domain to the frequency domain through fast Fourier transform, decomposes the spectrum of the input image into frequency components contributed by different spatial frequencies, and distinguishes the low-frequency and high-frequency areas of the input image through spectral analysis. It dynamically performs convolution operations according to different frequency areas and selects appropriate convolution kernel size and receptive field. The spectral information guided module embeds the SwinTransformer module through the residual structure, thereby enhancing the ability to capture the edge, texture and structural features of the input image, and collaboratively extracts the features of the input image to adapt to different frequency areas of the multi-scale input image, thereby helping the twin structural change detection network to better extract the multi-scale deep features of the input image and output feature maps. In this embodiment, four pairs of spectral information guided small-size object feature extraction modules output feature maps of four scales.

[0039] The SwinTransformer module included in the spectral information-guided small-scale object feature extraction module uses a window-based multi-head self-attention mechanism to model long-range dependencies within local windows and a shifted window-based multi-head self-attention mechanism to enable information interaction between different windows, thereby improving the fusion of global information. The spectral information-guided small-scale object feature extraction module can efficiently express features and better capture the complex structures in the input image. The spectral information-guided small-scale object feature extraction module performs a convolution operation on the input image and then inputs it into the triple attention fusion module.

[0040] The three-attention fusion module includes a channel attention module, a spatial attention module, and a frequency domain enhanced attention module, which act on feature maps of four scales in turn, gradually enhance and output the features in the feature maps, attach importance to the channels in the feature maps, and give greater weight to pixels in the feature map change area; in this embodiment, there are eight three-attention fusion modules.

[0041] The two spectrum information guidance modules perform shallow extraction of the features of the input image through the maximum pooling module and depth-wise separable convolution, and send them to the frequency domain feature fusion module for frequency domain feature fusion.

[0042] The frequency domain feature fusion module performs frequency domain feature fusion on the shallow features output by the spectral information guidance module and the features of the four scale feature maps output by the triple attention fusion module. It processes the high-resolution and low-resolution features of the shallow features and the four scale feature maps, respectively. It then upsamples the low-resolution features to a high-resolution space, fuses the feature information of the high-resolution and low-resolution feature maps, and uses the fused feature map as a low-resolution feature map, which is iteratively fused with the higher-resolution high-resolution feature map. In this embodiment, after four fusions, the final high-resolution feature map is sent to the change region prediction module.

[0043] The change region prediction module performs convolution processing on the features in the final high-resolution feature map and outputs the final prediction result that meets the number of prediction channels.

[0044] A method for finding changed areas in a remote sensing image is based on the above-mentioned model for finding changed areas in a remote sensing image. The method comprises the following steps:

[0045] Step 1: Construct a spectral information-guided small-size object feature extraction module, a triple attention fusion module, a frequency domain feature fusion module, and a change region prediction module in the twin structural change detection network.

[0046] Step 2: Obtain a dataset, perform data augmentation on the dataset, and send the augmented data to the twin structural change detection network for training. The training method uses Adam as the optimizer, with 8 images as a training batch. The initial learning rate of the twin structural change detection network is 1e-4, and a total of 200 epochs are trained. At the 100th epoch, the learning rate decays to 1 / 2 of the initial learning rate.

[0047] Step 3: The small-size object feature extraction module guided by spectrum information converts the input image X from the spatial domain to the frequency domain through fast Fourier transform, and decomposes the spectrum of the input image X into frequency components contributed by different spatial frequencies, as shown in formula (1).

[0048] (1)

[0049] in x and y Represents the coordinates of the input image X in the spatial domain, x ( x , y ) represents the grayscale value or eigenvalue at that location, u and v represents the frequency components in the horizontal and vertical directions, M represents the upper limit of the length of the input image in the X direction, and N represents the upper limit of the length of the input image in the Y direction, and j represents the imaginary part.

[0050] The spectrum Decomposed into multiple frequency components, as shown in formula (2), by frequency intensity | | to indicate the intensity of each frequency.

[0051] (2)

[0052] in, R represents the real part, I Represents the imaginary part.

[0053] The twin structural change detection network uses spectral analysis to distinguish low-frequency and high-frequency regions of the input image X, dynamically selecting the appropriate convolution kernel size and receptive field based on the frequency range. The high-frequency portion contains detailed information of the input image X, while the low-frequency portion contains background and large-scale feature information. In the high-frequency region, the input image X changes rapidly and contains rich details, so a smaller convolution kernel is chosen to better capture these details. In the low-frequency region, the input image X changes more gradually, so a larger convolution kernel helps capture more contextual information. This method enables the convolution operation to flexibly handle image features of different frequencies. After spectral analysis, the high-frequency portion of the input image X is extracted, and the appropriate convolution kernel size is selected based on the frequency intensity, achieving more refined feature extraction.

[0054] In this embodiment, a small convolution kernel is used K small To process high-frequency areas, use a large convolution kernel K large Process low-frequency areas. The size of the convolution kernel is selected based on the frequency intensity, and the frequency intensity determines the scale of the convolution kernel. , as shown in formula (3).

[0055] (3)

[0056] The receptive field refers to the area that the convolution kernel can "see" when it slides on the input image X. In order to enable the twin structural change detection network to obtain different feature learning capabilities in different regions, this method dynamically adjusts the expansion rate of the convolution kernel through the features in the image spectrum information to achieve an adaptive receptive field. In the low-frequency area of ​​the input image X, a larger receptive field can be used because there are fewer high-frequency details; in the high-frequency area of ​​the input image X, a smaller receptive field is used to retain more detail information. By calculating the spectral intensity of the feature map that has undergone convolution operation, the expansion rate is dynamically adjusted to achieve adaptive adjustment of the regional receptive field. The key to this method is to use spectral information to distinguish between low-frequency areas and high-frequency areas. In this embodiment, the spectrum of the input image X is represented as ,After spectrum analysis, we get which represents the frequency intensity of the region. |, according to the frequency intensity| The size of | determines the dilation rate used in different frequency regions d ( f ), as shown in formula (4):

[0057] (4)

[0058] in, is a tuning factor that controls the effect of frequency on the expansion rate; for higher frequency intensities | |, expansion rate d ( f ) is smaller, while for lower frequency intensities | |, expansion rate d ( f ) is large. Therefore, the small-size object feature extraction module guided by spectral information dynamically selects the appropriate convolution kernel size and receptive field corresponding to the expansion rate according to different frequency regions, and outputs the feature map.

[0059] The SwinTransformer module in the spectral information-guided small-scale object feature extraction module is connected to the spectral information-guided module via a residual connection. Through all the mechanisms in step 3, the ability to capture edge, texture, and structural features of the input image X is enhanced. By adaptively adjusting the frequency component extraction of features at different scales through the size of the convolution kernel and the size of the receptive field, the twin structural change detection network can focus on key high-frequency information and improve the accuracy of fine feature extraction. The SwinTransformer module uses a window-based multi-head self-attention mechanism to model long-range dependencies within local windows and a shifted window-based multi-head self-attention mechanism to enable information interaction between different windows and enhance the fusion of global information. The efficient feature expression of the spectral information-guided small-scale object feature extraction module enables this method to better capture complex structures in remote sensing images.

[0060] The spectral information-guided small-size object feature extraction module outputs each extracted scale feature map to the triple attention fusion module for feature enhancement, and also outputs it to the next spectral information-guided small-size object feature extraction module pair in the dual-branch structure of the twin structure change detection network for feature extraction.

[0061] Step 4: The three attention fusion modules in the model include the channel attention module CAM, the spatial attention module SAM, and the frequency domain enhanced attention module FAM. The channel attention module CAM, the spatial attention module SAM, and the frequency domain enhanced attention module FAM act in sequence on the feature map output by the spectral information-guided small-size object feature extraction module. The specific steps are as follows: the feature map first passes through a channel attention module CAM, then through the frequency domain enhanced attention module FAM, then through a spatial attention module SAM, and then through another frequency domain enhanced attention module FAM. The purpose of the channel attention module CAM is to allow the twin structure change detection network to weight the features in the feature map according to the importance of different channels, so that the twin structure change detection network pays more attention to important channels in the features and suppresses useless channels. The purpose of the spatial attention module SAM is to weight the features in the feature map according to the importance of different spatial positions, so that the twin structure change detection network pays more attention to important areas in the feature map, thereby improving the accuracy of feature representation. After frequency domain information enhancement, the three-attention fusion module automatically learns and assigns weights to the changed and unchanged areas of the feature map, enabling the twin structure change detection network to give greater weights to pixels in the changed area and suppress interference noise in the unchanged area, thereby better capturing change information, improving the accuracy of feature map representation and outputting features.

[0062] In this embodiment, the frequency domain enhanced attention module FAM refers to the design of the twin structure change detection network, such as Figure 2 As shown, the 2D Fourier transform is first applied to the input feature map f in Mapping to obtain feature maps f in The frequency representation of the feature map is obtained by filtering the real part of the image and then passing it through a 1×1 convolution + GeLU activation layer branch and a 3×3 convolution + BN layer to capture the feature map. f in The local frequency information is extracted and the high-frequency semantic response is enhanced to obtain the final frequency domain enhanced feature representation.

[0063] like Figure 2 As shown, the input feature map f in The size of is C×H×W. First, the channel attention module CAM takes the input feature map f in Perform channel refinement. The refinement process is to compress the input feature map through average pooling and maximum pooling. f in Get two tensors, input the two tensors into the multi-layer neural network MLP layer and combine the outputs. The combined tensor is passed through the Sigmoid function in the channel attention module CAM to obtain the channel weight vectorf c , channel weight vector f c The input feature map f in The weighting is performed to achieve the purpose of enhancing the change area to find the main channel and suppressing the secondary channel. Then, the feature map refined by the channel attention module CAM is subjected to frequency domain attention enhancement by the frequency domain enhancement attention module FAM, and then the spatial attention module SAM refines the feature map after attention enhancement in the spatial dimension; the refinement process is: 3×3 convolution, average pooling and maximum pooling operations are performed on the feature map after attention enhancement input by the spatial attention module SAM to obtain two tensors and splice them, and 7×7 convolution is used to capture the spatial information of the spliced ​​tensor, and the spatial information is converted into a higher-level spatial information feature representation. Finally, the higher-level spatial information feature obtains the spatial weight vector through the Sigmoid function in the spatial attention module SAM. f s , the spatial weight vector f s The feature map after frequency domain attention enhancement and 3*3 convolution by the frequency domain enhanced attention module FAM is multiplied element by element and multiple feature maps are output to achieve the purpose of emphasizing the changing area of ​​the feature map, suppressing the unchanged area, and improving the feature extraction effect.

[0064] Step 5: The frequency domain feature fusion module is used to enhance the details and quality of the image, such as Figure 3 As shown in the figure, the frequency domain feature fusion module performs frequency domain feature fusion on the features of multiple feature maps output by the three attention fusion module and the shallow features output by the spectrum information guidance module. After the low-pass filter and high-pass filter integrated inside the frequency domain feature fusion module, the high-resolution features and low-resolution features of multiple feature maps are combined to improve the image reconstruction quality and detail recovery ability.

[0065] The specific steps are as follows: First, the frequency domain feature fusion module performs channel compression on the high-resolution features and low-resolution features of multiple feature maps through convolution operations, so that the high-resolution features and low-resolution features of the feature maps are compressed to the same number of channels and compressed features are generated. The frequency domain feature fusion module performs frequency domain encoding on the compressed features to capture low-frequency information and high-frequency information, and generates a frequency domain filter mask through local area similarity calculation and adaptive convolution. Through the application of adaptive convolution and frequency domain filter masks, combined with the image reconstruction technology CARAFE, the frequency domain feature fusion module upsamples the low-resolution compressed features to a high-resolution space, and then fuses the feature information of the high-resolution and low-resolution feature maps. The fused feature map is used as a low-resolution feature map and iteratively fused with the higher-resolution high-resolution feature map. After multiple fusions, the final high-resolution feature map is obtained, and the final high-resolution feature map is sent to the change area prediction module.

[0066] Step 6: The change area prediction module performs convolution processing on the final high-resolution features output by the frequency domain feature fusion module and outputs the final prediction result that meets the prediction channel number, thus realizing a method for finding the change area of ​​remote sensing images based on spectrum information and frequency domain features.

[0067] Table 1 Comparison of test results of various models on the LEVIR-CD dataset

[0068]

[0069] The proposed method is compared with seven state-of-the-art algorithms, including UNet++, SNUNet, BIT, ChangeFormer, and USSFCNet. The results of these seven networks on the LEVIR-CD dataset are shown in Table 1. Precision represents the proportion of true change regions among the detected change regions. A high Precision indicates fewer false positives (FPs) among the detected change regions, reducing false detections. Recall represents the proportion of all true change regions that are correctly detected. A high Recall indicates a low missed detection rate. F1 is the harmonic mean of Precision and Recall, used to balance the relationship between the two. The Intersection over Union (IoU) measures the degree of overlap between the algorithm's predicted change regions and the true change regions. A higher IoU value indicates a higher degree of overlap between the predicted and true change regions, indicating more accurate change region search results. This method is suitable for evaluating tasks requiring precise region matching, such as change region search.

[0070] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0071] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A model for finding changed areas in remote sensing images, characterized in that: The model includes: a spectrum information-guided small-size object feature extraction module, a triple-attention fusion module, a frequency domain feature fusion module, and a change region prediction module; The spectral information-guided small-size object feature extraction module is connected by a spectral information-guided module and a SwinTransformer module residual structure. The input image is converted from the spatial domain to the frequency domain through a fast Fourier transform, the spectrum of the input image is decomposed into frequency components contributed by different spatial frequencies, and the low-frequency and high-frequency regions of the input image are distinguished through spectral analysis. The appropriate convolution kernel size and receptive field are dynamically selected according to different frequency regions. The spectral information-guided small-size object feature extraction module enhances the ability to capture the edge, texture and structural features of the image through the SwinTransformer module to adapt to different frequency regions of the multi-scale input image and output feature maps of multiple scales. The three-attention fusion module acts on the feature maps of multiple scales, gradually enhancing the features of multiple scales, so that the model pays attention to the channels in the features of multiple scales and gives greater weight to the pixels in the changing area of ​​the feature map; The frequency domain feature fusion module performs frequency domain feature fusion on the feature map output by the three attention fusion modules, upsamples the low-resolution features in the feature map to the high-resolution space, and fuses the feature information of the high-resolution and low-resolution feature maps; the fused feature map is used as the low-resolution feature map and iteratively fused with the higher-resolution high-resolution feature map; The change region prediction module performs convolution processing on the final high-resolution feature map output by the frequency domain feature fusion module, and outputs a final prediction result that meets the number of prediction channels.

2. A model for finding changed areas in remote sensing images according to claim 1, characterized in that: The three-attention fusion module includes a channel attention module, a spatial attention module and a frequency domain enhanced attention module.

3. The model for finding changed areas in remote sensing images according to claim 1, characterized in that: The model is a twin structural change detection network.

4. The model for finding changed areas in remote sensing images according to claim 1, characterized in that: The spectrum information guidance module performs shallow extraction on the features of the input image through the maximum pooling module and depthwise separable convolution, and sends the extracted features to the frequency domain feature fusion module for frequency domain feature fusion.

5. A method for finding changed areas in remote sensing images, based on the model for finding changed areas in remote sensing images according to any one of claims 1 to 4, characterized in that: The method comprises the following steps: Obtaining a data set, performing data augmentation on the data set, and feeding the augmented data into a model for training; The spectral information-guided small-size object feature extraction module converts the input image from the spatial domain to the frequency domain through fast Fourier transform, decomposes the input image spectrum into frequency components contributed by different spatial frequencies, and distinguishes the low-frequency and high-frequency regions of the input image through spectral analysis. It dynamically selects the appropriate convolution kernel size and receptive field according to different frequency regions and performs convolution calculations. It then adjusts the extraction of frequency components in the image through the learning weights of the SwinTransformer module embedded in the form of a residual structure, so that the model pays more attention to high-frequency information. The output multi-scale feature map is sent to the three-attention fusion module; The channel attention module, spatial attention module, and frequency domain enhanced attention module in the three-attention fusion module act on the feature map in sequence, gradually enhancing the feature map so that the model pays attention to important channels in the feature map and gives greater weight to pixels in the changing area of ​​the feature map, thereby improving the accuracy of the feature representation of the feature map and outputting features; The frequency domain feature fusion module performs frequency domain feature fusion on the shallow features output by the spectrum information guidance module and the features of the feature map output by the triple attention fusion module, upsamples the low-resolution features to the high-resolution space, and fuses the feature information of the high-resolution and low-resolution feature maps; The fused feature map is used as a low-resolution feature map and iteratively fused with a higher-resolution feature map; The change area prediction module performs convolution processing on the final high-resolution feature map output by the frequency domain feature fusion module, and outputs a final prediction result that meets the prediction channel number, thereby realizing a method for searching for change areas in remote sensing images based on spectrum information and frequency domain features.

6. The method for finding a changed area in a remote sensing image according to claim 5, characterized in that: The method of distinguishing low-frequency and high-frequency areas of the input image by spectrum analysis includes the following steps: calculating the spectrum and frequency intensity by frequency components contributed by different spatial frequencies in the input image, and setting a threshold value according to the frequency intensity. f threshold , the frequency is higher than f threshold The area below is considered as high frequency area, and the area below is considered as high frequency area. f threshold The area is the low-frequency area.

7. The method for searching for changed areas in remote sensing images according to claim 5, characterized in that: The receptive field is dynamically selected according to different frequency regions. The frequency spectrum of the input feature map is calculated and the expansion rate is dynamically adjusted to achieve adaptive adjustment of the regional receptive field.

8. The method for searching for changed areas in remote sensing images according to claim 5, characterized in that: The steps of the channel attention module, spatial attention module and frequency domain enhanced attention module acting on the feature map in sequence are: the feature map first passes through a channel attention module and then through the frequency domain enhanced attention module, and then passes through a spatial attention module and then through the frequency domain enhanced attention module.

9. The method for searching for changed areas in remote sensing images according to claim 5, characterized in that: It also includes an internally integrated low-pass filter and a high-pass filter combined with high-resolution image features to improve the image reconstruction quality and detail recovery ability. The specific steps are: first, the high-resolution image features and the low-resolution image features are channel compressed through a convolution operation, so that the high-resolution image features and the low-resolution image features are compressed to the same number of channels and compressed features are generated; the compressed features are frequency-domain encoded to capture low-frequency information and high-frequency information and a frequency-domain filter mask is generated through local area similarity calculation and adaptive convolution, and the image reconstruction technology CARAFE is combined to fuse the information of the high-resolution image features and the low-resolution image features.

Images