Remote Sensing Image Change Detection Method, Device, Electronic Device, Storage Medium and Computer Product

Abstract

The present application relates to the field of image processing technology, and provides a remote sensing image change detection method, apparatus, electronic device, storage medium, and computer product. The method includes: inputting two-phase remote sensing images to be detected into a change detection model to obtain the output change detection result; wherein, the change detection model is constructed based on an adjusted Siamese neural network, which includes at least two encoder branches and a decoder; each encoder branch is respectively used to extract multi-scale semantic encodings of the input image by combining a first branch and a second branch therein; the decoder is used to fuse and encode the multi-scale semantic encodings respectively extracted by each encoder branch through high-frequency and low-frequency information guidance to obtain the change detection result; the first branch is a pre-trained visual foundation model; the second branch is used to perform multi-stage semantic encoding on the input image, and the semantic encoding of each stage is respectively guided by high-frequency and low-frequency information. The present application can improve the accuracy of remote sensing image change detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular, to a remote sensing image change detection method, device, electronic device, storage medium, and computer product. Background Art

[0002] Remote sensing image change detection is an important application in remote sensing technology, aiming to identify and quantify changes in land cover or ground objects by analyzing multi-temporal remote sensing images. It can help monitor various phenomena such as land use changes, urban expansion, deforestation, natural disaster impacts, and glacier ablation, providing a scientific basis for environmental monitoring, resource management, urban planning, and disaster response.

[0003] In the process of remote sensing change detection, it is usually necessary to process multi-temporal optical images, synthetic aperture radar images, or other sensor data. Since images at different time points may be affected by factors such as sensor characteristics, imaging conditions, and illumination, change detection needs to overcome these differences and accurately extract change information.

[0004] In recent years, with the rapid development of deep learning technology, change detection methods based on deep neural networks have gradually become the mainstream. Deep learning models can automatically extract complex features in images and optimize detection performance by learning a large amount of labeled data.

[0005] In practical applications, remote sensing image change detection technology can help monitor land use changes, deforestation, urban expansion, natural disaster impacts, etc., providing important support for environmental management and decision-making. In existing remote sensing image change detection technologies, the wide application of the encoder-decoder structure has brought significant performance improvements, but at the same time, some problems have also emerged, especially the artifact problem. When processing high-resolution remote sensing images, due to its inherent upsampling and downsampling processes, this structure is prone to introducing artifacts, resulting in false changes or noise in the detection results, interfering with the recognition of real changes, and reducing the accuracy of remote sensing image change detection. Summary of the Invention

[0006] The present application aims to at least solve one of the technical problems existing in the related art. For this purpose, the present application proposes a remote sensing image change detection method, device, electronic device, storage medium, and computer product to solve the problem of easily introducing artifacts in existing remote sensing image change detection technologies and achieve improved accuracy of remote sensing image change detection.

[0007] The remote sensing image change detection method according to the first aspect embodiment of the present application includes:

[0008] Obtain two-temporal remote sensing images to be detected;

[0009] Input two-phase remote sensing images to be detected into the change detection model to obtain the change detection result output by the change detection model; wherein, the change detection model is constructed based on the adjusted Siamese neural network; the adjusted Siamese neural network includes at least two encoder branches and a decoder; each encoder branch is respectively used to extract the multi-scale semantic encoding of the input image by combining the first branch and the second branch therein; the decoder is used to fuse and encode the multi-scale semantic encodings respectively extracted by each encoder branch through high-frequency and low-frequency information guidance to obtain the result of change detection; the first branch is a pre-trained visual base model; the second branch is used to perform multi-stage semantic encoding on the input image, and the semantic encoding of each stage is respectively guided by high-frequency and low-frequency information.

[0010] According to an embodiment of the present application, the second branch is specifically used for:

[0011] Extract features from the input image to obtain the initial component;

[0012] Perform frequency domain decoupling on the initial component to obtain the high-frequency component and the low-frequency component;

[0013] Based on the initial component, the high-frequency component and the low-frequency component, and combining the feature components extracted by the first branch, perform multi-stage semantic encoding.

[0014] According to an embodiment of the present application, the performing multi-stage semantic encoding based on the initial component, the high-frequency component and the low-frequency component, and combining the feature components extracted by the first branch includes:

[0015] Perform cross-attention calculation based on the high-frequency component and the initial component to obtain the first component;

[0016] Perform information gain on the first component based on the feature components extracted by the first branch to obtain the second component;

[0017] Perform cross-attention calculation based on the high-frequency component and the second component to obtain the third component;

[0018] Perform information gain on the third component based on the feature components extracted by the first branch to obtain the fourth component;

[0019] Perform cross-attention calculation based on the low-frequency component and the fourth component to obtain the fifth component;

[0020] Perform information gain on the fifth component based on the feature components extracted by the first branch to obtain the sixth component;

[0021] Perform cross-attention calculation based on the low-frequency component and the sixth component to obtain the seventh component.

[0022] According to an embodiment of the present application, the decoder is specifically configured to:

[0023] Perform feature fusion on the first component and the third component respectively extracted by each encoder branch to obtain a first fusion feature;

[0024] Perform feature fusion on the fifth component and the seventh component respectively extracted by each encoder branch to obtain a second fusion feature;

[0025] Perform feature fusion on the fifth component and the seventh component respectively extracted by each encoder branch to obtain a third fusion feature;

[0026] Perform feature fusion on the first component, the third component, the fifth component, and the seventh component respectively extracted by each encoder branch to obtain a fourth fusion feature;

[0027] Perform feature fusion on the first fusion feature and the fourth fusion feature to obtain a fifth fusion feature;

[0028] Perform feature fusion on the second fusion feature and the third fusion feature to obtain a sixth fusion feature;

[0029] Perform change detection on the fifth fusion feature and the sixth fusion feature to obtain the result of change detection.

[0030] According to an embodiment of the present application, the performing feature fusion on the first component and the third component respectively extracted by each encoder branch to obtain a first fusion feature includes:

[0031] Perform differential fusion on the first component respectively extracted by each encoder branch to obtain a first differential fusion feature;

[0032] Perform differential fusion on the third component respectively extracted by each encoder branch to obtain a second differential fusion feature;

[0033] Perform scale alignment on the first differential fusion feature and the second differential fusion feature to obtain a first fusion feature.

[0034] According to an embodiment of the present application, the performing scale alignment on the first differential fusion feature and the second differential fusion feature to obtain a first fusion feature includes:

[0035] Perform upsampling on the second differential fusion feature to obtain a third differential fusion feature;

[0036] Perform feature splicing on the third differential fusion feature and the first differential fusion feature to obtain a first fusion feature.

[0037] A remote sensing image change detection device according to an embodiment of the second aspect of the present application includes:

[0038] An acquisition module, configured to acquire two-phase remote sensing images to be detected;

[0039] A detection module, configured to input the two-phase remote sensing images to be detected into a change detection model, and obtain a change detection result output by the change detection model; wherein, the change detection model is constructed based on an adjusted Siamese neural network; the adjusted Siamese neural network includes at least two encoder branches and a decoder; each encoder branch is respectively configured to extract multi-scale semantic encodings of the input image by combining a first branch and a second branch therein; the decoder is configured to fuse and encode the multi-scale semantic encodings respectively extracted by each encoder branch through high and low frequency information guidance to obtain a change detection result; the first branch is a pre-trained visual base model; the second branch is configured to perform multi-stage semantic encoding on the input image, and the semantic encoding of each stage is respectively guided by high and low frequency information.

[0040] An electronic device according to an embodiment of the third aspect of the present application includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the remote sensing image change detection method described in any one of the above is implemented.

[0041] A storage medium according to an embodiment of the fourth aspect of the present application, the storage medium is a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the remote sensing image change detection method described in any one of the above is implemented.

[0042] A computer program product according to an embodiment of the fifth aspect of the present application includes a computer program, and when the computer program is executed by a processor, the remote sensing image change detection method described in any one of the above is implemented.

[0043] One or more of the above technical solutions in the embodiments of the present application have at least the following technical effects:

[0044] A change detection model is constructed by an adjusted Siamese neural network. Since the adjusted Siamese neural network includes at least two encoder branches and one decoder. Each encoder branch is used to extract the multi-scale semantic encoding of the input image by combining a first branch which is a pre-trained visual foundation model and a second branch for multi-stage semantic encoding of the input image; while the decoder is used to fuse the multi-scale semantic encodings separately extracted by each encoder branch through high-frequency and low-frequency information guidance and then encode to obtain the change detection result. Therefore, after obtaining the two-phase remote sensing images to be detected, input the two-phase remote sensing images to be detected into the change detection model, and the change detection result output by the change detection model can be obtained. Since the semantic encoding of each stage is guided by high-frequency and low-frequency information respectively, and the multi-scale semantic encodings are also fused through high-frequency and low-frequency information guidance, making full use of the fusion and indication information of each stage, weakening the problems of high-frequency information failing in the deep layer and low-frequency information being blurred at the edge, thus significantly reducing the artifact problem in the change detection process and realizing the improvement of the accuracy of remote sensing image change detection.

[0045] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0047] Figure 1 It is a schematic flowchart of a remote sensing image change detection method provided by an embodiment of the present application.

[0048] Figure 2 It is a schematic structural diagram of a decoder in a remote sensing image change detection method provided by an embodiment of the present application.

[0049] Figure 3 It is a schematic structural diagram of an electronic device provided by the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The following will further describe the embodiments of the present application in detail in conjunction with the drawings and embodiments. The following embodiments are used to illustrate the present application, but cannot be used to limit the scope of the present application.

[0051] In the description of the embodiments of the present application, it should be noted that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the embodiments of the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the embodiments of the present application. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0052] In the description of the embodiments of the present application, it should be noted that unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0053] In the embodiments of the present application, unless otherwise clearly specified and limited, the first feature being "on" or "under" the second feature may be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on" the second feature may be that the first feature is directly above or obliquely above the second feature, or merely indicates that the first feature has a higher horizontal height than the second feature. The first feature being "under", "beneath" and "under" the second feature may be that the first feature is directly below or obliquely below the second feature, or merely indicates that the first feature has a lower horizontal height than the second feature.

[0054] In the description of this specification, the description referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0055] It should be noted that the artifact problem is also related to the complexity of data preprocessing. The input of multi-source heterogeneous remote sensing data increases the difficulty of preprocessing, such as high-precision correction and registration of multi-resolution images. The improvement of image spatial resolution and data quality also makes the pseudo-changes caused by imaging environmental factors such as illumination, terrain, and shadow more significant. These problems not only affect the accuracy of change detection, but also pose challenges to the generalization ability of the model and the actual application effect.

[0056] Specifically, the artifact regions mostly appear near the edge transition regions between the specific change regions and the unchanged regions. The low-confidence edges caused by the upsampling strategy are the fundamental reasons for the generation of artifacts. In the pixel-level change detection task based on deep learning, the artifact interference problem is an important bottleneck restricting the model performance. The specific manifestations are as follows: When upsampling and restoring the deep features extracted by the encoder, in the spatially discontinuous regions such as object boundaries and texture mutations (i.e., the transition zones between the regions where geographical entities change and the adjacent unchanged regions), strip-shaped or patchy abnormal response patterns often occur. This phenomenon stems from the modeling defects of conventional upsampling strategies (such as bilinear interpolation and transposed convolution) for high-frequency information - during the spatial dimension reconstruction of the feature map, there is a lack of special processing mechanisms for edge-sensitive regions, resulting in systematic biases in the confidence evaluation of the model for the transition regions. Such low-confidence edges make it difficult to accurately coordinate the semantic consistency of different-level features during the decoding stage, and finally two typical artifacts appear in the prediction map: one is the jagged structural distortion, and the other is the diffuse salt-and-pepper noise. Experiments show that when the shape of the target object shows a complex and irregular distribution (such as the junction area between urban building groups and natural landforms), the artifact effect will be significantly exacerbated with the increase of spatial heterogeneity.

[0057] Based on this, the present application proposes a remote sensing image change detection method, device, electronic device, storage medium, and computer product.

[0058] Figure 1 It is a schematic flowchart of the remote sensing image change detection method provided by the embodiment of the present application. As Figure 1 shown, the remote sensing image change detection method includes:

[0059] Step 110, obtain two-phase remote sensing images to be detected.

[0060] Step 120: Input two-phase remote sensing images to be detected into the change detection model to obtain the change detection result output by the change detection model. Among them, the change detection model is constructed based on the adjusted Siamese neural network. The adjusted Siamese neural network includes at least two encoder branches and one decoder. Each encoder branch is respectively used to extract the multi-scale semantic encoding of the input image by combining the first branch and the second branch therein. The decoder is used to fuse and encode the multi-scale semantic encodings respectively extracted by each encoder branch through the guidance of high and low frequency information to obtain the result of change detection. The first branch is a pre-trained visual base model. The second branch is used to perform multi-stage semantic encoding on the input image, and the semantic encoding of each stage is respectively guided by high and low frequency information.

[0061] It should be noted that the execution subject of the remote sensing image change detection method provided in the embodiments of the present application can be a computer device, such as a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle-mounted electronic device, a wearable device, an Ultra-mobile Personal Computer (UMPC), a netbook, or a Personal Digital Assistant (PDA), etc. It should be noted that all the data that needs to be obtained in the present application is obtained through formal channels after being authorized by relevant users.

[0062] A remote sensing image change detection device can be set or connected in the computer device of the present application, so as to control the remote sensing image change detection device to execute the remote sensing image change detection method of the present application.

[0063] Specifically, the present application can obtain remote sensing images such as multi-phase optical images, synthetic aperture radar images, or other sensor data.

[0064] Furthermore, one or more preprocessing operations such as registration, denoising, and normalization can be performed on the multi-phase remote sensing images to ensure the consistency of the images in space and spectrum and provide a basis for subsequent analysis.

[0065] It should be noted that the present application can adjust the structure of the traditional Siamese neural network so that the adjusted Siamese neural network includes at least two encoder branches and one decoder. In a specific embodiment, the adjusted Siamese neural network can include two encoder branches and one decoder.

[0066] Among them, each encoder branch is respectively used to extract the multi-scale semantic encoding of the input image by combining the first branch and the second branch therein.

[0067] The first branch is a pre-trained Visual Foundation Model (VFM), which is specifically pre-trained using a natural image dataset and undertakes the core task of extracting general features. This model is pre-trained based on a large-scale natural image dataset and can capture rich visual semantic information, thereby providing high-quality initial feature representations for subsequent tasks.

[0068] Specifically, the first branch can be Clip_Vit_Base, which is a multi-modal pre-trained model based on the Contrastive Language-Image Pretraining (CLIP) framework. Its design combines the architectural advantages of the Vision Transformer (ViT) and effectively models the global dependencies in images through the self-attention mechanism. At the same time, by leveraging the knowledge learned during the pre-training process, it significantly improves the generalization ability of the model in various downstream tasks. The natural image dataset used in the pre-training stage usually contains millions of annotated images, covering a wide range of visual scenes and object categories, which enables Clip_Vit_Base to learn general visual features and be applicable to various computer vision tasks such as image classification, object detection, and semantic segmentation. By using Clip_Vit_Base as the basic feature extractor, the dependence on data annotation in downstream tasks can be significantly reduced, while improving the performance and robustness of the model.

[0069] Therefore, the input image can be feature-extracted through the first branch to obtain the feature components of the input image.

[0070] The second branch can be defined as an expert network in this application, which is used to perform multi-stage semantic encoding on the input image, and the semantic encoding at each stage is guided by high-frequency and low-frequency information respectively.

[0071] Therefore, multi-scale semantic encodings extracted by the two encoder branches for the input image can be obtained, that is, each encoder branch extracts multi-scale semantic encodings.

[0072] The decoder is used to fuse and encode the multi-scale semantic encodings extracted by each encoder branch through the guidance of high-frequency and low-frequency information to obtain the result of change detection.

[0073] In this application, the decoder can use a frequency-domain information fusion decoder based on a Multilayer Perceptron (MLP) to achieve feature aggregation of multiple encoder branches and generation of prediction maps. Its core advantage lies in that through an efficient feature extraction and aggregation mechanism, it can significantly reduce the computational cost while maintaining high accuracy.

[0074] The MLP-based frequency-domain information fusion decoder effectively integrates feature maps of different scales through hierarchical feature fusion, thus achieving a balance between semantic information and spatial details. This design not only improves the model's detection ability for small targets and edge changes but also enhances its robustness to different scale changes by combining a deep supervision strategy with multi-scale prediction maps.

[0075] Furthermore, a change detection model is constructed based on the adjusted Siamese neural network and trained using the collected multi-temporal remote sensing data as training data, thereby realizing the training of the expert network. After training, the change detection model can detect whether there are changes in the input two-temporal remote sensing images and output the change detection results.

[0076] Thus, when remote sensing image change detection is required, the two-temporal remote sensing images to be detected can be input into the change detection model, and then the change detection model can perform change detection and obtain the change detection results output by the change detection model. Among them, before inputting into the change detection model, one or more of the above preprocessing operations can be performed on the two-temporal remote sensing images to be detected.

[0077] Furthermore, according to the change detection results, various phenomena such as monitoring land use changes, urban expansion, deforestation, natural disaster impacts, and glacier ablation can be realized according to specific scenarios, providing a scientific basis for environmental monitoring, resource management, urban planning, and disaster response.

[0078] According to the remote sensing image change detection method of the embodiments of the present application, a change detection model is constructed through the adjusted Siamese neural network. Since the adjusted Siamese neural network includes at least two encoder branches and one decoder. Each encoder branch is used to extract the multi-scale semantic encoding of the input image by combining the first branch, which is a pre-trained visual foundation model, and the second branch, which is used for multi-stage semantic encoding of the input image; while the decoder is used to fuse and encode the multi-scale semantic encodings separately extracted by each encoder branch through high and low frequency information guidance to obtain the change detection results. Therefore, after obtaining the two-temporal remote sensing images to be detected, inputting the two-temporal remote sensing images to be detected into the change detection model can obtain the change detection results output by the change detection model. Since the semantic encoding of each stage is guided by high and low frequency information respectively, and the multi-scale semantic encodings are also fused through high and low frequency information guidance, making full use of the fusion and indication information of each stage, weakening the problems of high frequency information becoming ineffective in the deep layer and low frequency information being blurred at the edge, thereby significantly reducing the artifact problem in the change detection process and realizing the improvement of the accuracy of remote sensing image change detection.

[0079] Based on the above embodiments, the second branch is specifically used for:

[0080] Feature extraction is performed on the input image to obtain the initial components;

[0081] Frequency-domain decoupling is performed on the initial components to obtain high-frequency components and low-frequency components;

[0082] Based on the initial components, high-frequency components and low-frequency components, combined with the feature components extracted by the first branch, multi-stage semantic encoding is performed.

[0083] Specifically, a frequency-domain decoupling module DeCoupe is provided in the second branch of the present application, which can obtain high and low frequency components for the input image at any certain time phase. Among them, obtaining high and low frequency components can extract low-frequency components through convolution and pooling operations, and then subtract the low-frequency components from the original components to obtain high-frequency components.

[0084] Specifically, the frequency-domain decoupling module can be used to first perform a convolution operation on the input image to capture local features and obtain the initial components; then, low-frequency components are extracted from the initial components through pooling (such as average pooling or max pooling) to retain the global structure; finally, the high-frequency components are obtained by subtracting the low-frequency components from the original components, which reflect details and edge information. This decomposition method can effectively separate the global and local information in the features and is suitable for tasks such as image processing and change detection. By combining high-frequency and low-frequency components, the module can enhance the feature expression ability and improve the model performance.

[0085] Based on this, multi-stage semantic encoding can be further performed according to the initial components, high-frequency components and low-frequency components, combined with the feature components extracted by the first branch.

[0086] Specifically, in the present application, initial-stage semantic encoding can be first performed based on the initial components and high-frequency components. Specifically, an encoding module based on the SegFormer structure can be used to introduce the DeFreqTransformer Block to realize the guided learning of the features of the original image in high and low frequency stages. SegFormer is a semantic segmentation model based on Transformer, which aims to solve the challenges faced by traditional convolutional neural networks in semantic segmentation tasks.

[0087] It should be noted that in the forward process of the DeFreqTransformer Block, the initial component x in each stage will go through two attention stages. First, self-attention is performed, which is the standard Transformer attention mechanism. After self-attention, x will go through cross-attention guided by the frequency-domain component (i.e., high-frequency component or low-frequency component) x_freq in each stage, that is, using x_freq as the query, and x as the key and value to complete the cross-attention calculation.

[0088] It should be noted that the output of each stage serves as the initial component for the next stage.

[0089] Furthermore, it should be noted that after each encoding stage of this application, the second branch can utilize the bridge module in the BAN framework to communicate with the visual base model. Specifically, through a simple cross-attention method, information gain can be performed on the components of the original information (i.e., the initial components of the next stage) based on the feature components extracted by the visual base model. In terms of the data flow direction, the original information passes through both the expert network and the visual base model synchronously. Among them, the BAN framework is a deep bilinear attention network (Bilinear Attention Network, abbreviated as BAN) framework.

[0090] Furthermore, multistage semantic encoding can be completed based on the initial components and high-frequency components to obtain a specified number of components in the high-frequency stage.

[0091] Further, after the components in the last high-frequency stage communicate with the visual base model, the obtained components are combined with the low-frequency components for multistage semantic encoding to obtain a specified number of components in the low-frequency stage.

[0092] In a specific embodiment, each encoder branch can perform two-stage high-frequency semantic encoding and two-stage low-frequency semantic encoding on the input image, and finally obtain two components in the high-frequency stage and two components in the low-frequency stage.

[0093] The two encoder branches can altogether obtain four components in the high-frequency stage and four components in the low-frequency stage.

[0094] This application guides the attention focus of the expert network by selecting suitable high- and low-frequency information in different downsampling stages, weakens the problems of high-frequency information becoming ineffective in the deep layer and low-frequency information being blurred at the edges, helps reduce the artifact problem in the change detection process, and improves the accuracy of remote sensing image change detection.

[0095] Based on the above embodiments, multistage semantic encoding is performed based on the initial components, high-frequency components, and low-frequency components, in combination with the feature components extracted by the first branch, including:

[0096] Performing cross-attention calculation based on the high-frequency components and the initial components to obtain the first component;

[0097] Performing information gain on the first component based on the feature components extracted by the first branch to obtain the second component;

[0098] Performing cross-attention calculation based on the high-frequency components and the second component to obtain the third component;

[0099] Perform information gain on the third component based on the feature components extracted from the first branch to obtain the fourth component;

[0100] Perform cross-attention calculation based on the low-frequency component and the fourth component to obtain the fifth component;

[0101] Perform information gain on the fifth component based on the feature components extracted from the first branch to obtain the sixth component;

[0102] Perform cross-attention calculation based on the low-frequency component and the sixth component to obtain the seventh component.

[0103] Specifically, the present application can utilize the DeFreqTransformer Block to perform cross-attention calculation based on the high-frequency component and the initial component to obtain the first component. Specifically, self-attention can be first performed on the initial component x, and this operation is a standard transformer attention mechanism. The initial component x after self-attention will undergo cross-attention guided by the frequency-domain component x_freq at each stage. Specifically, the initial component x after self-attention can be used as the key and value, and the high-frequency component x_freq can be used as the query for cross-attention calculation, and the result of the cross-attention is defined as the first component. The first component is the component of the first high-frequency stage.

[0104] Furthermore, use the bridge module to communicate with the visual base model, and then, through a simple cross-attention method, perform information gain on the first component based on the feature components extracted by the visual base model to obtain the second component.

[0105] Furthermore, based on the same manner as the first high-frequency stage, perform cross-attention calculation based on the high-frequency component and the second component to obtain the third component. The third component is the component of the second high-frequency stage. [[ID=X]]

[0106] Furthermore, based on the same information gain method, perform information gain on the third component based on the feature components extracted from the first branch to obtain the fourth component.

[0107] Furthermore, based on the same manner as the second high-frequency stage, perform cross-attention calculation based on the low-frequency component and the fourth component to obtain the fifth component. The fifth component is the component of the first low-frequency stage.

[0108] Furthermore, based on the same information gain method, perform information gain on the fifth component based on the feature components extracted from the first branch to obtain the sixth component.

[0109] Furthermore, based on the same manner as the first low-frequency stage, perform cross-attention calculation based on the low-frequency component and the sixth component to obtain the seventh component. The seventh component is the component of the second low-frequency stage.

[0110] Thus, the multi-stage semantic encoding corresponding to the encoder branch can be completed, and the corresponding number of components can be obtained.

[0111] Furthermore, the components of each encoder branch can be input into the decoder for subsequent processing.

[0112] By selecting appropriate high-frequency and low-frequency information in different downsampling stages, this application completes the guidance of the attention focus of the expert network, weakens the problems of high-frequency information failing in the deep layer and low-frequency information being blurred at the edge, helps reduce the artifact problem in the change detection process, and improves the accuracy of remote sensing image change detection.

[0113] Based on the above embodiments, the decoder is specifically configured to:

[0114] Perform feature fusion on the first component and the third component respectively extracted from each encoder branch to obtain a first fusion feature;

[0115] Perform feature fusion on the fifth component and the seventh component respectively extracted from each encoder branch to obtain a second fusion feature;

[0116] Perform feature fusion on the fifth component and the seventh component respectively extracted from each encoder branch to obtain a third fusion feature;

[0117] Perform feature fusion on the first component, the third component, the fifth component, and the seventh component respectively extracted from each encoder branch to obtain a fourth fusion feature;

[0118] Perform feature fusion on the first fusion feature and the fourth fusion feature to obtain a fifth fusion feature;

[0119] Perform feature fusion on the second fusion feature and the third fusion feature to obtain a sixth fusion feature;

[0120] Perform change detection on the fifth fusion feature and the sixth fusion feature to obtain the result of change detection.

[0121] Figure 2 It is a schematic structural diagram of the decoder in the remote sensing image change detection method provided by the embodiments of this application. As Figure 2 shown, a high-frequency differential feedforward network High_dis FFN, a low-frequency differential feedforward network Low_dis FFN, a staged differential feedforward network Staged FFN, a high-frequency feature fusion network High merge, a low-frequency feature fusion network Low merge, and a discriminator Discriminator can be set in the decoder of this application.

[0122] Therefore, the High_dis FFN (High-frequency Differential Feedforward Network) can be utilized to perform feature fusion based on the first component I and the third component II extracted from each encoder branch respectively, so as to obtain the first fusion feature.

[0123] The Low_dis FFN (Low-frequency Differential Feedforward Network) is used to perform feature fusion based on the fifth component III and the seventh component IV extracted from each encoder branch respectively, so as to obtain the second fusion feature.

[0124] The Staged FFN (Staged Feedforward Network) is used to perform feature fusion based on the fifth component and the seventh component extracted from each encoder branch respectively, so as to obtain the third fusion feature.

[0125] The Staged FFN (Staged Feedforward Network) is used to perform feature fusion based on the first component, the third component, the fifth component and the seventh component extracted from each encoder branch respectively, so as to obtain the fourth fusion feature.

[0126] It should be noted that the above-mentioned differential feedforward networks have similar structures. Essentially, they all add positional encoding through a convolutional layer on the basis of a multi-layer perceptron to achieve better learning effects and generalization capabilities. The High_dis FFN processes the encoded components in the high-frequency stage, and the principles of the other two differential feedforward networks are the same. In this part, the decoder obtains the inputs from the two time phases for the first time and combines them to perform the differential operation. Essentially, it is from the decoder that the network truly begins to perceive changes.

[0127] Furthermore, the High merge (High-frequency Feature Fusion Network) is used to perform feature fusion based on the first fusion feature and the fourth fusion feature, so as to obtain the fifth fusion feature.

[0128] Specifically, the High merge (High-frequency Feature Fusion Network) can be used to perform feature concatenation Concate on the first fusion feature and the fourth fusion feature, and the concatenated feature is defined as the fifth fusion feature.

[0129] [ The Low merge (Low-frequency Feature Fusion Network) is used to perform feature fusion based on the second fusion feature and the third fusion feature, so as to obtain the sixth fusion feature.

[0130] Specifically, the Low merge (Low-frequency Feature Fusion Network) can be used to perform feature concatenation Concate on the second fusion feature and the third fusion feature, and the concatenated feature is defined as the sixth fusion feature.

[0131] Finally, using the discriminator Discriminator, change detection is performed based on the fifth fusion feature and the sixth fusion feature to obtain the result of change detection. Specifically, the fifth fusion feature and the sixth fusion feature can be input into the discriminator Discriminator to obtain the result output of the change detection output by the discriminator Discriminator. Among them, the result of change detection can specifically be a binary image. Different values in the image indicate whether there are changes in the corresponding pixel points.

[0132] This application redesigned the frequency-domain fusion decoder suitable for the frequency-domain information stage guidance network, and fully utilized the fusion and indication information of each stage. Specifically, the decoder part processes the multi-scale semantic coding information through the high-frequency difference feed-forward network (High_dis FFN), the low-frequency difference feed-forward network (Low_dis FFN), and the stage difference feed-forward network (Staged FFN), and extracts high-frequency, low-frequency, and stage difference features respectively. These features are effectively integrated through the fusion method guided by high and low frequency information, fully utilizing the fusion and indication information of each stage, weakening the problem of high-frequency information failing in the deep layer and low-frequency information being blurred at the edge, helping to reduce the artifact problem in the change detection process, and improving the accuracy of remote sensing image change detection.

[0133] This application also improves the model's detection ability for small targets, edge changes, and different scale changes through multi-scale feature fusion and depth supervision strategies, achieving a balance between high accuracy and high efficiency.

[0134] Based on the above embodiments, feature fusion is performed on the first component and the third component respectively extracted from each encoder branch to obtain the first fusion feature, including:

[0135] Perform differential fusion on the first components respectively extracted from each encoder branch to obtain the first differential fusion feature;

[0136] Perform differential fusion on the third components respectively extracted from each encoder branch to obtain the second differential fusion feature;

[0137] Perform scale alignment on the first differential fusion feature and the second differential fusion feature to obtain the first fusion feature.

[0138] Specifically, this application takes the use of the high-frequency difference feed-forward network High_dis FFN to perform feature fusion on the first component and the third component respectively extracted from each encoder branch as an example to describe the process of feature fusion using the difference feed-forward network.

[0139] Specifically, the High_dis FFN (High-frequency Differential Feedforward Network) can be used to perform differential fusion on the first components separately extracted by each encoder branch to obtain the first differential fusion feature. For example, the first components separately extracted by two encoder branches are subjected to differential fusion, and the differential fusion result of the two first components is defined as the first differential fusion feature.

[0140] More specifically, the difference operation can be performed on the two first components, and the result of the difference operation is the first differential fusion feature.

[0141] Based on the same principle, the High_dis FFN (High-frequency Differential Feedforward Network) is used to perform differential fusion on the third components separately extracted by each encoder branch to obtain the second differential fusion feature.

[0142] Furthermore, the High_dis FFN (High-frequency Differential Feedforward Network) can be used to perform scale alignment on the first differential fusion feature and the second differential fusion feature, and the first fusion feature is obtained after the scale alignment is completed.

[0143] In this application, the multi-scale semantic encoding information is processed by the High-frequency Differential Feedforward Network (High_dis FFN), the Low-frequency Differential Feedforward Network (Low_disFFN), and the Staged Feedforward Network (Staged FFN) to separately extract high-frequency, low-frequency, and stage differential features. These features are effectively integrated through a fusion method guided by high and low frequency information, fully utilizing the fusion and indication information of each stage, weakening the problems of high-frequency information becoming invalid in the deep layer and low-frequency information being blurred at the edge, helping to reduce the artifact problem in the change detection process, and improving the accuracy of remote sensing image change detection.

[0144] Based on the above embodiments, performing scale alignment on the first differential fusion feature and the second differential fusion feature to obtain the first fusion feature includes:

[0145] Performing upsampling processing on the second differential fusion feature to obtain the third differential fusion feature;

[0146] Performing feature splicing on the third differential fusion feature and the first differential fusion feature to obtain the first fusion feature.

[0147] Specifically, after the first differential fusion feature and the second differential fusion feature are obtained, since the scales of the two differential fusion features are different, the second differential fusion feature with a smaller scale can be first upsampled so that the scale of the obtained third differential fusion feature is the same as the scale of the first differential fusion feature. Furthermore, feature splicing can be performed on the third differential fusion feature and the first differential fusion feature, and the first fusion feature is obtained after the splicing is completed.

[0148] This application aligns features at different scales to fuse high-frequency and low-frequency features, enhancing the model's ability to capture details and global information, making it more robust and adaptable in complex scenarios, and contributing to improving the accuracy of remote sensing image change detection.

[0149] This application reduces the problem of low-confidence region artifacts in the change detection process through phased guidance of the neural network attention mechanism. That is, the features decoded in the high-resolution small receptive field part are guided by high-frequency information for sharp edge segmentation, and the features decoded in the low-resolution large receptive field part are guided by low-frequency information for qualitative block segmentation.

[0150] Furthermore, this application can conduct comparative experiments based on the LEVIR-CD dataset. Among them, the LEVIR-CD dataset is one of the recognized important benchmark datasets in the field of remote sensing change detection. ChangeFormer and BAN are milestone technologies in the field of remote sensing change detection. Both of these two technologies and this application use dual-temporal input to obtain binary output. Under fair and identical conditions, this application is compared with the above two technologies on the LEVIR-CD dataset.

[0151] The experimental results are shown in Table 1 below:

[0152] Table 1 Comparison of the accuracy of DeFreq before and after improvement on the LEVIR-CD dataset

[0153]

[0154] The F1 score is the harmonic mean of precision and recall, used to comprehensively measure the accuracy and integrity of the model; the Intersection over Union (IoU) represents the overlap degree between the predicted region and the true region, used to evaluate the accuracy of segmentation or detection tasks; precision (Pre) measures the proportion of samples predicted as positive that are actually positive by the model, reflecting the reliability of the model's prediction.

[0155] This application is superior to the current advanced technology BAN in all major indicators, achieving IoU +0.77%, F1 +0.45%, and Pre +1.37% respectively, and is at the leading level. The experiment shows that the phased frequency-domain attention guidance mechanism can improve the accuracy.

[0156] The remote sensing image change detection device provided by this application is described below. The remote sensing image change detection device described below can be mutually referred to the remote sensing image change detection method described above.

[0157] Furthermore, this application also provides a remote sensing image change detection device.

[0158] The remote sensing image change detection device described above includes:

[0159] An acquisition module, configured to acquire two-phase remote sensing images to be detected;

[0160] A detection module, configured to input the two-phase remote sensing images to be detected into a change detection model, and obtain a change detection result output by the change detection model; wherein, the change detection model is constructed based on an adjusted Siamese neural network; the adjusted Siamese neural network includes at least two encoder branches and a decoder; each encoder branch is respectively configured to extract multi-scale semantic encodings of the input image by combining a first branch and a second branch therein; the decoder is configured to fuse and encode the multi-scale semantic encodings respectively extracted by each encoder branch through high-frequency and low-frequency information guidance to obtain a change detection result; the first branch is a pre-trained visual foundation model; the second branch is configured to perform multi-stage semantic encoding on the input image, and the semantic encoding of each stage is respectively guided by high-frequency and low-frequency information.

[0161] The remote sensing image change detection device of the present application constructs a change detection model through an adjusted Siamese neural network. Since the adjusted Siamese neural network includes at least two encoder branches and a decoder. Each encoder branch is configured to extract multi-scale semantic encodings of the input image by combining a first branch which is a pre-trained visual foundation model and a second branch which is configured to perform multi-stage semantic encoding on the input image; and the decoder is configured to fuse and encode the multi-scale semantic encodings respectively extracted by each encoder branch through high-frequency and low-frequency information guidance to obtain a change detection result. Therefore, after acquiring the two-phase remote sensing images to be detected, inputting the two-phase remote sensing images to be detected into the change detection model can obtain the change detection result output by the change detection model. Since the semantic encoding of each stage is respectively guided by high-frequency and low-frequency information, and the multi-scale semantic encodings are also fused through high-frequency and low-frequency information guidance, the fusion and indication information of each stage are fully utilized, weakening the problems of high-frequency information failure in the deep layer and low-frequency information blurring at the edge, thereby significantly reducing the artifact problem in the change detection process and achieving improved accuracy of remote sensing image change detection.

[0162] Figure 3 An example of a schematic physical structure diagram of an electronic device is shown as Figure 3 shown. The electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340. Among them, the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call logic instructions in the memory 330 to execute the following method: acquire two-phase remote sensing images to be detected;

[0163] Input two - phase remote sensing images to be detected into the change detection model to obtain the change detection result output by the change detection model; wherein, the change detection model is constructed based on the adjusted Siamese neural network; the adjusted Siamese neural network includes at least two encoder branches and a decoder; each encoder branch is respectively used to extract the multi - scale semantic encoding of the input image by combining the first branch and the second branch therein; the decoder is used to fuse and encode the multi - scale semantic encodings respectively extracted by each encoder branch through the guidance of high - and low - frequency information to obtain the result of change detection; the first branch is a pre - trained visual basic model; the second branch is used to perform multi - stage semantic encoding on the input image, and the semantic encoding of each stage is respectively guided by high - and low - frequency information.

[0164] In addition, when the logical instructions in the above - mentioned memory 330 can be implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer - readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the related technology, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read - only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0165] On the other hand, the embodiments of the present application also provide a non - transitory computer - readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the methods provided in the above - mentioned embodiments, for example, including: obtaining two - phase remote sensing images to be detected;

[0166] Input two - phase remote - sensing images to be detected into the change detection model to obtain the change detection result output by the change detection model. Among them, the change detection model is constructed based on the adjusted Siamese neural network. The adjusted Siamese neural network includes at least two encoder branches and a decoder. Each encoder branch is respectively used to extract the multi - scale semantic encoding of the input image by combining the first branch and the second branch therein. The decoder is used to fuse and encode the multi - scale semantic encodings respectively extracted by each encoder branch through the guidance of high - and low - frequency information to obtain the change detection result. The first branch is a pre - trained visual basic model. The second branch is used to perform multi - stage semantic encoding on the input image, and the semantic encoding of each stage is respectively guided by high - and low - frequency information.

[0167] In another aspect, an embodiment of the present application also provides a computer program product, on which a computer program is stored. When the computer program is executed by a processor, it is used to execute the methods provided in the above - mentioned embodiments, for example, including: obtaining two - phase remote - sensing images to be detected;

[0168] Input two - phase remote - sensing images to be detected into the change detection model to obtain the change detection result output by the change detection model. Among them, the change detection model is constructed based on the adjusted Siamese neural network. The adjusted Siamese neural network includes at least two encoder branches and a decoder. Each encoder branch is respectively used to extract the multi - scale semantic encoding of the input image by combining the first branch and the second branch therein. The decoder is used to fuse and encode the multi - scale semantic encodings respectively extracted by each encoder branch through the guidance of high - and low - frequency information to obtain the change detection result. The first branch is a pre - trained visual basic model. The second branch is used to perform multi - stage semantic encoding on the input image, and the semantic encoding of each stage is respectively guided by high - and low - frequency information.

[0169] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0170] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the embodiments, those of ordinary skill in the art should understand that various combinations, modifications, or equivalent replacements of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application.

Claims

1. A remote sensing image change detection method, characterized in that Including: Obtain two-phase remote sensing images to be detected; Input the two-phase remote sensing images to be detected into a change detection model to obtain a change detection result output by the change detection model; wherein, the change detection model is constructed based on an adjusted Siamese neural network; the adjusted Siamese neural network includes at least two encoder branches and a decoder; each encoder branch is respectively used to extract multi-scale semantic encodings of the input image by combining a first branch and a second branch therein; the decoder is used to fuse and decode the multi-scale semantic encodings respectively extracted by each encoder branch through high and low frequency information guidance to obtain a change detection result; the first branch is a pre-trained visual foundation model; the second branch is used to perform multi-stage semantic encoding on the input image, and the semantic encoding of each stage is respectively guided by high and low frequency information; The second branch is specifically used for: Extract features from the input image to obtain an initial component; Perform frequency domain decoupling on the initial component to obtain a high-frequency component and a low-frequency component; Based on the initial component, the high-frequency component, and the low-frequency component, combined with the feature components extracted by the first branch, perform multi-stage semantic encoding; The performing multi-stage semantic encoding based on the initial component, the high-frequency component, and the low-frequency component, combined with the feature components extracted by the first branch, includes: Perform cross-attention calculation based on the high-frequency component and the initial component to obtain a first component; Perform information gain on the first component based on the feature components extracted by the first branch to obtain a second component; Perform cross-attention calculation based on the high-frequency component and the second component to obtain a third component; Perform information gain on the third component based on the feature components extracted by the first branch to obtain a fourth component; Perform cross-attention calculation based on the low-frequency component and the fourth component to obtain a fifth component; Perform information gain on the fifth component based on the feature components extracted by the first branch to obtain a sixth component; Perform cross-attention calculation based on the low-frequency component and the sixth component to obtain a seventh component.

2. The remote sensing image change detection method according to claim 1, wherein The decoder is specifically used for: Perform feature fusion on the first component and the third component respectively extracted by each encoder branch to obtain a first fusion feature; Perform feature fusion on the fifth component and the seventh component respectively extracted by each encoder branch to obtain a second fusion feature; Perform feature fusion on the fifth component and the seventh component respectively extracted by each encoder branch to obtain a third fusion feature; Perform feature fusion on the first component, the third component, the fifth component, and the seventh component respectively extracted by each encoder branch to obtain a fourth fusion feature; Perform feature fusion on the first fusion feature and the fourth fusion feature to obtain a fifth fusion feature; Perform feature fusion on the second fusion feature and the third fusion feature to obtain a sixth fusion feature; Perform change detection on the fifth fusion feature and the sixth fusion feature to obtain a change detection result.

3. The remote sensing image change detection method according to claim 2, characterized in that The performing feature fusion on the first component and the third component respectively extracted by each encoder branch to obtain a first fusion feature, includes: Perform differential fusion on the first components separately extracted by each encoder branch to obtain a first differentially fused feature; Perform differential fusion on the third components separately extracted by each encoder branch to obtain a second differentially fused feature; Perform scale alignment on the first differentially fused feature and the second differentially fused feature to obtain a first fused feature.

4. The remote sensing image change detection method according to claim 3, characterized in that The performing scale alignment on the first differentially fused feature and the second differentially fused feature to obtain a first fused feature includes: Perform upsampling on the second differentially fused feature to obtain a third differentially fused feature; Perform feature concatenation on the third differentially fused feature and the first differentially fused feature to obtain a first fused feature.

5. A remote sensing image change detection device, characterized in that, Includes: An acquisition module for acquiring two-temporal remote sensing images to be detected; A detection module for inputting the two-temporal remote sensing images to be detected into a change detection model to obtain a change detection result output by the change detection model; wherein, the change detection model is constructed based on an adjusted Siamese neural network; the adjusted Siamese neural network includes at least two encoder branches and a decoder; each encoder branch is respectively used to extract multi-scale semantic encodings of the input image by combining a first branch and a second branch therein; the decoder is used to fuse the multi-scale semantic encodings separately extracted by each encoder branch through high-frequency and low-frequency information guidance and then decode to obtain a change detection result; the first branch is a pre-trained visual base model; the second branch is used to perform multi-stage semantic encoding on the input image, and the semantic encoding at each stage is respectively guided by high-frequency and low-frequency information; The second branch is specifically used for: performing feature extraction on the input image to obtain an initial component; performing frequency-domain decoupling on the initial component to obtain a high-frequency component and a low-frequency component; performing multi-stage semantic encoding based on the initial component, the high-frequency component, and the low-frequency component in combination with the feature components extracted by the first branch; The performing multi-stage semantic encoding based on the initial component, the high-frequency component, and the low-frequency component in combination with the feature components extracted by the first branch includes: performing cross-attention calculation on the high-frequency component and the initial component to obtain a first component; performing information gain on the first component based on the feature components extracted by the first branch to obtain a second component; performing cross-attention calculation on the high-frequency component and the second component to obtain a third component; performing information gain on the third component based on the feature components extracted by the first branch to obtain a fourth component; performing cross-attention calculation on the low-frequency component and the fourth component to obtain a fifth component; performing information gain on the fifth component based on the feature components extracted by the first branch to obtain a sixth component; performing cross-attention calculation on the low-frequency component and the sixth component to obtain a seventh component.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the remote sensing image change detection method according to any one of claims 1-4.

7. A storage medium, the storage medium being a non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, it implements the remote sensing image change detection method according to any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the remote sensing image change detection method according to any one of claims 1-4.

Images