Image processing method, processing device and computer program product

Abstract

The application provides an image processing method, a processing device and a computer program product, relates to the technical field of image processing, and the image processing method comprises the following steps: acquiring a first remote sensing image and a second remote sensing image of a same region acquired at different times; inputting the first remote sensing image and the second remote sensing image into a shared weight double-branch encoder to extract a multi-scale double-time feature map; inputting the multi-scale double-time feature map into a double-time change information fusion module for difference fusion to obtain change fusion features; inputting the change fusion features into a large receptive field feature aggregation module for feature recovery and aggregation to obtain recovered features; and performing convolution classification on the recovered features to generate a change detection result map. The application can efficiently model long-distance spatial dependence and retain local details, effectively capture long-distance spatial dependence of large-scale ground objects while retaining local details at a pixel level, improve change detection precision, and reduce computational complexity.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, processing device, and computer program product. Background Technology

[0002] Remote sensing image change detection, as a core application of remote sensing technology, is mainly used to monitor dynamic changes on the Earth's surface, such as urban expansion, farmland conversion, and the impact of natural disasters. However, existing technologies have significant limitations when processing high-resolution remote sensing data. Processing high-resolution remote sensing images consumes a large amount of computing resources and memory, making it difficult to meet the efficiency requirements of practical applications. Summary of the Invention

[0003] The main objective of this invention is to provide an image processing method that aims to improve the accuracy of change detection and reduce computational complexity.

[0004] To achieve the above objectives, the present invention provides an image processing method applied to remote sensing image processing, the image processing method comprising:

[0005] Acquire the first and second remote sensing images of the same area acquired at different times;

[0006] The first remote sensing image and the second remote sensing image are input into a shared weight dual-branch encoder to extract multi-scale dual-temporal feature maps;

[0007] The multi-scale dual-temporal feature map is input into the dual-temporal change information fusion module for difference fusion to obtain the change fusion feature, wherein the dual-temporal change information fusion module has a built-in large convolutional kernel state space module.

[0008] The change fusion features are input into the large receptive field feature aggregation module for feature recovery and aggregation to obtain the recovered features, wherein the large receptive field feature aggregation module includes the large convolutional kernel state space module.

[0009] Perform convolutional classification on the recovered features to generate a change detection result map.

[0010] Optionally, the step of inputting the first remote sensing image and the second remote sensing image into a shared weighted dual-branch encoder to extract multi-scale dual-temporal feature maps includes:

[0011] The first remote sensing image and the second remote sensing image are respectively input into the shared weight dual-branch encoder;

[0012] The first remote sensing image is used to extract features through the shared weight dual-branch encoder to obtain first shallow features and first deep features;

[0013] The second remote sensing image is used to extract features through the shared weight dual-branch encoder to obtain second shallow features and second deep features;

[0014] The first shallow feature, the first deep feature, the second shallow feature, and the second deep feature are combined to form the multi-scale dual-temporal feature map.

[0015] Optionally, the shared weight dual-branch encoder is a twin feature extraction network that uses the same network structure and shares parameters;

[0016] The first shallow feature and the second shallow feature have a first downsampling ratio relative to the first remote sensing image and the second remote sensing image; the first deep feature and the second deep feature have a second downsampling ratio relative to the first remote sensing image and the second remote sensing image; wherein the second downsampling ratio is greater than the first downsampling ratio.

[0017] Optionally, the step of inputting the multi-scale dual-temporal feature map into the dual-temporal change information fusion module for difference fusion to obtain change fusion features, wherein the dual-temporal change information fusion module has a built-in large convolutional kernel state space module, including:

[0018] Based on the scale of the multi-scale dual-temporal feature map, the multi-scale dual-temporal feature map is divided into at least two scale-integrated feature pairs, each scale-integrated feature pair including a first temporal feature and a second temporal feature;

[0019] For each of the same-scale feature pairs, the first temporal feature and the second temporal feature are concatenated along the channel dimension to obtain the concatenated feature;

[0020] The spliced ​​features are normalized and processed by a multilayer perceptron to obtain intermediate features;

[0021] The intermediate features are input into the large convolutional kernel state space module for spatial modeling to obtain deep aggregated features;

[0022] Perform convolutional channel compression on the spliced ​​features to obtain the basic convolutional features;

[0023] The deep aggregated features are fused with the basic convolutional features weighted by learnable coefficients, and the corresponding scale-dependent fused features are obtained through convolution processing.

[0024] The change fusion features corresponding to each scale are combined into the change fusion features.

[0025] Optionally, the change fusion feature includes a first change fusion feature and a second change fusion feature, wherein the downsampling ratio corresponding to the second change fusion feature is greater than the downsampling ratio corresponding to the first change fusion feature;

[0026] The step of inputting the fused feature into the large receptive field feature aggregation module for feature recovery and aggregation to obtain the recovered features includes:

[0027] The second change fusion feature is input into the large receptive field feature aggregation module for processing and upsampling to obtain the second recovery feature;

[0028] The first change fusion feature is fused with the second recovery feature, the fused feature is input into the large receptive field feature aggregation module for processing, and upsampling is performed to obtain the first recovery feature, which is then used as the recovery feature.

[0029] The second change fusion feature is input into the large receptive field feature aggregation module for processing, including:

[0030] The second variation fusion feature input to the large receptive field feature aggregation module is subjected to convolution, normalization, and activation processing to obtain compressed features;

[0031] The compressed features are subjected to average pooling and linear transformation to obtain channel weights;

[0032] The compressed features are input into the large convolutional kernel state space module to obtain context-enhanced features;

[0033] The context enhancement features are multiplied element-wise with the channel weights and then convolution is performed to obtain the aggregated output features.

[0034] Optionally, performing convolutional classification on the recovered features to generate a change detection result map includes:

[0035] Perform large-size convolution processing on the restored features to map the restored features into pixel-level change probability maps;

[0036] The change detection result map is generated based on the change probability map;

[0037] The kernel size for the large-size convolution processing is N*N.

[0038] Optionally, the image processing method further includes a training process, which includes:

[0039] Obtain a training sample set carrying pixel-level change labels;

[0040] The initial image processing network is iteratively trained using the training sample set. In each iteration, the training samples are input into the image processing network to obtain the predicted change detection result map.

[0041] Based on the predicted change detection result map and the pixel-level change label, calculate the value of the joint loss function;

[0042] Based on the value of the joint loss function, the network parameters of the image processing network are updated through backpropagation;

[0043] The joint loss function is a combination of the cross-entropy loss function and the normalized exponential loss function based on the Lovarz extension.

[0044] Optionally, the large convolution kernel state space module includes a reparameterized large convolution branch and a two-dimensional state space scan branch;

[0045] The large convolutional kernel state space module is used to process the module input features to obtain the module output features. The processing includes:

[0046] The module input features are input into the reparameterized large convolution branch, and multi-scale local spatial features are extracted through multiple depthwise separable convolutions with different dilation rates;

[0047] The multi-scale local spatial features are input into the two-dimensional state space scanning branch, and the long-range dependency modeling of the multi-scale local spatial features is performed through a four-way selective scanning mechanism to obtain long-distance correlation features.

[0048] The long-range correlation features are subjected to convolution, normalization, and activation processing, and residual fusion is performed with the module input features to obtain the module output features;

[0049] During the training phase, the reparameterized large convolutional branch extracts multi-scale receptive field features in parallel through multiple branches. During the inference phase, the parameters of the multiple depthwise separable convolutions with different dilation rates are folded into a single equivalent large convolutional kernel. The equivalent receptive field size of the equivalent large convolutional kernel is equal to the difference between the kernel size and the dilation rate, multiplied by one.

[0050] In addition, to achieve the above objectives, the present invention also provides a processing apparatus, the processing apparatus comprising: a memory, a processor, and an image processing program stored in the memory and executable on the processor, the image processing program being configured to implement the image processing method as described above.

[0051] In addition, to achieve the above objectives, the present invention also provides a computer program product, including the processing apparatus described above.

[0052] This invention acquires first and second remote sensing images of the same area at different times. These images are then input into a shared-weight dual-branch encoder to extract multi-scale dual-temporal feature maps. These feature maps are then input into a dual-temporal change information fusion module for difference fusion to obtain change fusion features. This module incorporates a large convolutional kernel state space module. The change fusion features are then input into a large receptive field feature aggregation module for feature recovery and aggregation to obtain recovered features. This module also includes a large convolutional kernel state space module. Finally, convolutional classification is performed on the recovered features to generate a change detection result map. This approach efficiently models long-distance spatial dependencies while preserving local details, effectively capturing long-distance spatial dependencies of large-scale ground features, retaining pixel-level local details, improving change detection accuracy, and reducing computational complexity. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a schematic flowchart of an image processing method according to an embodiment of the present invention;

[0056] Figure 2 for Figure 1 A flowchart illustrating step S200 in the diagram;

[0057] Figure 3 for Figure 1 A flowchart illustrating step S300 in the diagram;

[0058] Figure 4 for Figure 1 A flowchart illustrating step S400 in the diagram;

[0059] Figure 5 for Figure 4 A flowchart illustrating step S410 in the diagram;

[0060] Figure 6 for Figure 1 A flowchart illustrating the method of step S500 in the diagram;

[0061] Figure 7 This is a schematic flowchart of an image processing method according to another embodiment of the present invention;

[0062] Figure 8 This is a schematic flowchart of an image processing method according to another embodiment of the present invention.

[0063] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0064] Remote sensing image change detection, a core application of remote sensing technology, is primarily used to monitor dynamic changes on the Earth's surface, such as urban expansion, farmland conversion, and the impact of natural disasters. This task identifies changed pixel regions by comparing remote sensing images of the same area acquired at different times. However, existing technologies have significant limitations when processing high-resolution remote sensing data. While convolutional neural networks possess strong local feature extraction capabilities, their receptive fields are fixed and narrow, making it difficult to effectively capture long-distance spatial dependencies of large-scale features (such as building complexes or rivers), leading to information loss when detecting large-scale changes. Transformer-based models, although capable of modeling global context, suffer from computational complexity that increases quadratically with image size, consuming significant computational resources and memory when processing high-resolution remote sensing images, failing to meet the efficiency requirements of practical applications. State-space models, while offering linear computational complexity and global modeling advantages, tend to overlook pixel-level local spatial details during global information aggregation, resulting in blurred boundaries and lost details in change detection results, leading to missed and false detections. These shortcomings make it difficult for existing methods to simultaneously achieve computational efficiency, global context awareness, and local feature preservation in remote sensing image change detection tasks, severely limiting detection accuracy and practicality.

[0065] Remote sensing image change detection (RSCD) aims to identify surface changes by comparing two-phase remote sensing images.

[0066] Existing technological shortcomings:

[0067] Convolutional Neural Networks (CNNs): They have strong feature representation capabilities, but are limited by a fixed and narrow receptive field, making it difficult to capture long-distance spatial dependencies.

[0068] Transformer: Although it has long-range modeling capabilities, the computational complexity of its self-attention mechanism increases quadratically with image size, resulting in huge resource consumption when processing high-resolution remote sensing images.

[0069] State-space model (SSM / Mamba): Although it has linear computational complexity and global modeling capabilities, simple global aggregation can easily sacrifice pixel-level local spatial details.

[0070] The problem solved by this invention is: how to obtain global contextual information with a large receptive field while maintaining linear computational complexity, and accurately preserve local pixel-level features.

[0071] The main solution of this application embodiment is as follows: First and second remote sensing images of the same area acquired at different times are obtained. The first and second remote sensing images are then input into a shared weighted dual-branch encoder to extract multi-scale dual-temporal feature maps. The multi-scale dual-temporal feature maps are then input into a dual-temporal change information fusion module for difference fusion to obtain change fusion features. The dual-temporal change information fusion module has a built-in large convolutional kernel state space module. The change fusion features are then input into a large receptive field feature aggregation module for feature recovery and aggregation to obtain recovered features. The large receptive field feature aggregation module includes a large convolutional kernel state space module. Finally, convolutional classification is performed on the recovered features to generate a change detection result map.

[0072] In this embodiment, for ease of description, the following description uses the processing device as the execution subject.

[0073] This application provides a solution that can efficiently model long-distance spatial dependencies while preserving local details. It can effectively capture long-distance spatial dependencies of large-scale ground features, while retaining pixel-level local details, improving change detection accuracy, and reducing computational complexity.

[0074] The overall concept of this invention is as follows:

[0075] This invention proposes a large convolutional kernel state space model (LKMamba-CD), which combines a reparameterized large convolutional kernel (RepLK) with a two-dimensional state space model (SS2D) to construct the core component LKS module, thereby achieving efficient long-range feature association modeling under a large receptive field.

[0076] The core module is designed as follows:

[0077] The principle of the Large Convolutional Kernel SS2D module (LKS) is as follows: using reparameterization technology, depthwise separable convolutions (DWC) with different dilation rates are fused into an equivalent large convolutional kernel, and multi-scale local features are extracted first.

[0078] Combination: Based on local features, the SS2D scanning mechanism is introduced to capture global dependencies using its four-way scanning mechanism.

[0079] Dual-Temporal Variation Information Fusion Module (BCIF): Used to aggregate the dual-temporal features extracted by the encoder. It models spatiotemporal differences through the LKS module and uses a learnable parameter $\lambda$ to balance deep aggregated features with basic features, effectively suppressing "pseudo-variations" caused by illumination, seasons, etc.

[0080] Large Receptive Field Feature Aggregation Module (LRFA): During the decoding stage, an efficient attention mechanism and the LKS module are used to enhance the correlation of features with different dimensions of change, so as to restore the integrity of large-scale ground feature information while recovering high-dimensional semantics.

[0081] The technical process is as follows:

[0082] Feature extraction: A dual-branch UniRepLKNet with shared weights is used as the encoder to extract features from bi-temporal images at scales of 1 / 8 and 1 / 32.

[0083] Difference fusion: The extracted features are input into the BCIF module for spatiotemporal semantic alignment to generate a fused representation of the changes.

[0084] Feature recovery: The LRFA module upsamples step by step and aggregates contextual information.

[0085] Classification output: Using a 7x7 large convolutional layer as a classifier, the final change detection map is output.

[0086] Technical advantages of this invention (innovation of this invention)

[0087] Architectural innovation: For the first time, highly parameterized large convolutions are seamlessly integrated with the Mamba architecture, combining the local representation advantages of CNNs with the linear global modeling capabilities of Mamba.

[0088] Receptive field optimization: Through the design of the LKS module, an effective receptive field far exceeding that of traditional CNNs is achieved, which can better capture the overall changes of large-scale land features (such as building complexes and farmland).

[0089] Computational efficiency: It maintains linear computational complexity and has significant speed and memory advantages when processing high-resolution remote sensing data compared to Transformer-type models.

[0090] Accuracy improvement: Experiments on standard datasets such as S2Looking and WHU-CD show that the F1 scores reach 62.02% and 92.52% respectively, significantly reducing the false negative and false positive rates.

[0091] Therefore, the present invention proposes an image processing method; it is understood that the computer program product is provided with a processing device for storing and executing the following method, and the processing device can be implemented by a main controller, such as MCU (Microcontroller Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), SOC (System On Chip), etc.

[0092] Traditional remote sensing image change detection methods face challenges when processing high-resolution images. Convolutional neural networks are limited by a fixed receptive field, making it difficult to capture long-range spatial dependencies; Transformer models have high computational complexity and consume significant resources; while state-space models possess global modeling capabilities, they tend to sacrifice local details. These limitations make it difficult for existing methods to maintain computational efficiency while accurately acquiring both global contextual information and pixel-level local features.

[0093] Therefore, referring to Figure 1 In one embodiment of the present invention, the image processing method is applied to remote sensing image processing. The image processing method includes steps S100-S500, wherein:

[0094] S100: Acquire the first and second remote sensing images of the same area acquired at different times;

[0095] S200. Input the first remote sensing image and the second remote sensing image into the shared weight dual-branch encoder to extract multi-scale dual-temporal feature maps;

[0096] S300. Input the multi-scale dual-temporal feature map into the dual-temporal change information fusion module for difference fusion to obtain the change fusion feature. The dual-temporal change information fusion module has a built-in large convolutional kernel state space module.

[0097] S400. Input the change fusion features into the large receptive field feature aggregation module for feature recovery and aggregation to obtain the recovered features. The large receptive field feature aggregation module includes a large convolutional kernel state space module.

[0098] S500: Perform convolutional classification on the recovered features to generate a change detection result map.

[0099] Remote sensing image processing refers to the analysis, interpretation, and transformation of image data acquired through remote sensing technology, aiming to extract useful information such as land cover change and disaster monitoring. A shared-weight bi-branch encoder is a neural network structure containing two branches with the same network structure and shared parameters. This encoder is used to process input images from different time phases to ensure consistency and comparability between time phases during feature extraction. Multi-scale bi-temporal feature maps refer to feature representations extracted from remote sensing images from different time phases, possessing different resolutions and levels of abstraction. These feature maps contain texture, shape, and semantic information of the image at different scales and preserve the correspondence between the bi-temporal images. The bi-temporal change information fusion module is used to perform differential analysis and fusion of multi-scale feature maps from different time phases. This module aims to identify and aggregate change information of the image at different time points, thereby generating feature representations that highlight areas of change.

[0100] The large convolutional kernel state space module is a feature processing unit that combines the advantages of large convolutional kernels and state space models. This module effectively captures local details in images and achieves long-distance dependency modeling through the state space model, thus obtaining global contextual information with a large receptive field while maintaining computational efficiency. The change fusion feature is the feature representation obtained after processing by the dual-temporal change information fusion module. This feature fuses the difference information of the two-temporal images and is designed for subsequent change detection tasks, clearly indicating the regions in the image where changes have occurred. The large receptive field feature aggregation module further processes the change fusion feature to restore feature details and aggregate global contextual information. This module, through its large receptive field characteristics, can effectively integrate information from different regions, thereby generating more complete and robust restored features. The restored feature is the final feature representation obtained after processing by the large receptive field feature aggregation module. This feature contains rich change information and spatial context, providing high-quality input for subsequent pixel-level classification.

[0101] Convolutional classification refers to pixel-level classification of feature maps using convolutional layers. By scanning the feature map with convolutional kernels, the probability of each pixel belonging to the "changed" or "unchanged" category can be predicted, thus generating a change detection result map. The change detection result map is the final output of the image processing method, representing the changed areas in the remote sensing image as a binary image or probability map. Different colors or grayscale values ​​are typically used to distinguish changed and unchanged areas in the image.

[0102] The image processing method in this embodiment first acquires a first remote sensing image and a second remote sensing image of the same area acquired at different times. These images can be data from satellites, aircraft, or other remote sensing platforms. For example, they can be acquired by retrieving from a remote sensing image database or by directly acquiring them through sensors. In practical applications, image acquisition may involve preprocessing steps such as georegistration and radiometric correction to ensure that the two-temporal images are comparable in spatial and radiometric characteristics.

[0103] Subsequently, the first and second remote sensing images are input into a shared-weight dual-branch encoder to extract multi-scale bi-temporal feature maps. This encoder can consist of two independent feature extraction networks, each processing one temporal image, with weight sharing indirectly achieved during training through a shared loss function or parameter regularization. Alternatively, the bi-temporal images can be concatenated along the channel dimension and then processed by a single feature extraction network to obtain feature maps containing bi-temporal information. These feature maps can contain feature layers of different resolutions, for example, obtained through successive downsampling operations.

[0104] Next, the multi-scale dual-temporal feature map is input into the dual-temporal change information fusion module for difference fusion to obtain the change fused features. This fusion module can be implemented in various ways. For example, it can highlight change regions by performing simple element-wise subtraction or addition operations on the dual-temporal feature maps. Alternatively, the dual-temporal feature maps can be concatenated along the channel dimension and then transformed and fused through one or more convolutional layers. The dual-temporal change information fusion module incorporates a large convolutional kernel state space module, which can spatially model the input features to capture local and long-range dependencies.

[0105] The fused feature is input into a large receptive field feature aggregation module for feature recovery and aggregation, yielding the recovered features. This aggregation module can consist of a series of upsampling layers and convolutional layers to progressively restore the spatial resolution of the feature map and integrate variation information at different scales. For example, the low-resolution fused feature can be directly upsampled to the original image resolution and then refined using convolutional layers. Alternatively, skip connections can be used to fuse shallow features extracted from the encoder with upsampled deep features to supplement detailed information. The large receptive field feature aggregation module includes a large convolutional kernel state space module, which further enhances the context awareness of the features at this stage.

[0106] Finally, convolutional classification is performed on the recovered feature to generate a change detection result map. This classification process can be implemented using a simple 1x1 convolutional layer, mapping the number of channels in the recovered feature to the number of classes. This convolutional layer generates a classification score or probability value for each pixel, and then, through thresholding or post-processing steps, these scores are converted into the final binary change detection result map. For example, a fixed threshold can be set, marking pixels above the threshold as changed regions and pixels below the threshold as unchanged regions.

[0107] The image processing method in this embodiment effectively extracts the difference information from dual-temporal remote sensing images by introducing a shared-weight dual-branch encoder and multi-scale feature processing. In particular, the built-in large convolutional kernel state space module in the change information fusion and feature aggregation stages enables the model to effectively capture global contextual information with a large receptive field while maintaining linear computational complexity and accurately preserving pixel-level local spatial details. Therefore, this method overcomes the problems of insufficient long-distance dependency modeling and high computational resource consumption in traditional methods for change detection in remote sensing images, significantly improving the accuracy and efficiency of change detection.

[0108] This embodiment acquires first and second remote sensing images of the same area at different times. These images are then input into a shared-weight dual-branch encoder to extract multi-scale dual-temporal feature maps. These feature maps are then input into a dual-temporal change information fusion module for difference fusion to obtain change fusion features. This module incorporates a large convolutional kernel state space module. The change fusion features are then input into a large receptive field feature aggregation module for feature recovery and aggregation to obtain recovered features. This module also includes a large convolutional kernel state space module. Finally, convolutional classification is performed on the recovered features to generate a change detection result map. This approach efficiently models long-distance spatial dependencies while preserving local details, effectively capturing long-distance spatial dependencies of large-scale ground features, and retaining pixel-level local details, thereby improving change detection accuracy and reducing computational complexity.

[0109] The key to subsequent change detection lies in effectively inputting the first and second remote sensing images into a shared-weight dual-branch encoder and systematically extracting multi-scale feature maps that reflect dual-temporal information. Insufficiently refined feature extraction or failure to fully utilize dual-temporal information may affect the accuracy of the final change detection.

[0110] Therefore, referring to Figure 2 Another embodiment of the present invention provides an image processing method based on the above. Figure 1The embodiment shown inputs a first remote sensing image and a second remote sensing image into a shared weighted dual-branch encoder to extract multi-scale dual-temporal feature maps, including steps S210-S240, wherein:

[0111] S210. Input the first remote sensing image and the second remote sensing image into the shared weight dual-branch encoder respectively;

[0112] S220. The first remote sensing image is subjected to feature extraction by a shared weight dual-branch encoder to obtain the first shallow feature and the first deep feature.

[0113] S230. The second remote sensing image is used to extract features by a shared weight dual-branch encoder to obtain the second shallow features and the second deep features.

[0114] S240. Combine the first shallow layer feature, the first deep layer feature, the second shallow layer feature, and the second deep layer feature into a multi-scale dual-temporal feature map.

[0115] By independently inputting the first and second remote sensing images into a shared-weight dual-branch encoder, it is ensured that features can be extracted independently from each temporal phase image, and the shared weights of the encoder are used to guarantee the consistency of feature representations across different temporal phases. This separate input mechanism is the foundation for maintaining the independence of temporal information and providing comparable features for subsequent change detection.

[0116] Subsequently, the shared-weight dual-branch encoder extracts features from the first remote sensing image, obtaining first shallow features and first deep features. The shared-weight dual-branch encoder processes the first remote sensing image to generate features at different levels of abstraction. The first shallow features typically capture low-level visual information such as local details, textures, and edges, and have relatively high spatial resolution. The first deep features, on the other hand, tend to encode higher-level semantic information and a wider receptive field, and typically have lower spatial resolution. Extracting multi-scale features is crucial for a comprehensive understanding of image content and capturing changes at different scales. Similarly, the shared-weight dual-branch encoder extracts features from the second remote sensing image, obtaining second shallow features and second deep features. The shared-weight dual-branch encoder performs the same feature extraction process on the second remote sensing image. Because the encoder uses shared weights, the second shallow features and second deep features extracted from the second remote sensing image are semantically comparable to those extracted from the first remote sensing image. This consistency is important for accurately comparing features from two different time periods and identifying genuine changes rather than differences in feature representations.

[0117] Finally, the first shallow feature, the first deep feature, the second shallow feature, and the second deep feature are combined to form the multi-scale dual-temporal feature map. This integrates features extracted from two temporal images at different levels. The combination methods may include, but are not limited to, concatenation along the channel dimension, element-wise summation, or other fusion strategies. By combining these features, a multi-scale dual-temporal feature map containing rich spatial details and high-level semantic information can be formed, simultaneously reflecting the states of two temporal phases.

[0118] The above technical solution clarifies the specific processing flow of the first and second remote sensing images in the shared-weight dual-branch encoder: they are input separately, shallow and deep features are extracted separately, and then combined. This structured feature extraction method ensures semantic consistency of features obtained from the two temporal images, facilitating accurate comparison later. Furthermore, extracting multi-scale shallow and deep features allows the model to simultaneously focus on local detail changes and global semantic changes in the image, thus providing comprehensive and high-quality input to the dual-temporal change information fusion module. This significantly improves the accuracy and robustness of change detection and effectively solves the problem of insufficiently refined feature extraction or failure to fully utilize dual-temporal information.

[0119] In practical applications, ensuring good consistency in feature extraction from images across different time phases while effectively capturing changes at different scales is a problem that requires further consideration. Inappropriate encoder structure and parameters may lead to unstable feature extraction or insensitivity to subtle changes.

[0120] Therefore, the shared weight dual-branch encoder is a twin feature extraction network that uses the same network structure and shares parameters; the first shallow feature and the second shallow feature have a first downsampling ratio relative to the first remote sensing image and the second remote sensing image; the first deep feature and the second deep feature have a second downsampling ratio relative to the first remote sensing image and the second remote sensing image; wherein, the second downsampling ratio is greater than the first downsampling ratio.

[0121] The shared-weight dual-branch encoder is configured as a Siamese feature extraction network with the same network structure and shared parameters. A Siamese feature extraction network is a deep learning network architecture whose core consists of two or more sub-networks that are structurally identical and share all network parameters (weights and biases). In remote sensing image change detection tasks, this design ensures that the same transformation function and feature space mapping are applied when extracting features from first and second remote sensing images from different time periods, effectively eliminating inconsistencies caused by differences in network structure or parameters. This makes the features extracted from images from different time periods more comparable and more accurately reflects the real changes between images, rather than biases introduced by the network itself. This Siamese network can be built using various convolutional neural network (CNN) architectures, such as ResNet, VGG, or U-Net, by synchronously updating the shared parameters of the two branches during training.

[0122] Furthermore, the first and second shallow features have a first downsampling ratio relative to the first and second remote sensing images, while the first and second deep features have a second downsampling ratio relative to the first and second remote sensing images. The second downsampling ratio is greater than the first downsampling ratio. This design aims to achieve effective extraction of multi-scale features. Shallow features are typically extracted in the first few layers of the network, possessing high spatial resolution and a small receptive field, preserving rich detail information in the image and exhibiting high sensitivity to local, subtle changes. Deep features are extracted in deeper layers of the network, undergoing more downsampling operations, thus possessing lower spatial resolution and a larger receptive field, capturing more abstract and global semantic information, and being more effective for large-scale, structural changes. The downsampling ratio is typically achieved by introducing pooling layers (such as max pooling or average pooling) or convolutional layers with a stride greater than 1 into the encoder network. For example, the first downsampling ratio can be 2x or 4x, while the second downsampling ratio can be 8x, 16x, or even higher. In this way, this application is able to acquire image features at different granularities to adapt to targets of varying scales that may exist in remote sensing images.

[0123] By configuring the shared-weight dual-branch encoder as a twin feature extraction network with the same network structure and shared parameters, this application ensures the use of a unified feature mapping mechanism when extracting features from first and second remote sensing images at different time phases. This effectively avoids feature inconsistencies caused by differences in network parameters, making the extracted first shallow features, first deep features, second shallow features, and second deep features more comparable and significantly improving the accuracy of change detection. Furthermore, by explicitly defining the first and second shallow features with a first downsampling factor, while the first and second deep features have a larger second downsampling factor, this application achieves effective extraction of multi-scale features. This allows shallow features to retain more detailed information and be sensitive to subtle local changes; while deep features can capture more abstract semantic information and are effective for large-scale structural changes. This multi-scale, consistent feature representation provides comprehensive and high-quality input for the subsequent dual-time-phase change information fusion module, enabling more comprehensive and accurate identification of various scale changes in remote sensing images, thus improving the robustness and accuracy of change detection.

[0124] While some image processing methods can acquire multi-scale, dual-temporal feature maps, effectively fusing these feature maps differentially to accurately identify changes in remote sensing images remains a challenge. Simple feature combinations may not adequately capture subtle changes and long-range dependencies at different scales, thus affecting the accuracy and robustness of change detection.

[0125] Therefore, referring to Figure 3 Another embodiment of the present invention provides an image processing method based on the above. Figure 1 The embodiment shown inputs multi-scale dual-temporal feature maps into a dual-temporal change information fusion module for difference fusion to obtain change fusion features. The dual-temporal change information fusion module incorporates a large convolutional kernel state space module, including steps S310-S370, wherein:

[0126] S310. Based on the scale of the multi-scale dual-temporal feature map, divide the multi-scale dual-temporal feature map into at least two scale-integrated feature pairs, each scale-integrated feature pair including a first-temporal feature and a second-temporal feature.

[0127] S320. For each feature pair at the same scale, the first phase feature and the second phase feature are concatenated along the channel dimension to obtain the concatenated feature.

[0128] S330. Normalize the spliced ​​features and process them using a multilayer perceptron to obtain intermediate features;

[0129] S340. Input the intermediate features into the large convolutional kernel state space module to perform spatial modeling and obtain deep aggregated features;

[0130] S350. Perform convolutional channel compression on the spliced ​​features to obtain the basic convolutional features;

[0131] S360: Fuse deep aggregated features with basic convolutional features weighted by learnable coefficients, and obtain the corresponding scale-based fused features through convolution processing;

[0132] S370. Combine the change fusion features corresponding to each scale into a change fusion feature.

[0133] In processing multi-scale dual-temporal feature maps, they are first meticulously divided into feature pairs of at least two scales based on their inherent scale information. Each feature pair contains first-temporal features and second-temporal features from the same spatial resolution. This division ensures that subsequent fusion operations are performed between features with comparable spatial information, avoiding information distortion or loss of detail that may result from scale mismatch, thus laying the foundation for accurate change detection.

[0134] For each feature pair at the same scale, the features from the first and second time phases are concatenated along the channel dimension. This operation integrates feature information from different time phases but at the same scale into a unified feature representation, thus providing a rich input for subsequent difference learning and feature extraction without losing the original time phase information. Through channel concatenation, the model can simultaneously access and compare features from two time phases, thereby better capturing the differences between them.

[0135] Subsequently, the concatenated features are normalized and processed by a multilayer perceptron to obtain intermediate features. Normalization helps stabilize the training process, accelerates model convergence, and enhances the model's generalization ability. The multilayer perceptron (MLP) then uses nonlinear transformations to map the concatenated original features to a higher-dimensional, more discriminative feature space, thereby extracting more abstract and richer differential information and providing high-quality input for subsequent spatial modeling.

[0136] Intermediate features are input into a large convolutional kernel state space module for spatial modeling, resulting in deep aggregated features. This module effectively captures local details and long-range dependencies in the feature maps. Spatial modeling using this module allows for global context awareness of intermediate features, generating deep aggregated features rich in spatial context information. This is crucial for identifying variations across large areas or with complex spatial patterns in remote sensing images.

[0137] Furthermore, convolutional channel compression is performed on the concatenated features to obtain basic convolutional features, typically implemented using 1x1 convolutions. This aims to extract a more compact, low-dimensional feature representation from the concatenated features. These basic convolutional features retain the core local information of the concatenated features, serving as a supplementary path for fusion with deep aggregated features, ensuring that important local details are not lost while capturing the global context.

[0138] Next, the deep aggregated features are fused with the base convolutional features weighted by learnable coefficients, and the resulting scale-specific fused features are obtained through convolution processing. By introducing learnable coefficients to weight the base convolutional features, the model can adaptively adjust the importance of local details in the fusion process. This fusion strategy cleverly combines the global contextual information provided by the large convolutional kernel state space module with the local details provided by convolutional channel compression, and through further convolutional processing, generates fused features that comprehensively encode the changes at the current scale.

[0139] Finally, the variation fusion features corresponding to each scale are combined into a variation fusion feature. Since the above fusion process is performed independently for feature pairs at different scales, it is necessary to integrate the variation fusion features generated at all scales. This combination can be achieved through various methods such as concatenation, summation, or pooling, aiming to form a comprehensive, multi-scale variation fusion feature, providing a unified and rich input for the subsequent feature recovery and aggregation modules.

[0140] Through the above technical solutions, this embodiment can systematically process multi-scale dual-temporal feature maps and effectively perform difference fusion. By grouping features by scale and stitching channels together, the integrity of temporal information is ensured. Normalization and multilayer perceptron processing enhance the discriminative ability of the features. Most importantly, the introduction of a large convolutional kernel state space module for spatial modeling enables the model to simultaneously capture subtle local changes and long-distance dependencies in remote sensing images, overcoming the limitation of the limited receptive field of traditional convolutions. Furthermore, by generating basic convolutional features through parallel paths and adaptively weighting and fusing them with deep aggregated features, it ensures that while acquiring global contextual information, it also fully utilizes local detailed information. This multi-path, multi-scale fusion strategy allows the final change fusion features to more comprehensively and accurately represent changes in remote sensing images, significantly improving the accuracy and robustness of change detection, especially when dealing with complex and variable terrain scenes, effectively distinguishing real changes from noise or irrelevant disturbances.

[0141] If the subsequent large receptive field feature aggregation module fails to fully utilize the inherent multi-scale information of the change fusion features during feature restoration and aggregation, or fails to uniformly process features at different scales, it may result in the loss of fine spatial details during the restoration process, or difficulty in effectively aggregating change information at different scales, thereby affecting the accuracy and robustness of the final change detection results.

[0142] Optionally, refer to Figure 4 In another embodiment of the present invention, an image processing method is provided, based on the above. Figure 1 The embodiment shown includes a first change fusion feature and a second change fusion feature, wherein the downsampling factor corresponding to the second change fusion feature is greater than the downsampling factor corresponding to the first change fusion feature; the change fusion feature is input into a large receptive field feature aggregation module for feature recovery and aggregation to obtain the recovered feature, including steps S410-S420, wherein:

[0143] S410. Input the second change fusion feature into the large receptive field feature aggregation module for processing and upsampling to obtain the second restored feature;

[0144] S420. The first change fusion feature and the second recovery feature are fused together. The fused feature is input into the large receptive field feature aggregation module for processing and upsampling to obtain the first recovery feature. The first recovery feature is used as the recovery feature.

[0145] Optionally, refer to Figure 5 The second change fusion feature is input into the large receptive field feature aggregation module for processing, including steps S411-S414, wherein:

[0146] S411. Perform convolution, normalization, and activation processing on the second change fusion feature input to the large receptive field feature aggregation module to obtain compressed features;

[0147] S412. Perform average pooling and linear transformation on the compressed features to obtain the channel weights;

[0148] S413. Input the compressed features into the large convolutional kernel state space module to obtain context-enhanced features;

[0149] S414. Multiply the context enhancement features element-wise with the channel weights and then perform convolution to obtain the aggregated output features.

[0150] In this process, the second transformation fusion feature input to the large receptive field feature aggregation module is subjected to convolution, normalization, and activation to obtain compressed features. This provides an initial compression and nonlinear transformation of the input features, extracting more discriminative feature representations. Next, average pooling and linear transformation are performed on the compressed features to obtain channel weights. Average pooling aggregates information in the spatial dimension, and then linear transformation generates channel weights, allowing the network to dynamically learn the importance of each channel. This allows for different levels of attention to different channels in subsequent processing, enhancing the expressive power of key channels. Furthermore, the compressed features are input into the large convolutional kernel state space module to obtain context-enhanced features. The large convolutional kernel state space module aims to capture global contextual information of features through its unique mechanism, generating context-enhanced features, which is crucial for understanding large-scale change patterns or long-distance dependencies in remote sensing images. Finally, the context-enhanced features are multiplied element-wise with the channel weights and then convolutional to obtain the aggregated output features. This aggregation method, combining global context and channel attention, results in output features that contain rich spatial information and possess strong semantic expressive power.

[0151] Through the above technical solution, this embodiment effectively solves the problem of how to efficiently and accurately perform feature recovery and aggregation after multi-scale variation information fusion. Specifically, by decomposing the variation fusion features into features of different scales and employing a coarse-to-fine hierarchical processing strategy, the lower-resolution second variation fusion feature is first processed and upsampled, then fused with the higher-resolution first variation fusion feature, and processed and upsampled again, thereby achieving effective integration and refined recovery of multi-scale variation information. This hierarchical aggregation method fully utilizes the advantages of features at different scales, preserving the detailed information of high-resolution features while incorporating the global contextual information of low-resolution features. Furthermore, the channel weight generation mechanism introduced in the feature aggregation process enables the network to adaptively focus on important feature channels, improving the expressive power of the features; while the use of large convolutional kernel state space modules further enhances the contextual awareness and long-range dependency modeling capabilities of the features. The synergistic effect of these mechanisms significantly improves the spatial accuracy and semantic richness of the final restored features, providing higher-quality input for subsequent convolutional classification. This significantly enhances the accuracy and robustness of remote sensing image change detection, especially when dealing with complex and varied land cover types and change patterns, enabling more effective identification of subtle or large-scale change areas. This coarse-to-fine hierarchical processing strategy effectively combines macroscopic contextual information with local fine details, avoiding the computational burden of directly processing high-resolution features while ensuring the restoration of details.

[0152] In remote sensing image change detection tasks, due to factors such as the complexity of ground features, scale diversity, and background noise, traditional convolutional classification may be unable to fully capture pixel-level contextual information, leading to blurred boundaries or misjudgments of changed areas, thus affecting the accuracy and robustness of the final change detection result map.

[0153] Optionally, refer to Figure 6 The present invention also provides an image processing method based on the above. Figure 1 The embodiment shown performs convolutional classification on the recovered features to generate a change detection result map, including steps S510-S520, wherein:

[0154] S510. Perform large-size convolution processing on the restored features to map the restored features into pixel-level change probability maps;

[0155] S520. Generate a change detection result map based on the change probability map;

[0156] Among them, the kernel size for large-size convolution processing is N*N.

[0157] The process involves performing large-size convolutions on the recovered features to conduct in-depth semantic analysis on the features after multi-stage feature extraction, fusion, and aggregation. Large-size convolutions, such as using a 7x7 kernel size, effectively expand the receptive field of the kernel, allowing each output pixel to consider a wider input feature region during computation. This is particularly important for change detection in remote sensing images, as changes in ground features often involve regional contextual information rather than isolated pixels. Through large-size convolutions, the model can better understand spatial relationships and semantic information within local regions, thus providing richer contextual support for subsequent pixel-level classification. After performing large-size convolutions, the resulting feature map is further mapped to a pixel-level change probability map. This means that for each pixel in the image, a probability value between 0 and 1 is output, representing the confidence level that the pixel has changed. This mapping is typically achieved by adding an activation function (such as the sigmoid function) after the convolutional layer, converting the output of the convolutional layer into a probability distribution. The pixel-level probability map provides a refined foundation for generating subsequent change detection results, allowing for flexible decisions based on probability thresholds. After obtaining the pixel-level change probability map, it needs to be converted into a final binary or multi-value change detection result map. This is usually achieved by setting a predefined threshold. For example, pixels with a probability value higher than a certain threshold (such as 0.5) can be marked as "changed," while pixels with a probability value lower than the threshold can be marked as "unchanged." Furthermore, post-processing techniques, such as morphological operations (erosion, dilation) or connected component analysis, can be combined to eliminate isolated noise points or smooth the boundaries of change regions, thereby generating a more accurate and regular change detection result map. Large-size convolution processing with a kernel size of N*N (7*7) is a specific example of "large-size convolution processing." Choosing a 7x7 kernel size significantly increases the receptive field of the convolutional layer compared to commonly used 3x3 or 1x1 kernels. In remote sensing image processing, a larger receptive field helps capture richer spatial contextual information, such as change patterns in larger areas like land parcels, building clusters, or bodies of water. This allows the model to comprehensively consider information from a wider area around a single pixel when determining whether it has changed, thereby effectively reducing false positives and improving the accuracy and robustness of change detection.

[0158] By employing the aforementioned technical solution, large-size convolution processing (e.g., a 7x7 kernel size) is used for the final classification of the restored features. This significantly expands the receptive field, allowing the model to fully utilize broader contextual information when determining whether each pixel has changed. This deep mining of spatial contextual information helps overcome challenges such as complex features and blurred boundaries in remote sensing images, thereby mapping the restored features more accurately to pixel-level change probability maps. Ultimately, the change detection result map generated based on the high-precision change probability map can more accurately identify changed areas, reduce false positives and false negatives, and significantly improve the accuracy and reliability of change detection in remote sensing images.

[0159] The complexity and diversity of remote sensing images make it difficult to achieve optimal change detection performance with preset parameters or fixed models, which may result in insufficient detection accuracy or limited generalization ability, making it difficult to adapt to change detection needs in different scenarios.

[0160] Optionally, refer to Figure 7 Another embodiment of the present invention provides an image processing method based on the above. Figure 1 The illustrated embodiment further includes a training process, comprising steps S600-S900, wherein:

[0161] S600: Obtain a training sample set carrying pixel-level change labels;

[0162] S700. The initial image processing network is iteratively trained using the training sample set. In each iteration, the training samples are input into the image processing network to obtain the predicted change detection result map.

[0163] S800: Based on the predicted change detection result map and pixel-level change labels, calculate the value of the joint loss function;

[0164] S900. Update the network parameters of the image processing network through backpropagation based on the value of the joint loss function.

[0165] The joint loss function is a combination of the cross-entropy loss function and the normalized exponential loss function based on the Lovarz extension.

[0166] The training sample set typically consists of multiple pairs of remote sensing images of the same region acquired at different times (i.e., the first and second remote sensing images) and pixel-level change labels corresponding to each pair of images. The pixel-level change label is a binary mask that precisely indicates whether each pixel in the image has changed, providing a realistic supervision signal for model learning.

[0167] Secondly, the initial image processing network is iteratively trained using the training sample set. In each iteration, the training samples are input into the image processing network for forward propagation, thereby obtaining the predicted change detection result map. The predicted change detection result map is the output of the network based on the current parameters when performing change detection on the input image pair.

[0168] Next, based on the predicted change detection result map and pixel-level change labels, the value of the joint loss function is calculated. The joint loss function is used to quantify the difference between the model's prediction results and the true labels, and its value directly reflects the current performance of the model.

[0169] Finally, based on the value of the joint loss function, the network parameters of the image processing network are updated through backpropagation. Backpropagation is an optimization algorithm that iteratively adjusts the network weights and biases based on the gradient information of the loss function with respect to the network parameters, gradually reducing the value of the loss function and thus improving the accuracy of the model.

[0170] It is worth noting that the joint loss function is a combination of the cross-entropy loss function and the Lovars-extended normalized exponential loss function. The cross-entropy loss function performs well in binary classification tasks, effectively measuring the difference between the predicted probability distribution and the true label distribution, prompting the model to learn the correct classification boundary. The Lovars-extended normalized exponential loss function, on the other hand, is a loss function optimized for the Jaccard exponent (IoU), particularly suitable for pixel-level classification tasks such as image segmentation and change detection. It is especially effective in handling class imbalance or small object variation regions, better optimizing the consistency between the detection boundary and the overall region. The combination of these two loss functions leverages their respective advantages, providing the model with a more comprehensive and robust optimization objective.

[0171] Through the training process described above, the image processing method in this embodiment can learn and optimize its internal parameters from a large amount of remote sensing image data, thereby overcoming the limitations of fixed-parameter models when processing complex and variable remote sensing images. Specifically, iterative training is performed using a training sample set carrying pixel-level change labels, and the backpropagation update of network parameters is guided by a joint loss function (especially a combination of the cross-entropy loss function and the Lovarz extension-based normalized exponential loss function). This enables the network to more accurately capture subtle changes in remote sensing images, improving the accuracy and robustness of change detection. This training mechanism ensures that the model can adapt to change detection tasks under different geographical environments and temporal conditions, significantly improving the model's generalization ability and practical application performance, avoiding false detections or missed detections due to inappropriate parameters, and thus obtaining high-quality change detection result images.

[0172] While traditional convolutional operations excel at extracting local features, they are less efficient at modeling long-range dependencies, and simply increasing the kernel size significantly increases the computational burden. Furthermore, some long-range dependency modeling methods may be inadequate in capturing local details or have high computational complexity, which is particularly pronounced when processing high-resolution remote sensing images, potentially limiting the accuracy of variation information fusion and feature aggregation.

[0173] Optionally, refer to Figure 8 Another embodiment of the present invention provides an image processing method based on the above. Figure 1 The illustrated embodiment includes a large convolutional kernel state space module comprising a reparameterized large convolutional branch and a two-dimensional state space scanning branch; the large convolutional kernel state space module is used to process the module input features to obtain the module output features, and the processing includes steps S530-S550, wherein:

[0174] S530: Input the module input features into a reparameterized large convolution branch, and extract multi-scale local spatial features through multiple depthwise separable convolutions with different dilation rates;

[0175] S540. Input the multi-scale local spatial features into the two-dimensional state space scanning branch, and perform long-range dependency modeling on the multi-scale local spatial features through the four-way selective scanning mechanism to obtain long-range correlation features.

[0176] S550: Perform convolution, normalization, and activation processing on long-range correlated features, and fuse them with residuals to obtain the module output features;

[0177] During the training phase, the reparameterized large convolutional branch extracts multi-scale receptive field features in parallel through multiple branches. During the inference phase, the parameters of multiple depthwise separable convolutions with different dilation rates are folded into a single equivalent large convolutional kernel. The equivalent receptive field size of the equivalent large convolutional kernel is equal to the difference between the kernel size and the dilation rate, plus one.

[0178] The large convolutional kernel state space module is designed to efficiently process the input feature map to generate output features containing rich contextual information. Its core function is to simultaneously capture local details and global dependencies in the image, providing high-quality feature representations for subsequent change information fusion and feature aggregation.

[0179] The reparameterized large convolution branch is primarily responsible for extracting multi-scale local spatial features from the module input features. This is achieved by using multiple depthwise separable convolutions with different dilation rates. Depthwise separable convolutions significantly reduce computational complexity and the number of model parameters by decomposing standard convolutions into depthwise and pointwise convolutions, while maintaining feature extraction capabilities. The introduction of different dilation rates allows the convolutional kernel to perceive local region information at different scales without increasing parameters or reducing resolution, effectively capturing a variety of local features from fine textures to large local structures. During training, the reparameterized large convolution branch extracts multi-scale receptive field features through multiple parallel branches. Each branch may correspond to a depthwise separable convolution with a specific dilation rate; this parallel structure helps the network learn richer and more robust local feature representations. To improve inference efficiency, after model training is complete, the reparameterized large convolution branch folds (or fuses) the parameters of the multiple depthwise separable convolutions with different dilation rates from the training phase into a single equivalent large convolutional kernel. This parameter folding technique reduces computational load and memory usage during inference without altering model functionality, enabling faster feature extraction in practical applications. The equivalent receptive field size of the equivalent large convolutional kernel can be calculated using the formula: the difference between the kernel size and the dilation rate multiplied by one, plus one. This ensures that the folded convolutional kernel still covers the broad receptive field learned during training.

[0180] The core function of the two-dimensional state-space scanning branch is to model long-range dependencies of multi-scale local spatial features extracted by the heavily parameterized large convolutional branch. It employs a four-directional selective scanning mechanism, enabling it to scan the feature map from multiple directions (e.g., left-to-right, right-to-left, top-to-bottom, bottom-to-top), thereby effectively capturing the long-range correlations between any two pixels in the image. This mechanism overcomes the limitations of the local receptive field in traditional convolution, allowing the module to understand the global context of the image and generate features with long-range correlations.

[0181] After obtaining long-range relevance features, these features undergo convolution, normalization, and activation processing, and are then residually fused with the original module input features. Residual fusion is a commonly used connection method that allows the network to retain the original input information while learning new feature representations. This helps alleviate the vanishing gradient problem and promotes the training stability of the network, enabling the module output features to contain both local details and global contextual information.

[0182] Through the above technical solutions, the large convolutional kernel state space module proposed in this embodiment can effectively solve the challenge of simultaneously and efficiently capturing local fine information and global long-range dependencies in remote sensing image change detection. Specifically, the reparameterized large convolutional branch, through its multi-scale, depth-separable convolutional design, can accurately capture local details and multi-scale local context of the image with low computational cost, and further improves efficiency through parameter reparameterization during the inference stage. Meanwhile, the two-dimensional state space scanning branch focuses on efficiently modeling long-range dependencies in the image using a four-way selective scanning mechanism, thereby obtaining global contextual information. This organic combination of local feature extraction and long-range dependency modeling, integrated through a residual fusion mechanism, ensures that the module's output features contain both rich local details and comprehensive global perception capabilities. Therefore, when these enhanced features are input into the dual-temporal change information fusion module and the large receptive field feature aggregation module, the recognition accuracy of subtle and large-scale changes in remote sensing images can be significantly improved, generating more accurate and robust change detection result maps, effectively avoiding the false detection and missed detection problems that may occur in traditional methods when dealing with complex remote sensing scenes.

[0183] The present invention also proposes a processing apparatus, which includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor, the image processing program being configured to implement the image processing method described above.

[0184] It is worth noting that since the processing device of the present invention is based on the above-described image processing method, the embodiments of the processing device of the present invention include all the technical solutions of all embodiments of the above-described image processing method, and the technical effects achieved are exactly the same, so they will not be repeated here.

[0185] The present invention also proposes a computer program product, which includes a processing device as described in the above embodiments.

[0186] It is worth noting that, since the computer program product of the present invention is based on the above-described processing device, the embodiments of the computer program product of the present invention include all the technical solutions of all embodiments of the above-described processing device, and the technical effects achieved are exactly the same, which will not be repeated here.

[0187] The technical solution of this embodiment is described below with reference to a specific example:

[0188] Suppose user A needs to monitor urban development at location A to identify land use changes occurring over different time periods, such as new buildings, road expansion, or green space reduction. Traditional methods often face challenges in capturing large-scale land feature changes (such as the emergence of large building complexes) or consuming enormous computational resources when processing high-resolution remote sensing images. This solution aims to address these issues, effectively capturing long-distance spatial dependencies and preserving pixel-level local details while maintaining computational efficiency.

[0189] First, the image processing method acquires a first remote sensing image and a second remote sensing image of the same area taken at different times. For example, it acquires a first remote sensing image of location A taken in 2020 and a second remote sensing image taken in 2022. These two images have the same spatial extent and resolution, but reflect the surface conditions at different points in time.

[0190] Subsequently, the first and second remote sensing images are input into a shared-weight dual-branch encoder to extract multi-scale, dual-temporal feature maps. This encoder employs a Siamese feature extraction network with the same network structure and shared parameters. The first and second remote sensing images are input into this encoder respectively. The encoder extracts features from the first remote sensing image, obtaining first shallow features and first deep features; simultaneously, it extracts features from the second remote sensing image, obtaining second shallow features and second deep features. The first and second shallow features have a smaller downsampling ratio compared to the original image, preserving more spatial details; while the first and second deep features have a larger downsampling ratio, capturing more abstract semantic information. This multi-scale feature extraction method can comprehensively capture variation information at different granularities. Compared to traditional single-branch encoders, the shared-weight design ensures consistency in feature extraction from dual-temporal images, reduces the number of model parameters, and improves training efficiency.

[0191] Next, the multi-scale dual-temporal feature map is input into the dual-temporal change information fusion module for difference fusion to obtain change fusion features. This fusion module has a built-in large convolutional kernel state space module. The specific processing is as follows: based on the scale of the multi-scale dual-temporal feature map, it is divided into feature pairs of at least two scales, each feature pair including a first-temporal feature and a second-temporal feature. For each feature pair of the same scale, the first-temporal feature and the second-temporal feature are concatenated in the channel dimension to integrate the dual-temporal information and obtain the concatenated feature. The concatenated feature is normalized and processed by a multilayer perceptron to obtain intermediate features. Subsequently, the intermediate features are input into the built-in large convolutional kernel state space module for spatial modeling to obtain deep aggregated features. This large convolutional kernel state space module is one of the core innovations of this scheme. It extracts multi-scale local spatial features through reparameterized large convolutional branches, and at the same time, it performs long-range dependency modeling on these local features through a two-dimensional state space scanning branch to obtain long-range correlated features. Unlike traditional convolutional neural networks (CNNs) which are limited by a fixed and narrow receptive field, large convolutional kernel branches can effectively expand the receptive field and capture a wider range of local contextual information. Simultaneously, compared to the quadratic computational complexity of the Transformer's self-attention mechanism, the two-dimensional state-space scanning branch achieves global correlation with linear computational complexity, avoiding resource bottlenecks when processing high-resolution remote sensing images. This solves the problems of narrow receptive fields in CNNs and high computational costs in Transformers in existing technologies. After obtaining deep aggregated features, convolutional channel compression is performed on the original stitched features to obtain basic convolutional features. Finally, the deep aggregated features are fused with the basic convolutional features weighted by learnable coefficients, and the corresponding scale-specific variation fusion features are obtained through convolution processing. The variation fusion features corresponding to each scale are combined into the final variation fusion feature. This fusion mechanism ensures that it can utilize the global context and long-range dependencies captured by the large convolutional kernel state-space module while preserving the local details in the basic convolutional features.

[0192] Then, the variation fusion features are input into the large receptive field feature aggregation module for feature recovery and aggregation to obtain the recovered features. This aggregation module also includes a large convolutional kernel state space module. The variation fusion features include a first variation fusion feature and a second variation fusion feature, where the downsampling ratio of the second variation fusion feature is greater than that of the first variation fusion feature. First, the second variation fusion feature is input into the large receptive field feature aggregation module for processing and upsampling to obtain the second recovered feature. The specific processing includes performing convolution, normalization, and activation processing on the second variation fusion feature input to this module to obtain compressed features; performing average pooling and linear transformation on the compressed features to obtain channel weights; inputting the compressed features into the large convolutional kernel state space module to obtain context enhancement features; and performing convolution processing after element-wise multiplication of the context enhancement features with the channel weights to obtain the aggregated output features. This processing method further enhances the contextual information of the features using the large convolutional kernel state space module. Subsequently, the first variation fusion feature and the second recovered feature are fused, and the fused feature is input into the large receptive field feature aggregation module again for processing and upsampling to obtain the first recovered feature, which is used as the final recovered feature. This step-by-step recovery and aggregation process, from deep to shallow, combines multi-scale information and utilizes a large convolutional kernel state space module for context enhancement at each stage. This ensures that while recovering high-resolution details, it fully utilizes the global change information captured by the previous fusion module, avoiding the loss of details or artifacts that may be caused by pure upsampling. It also solves the problem that simple global aggregation of state space models can easily sacrifice pixel-level local spatial details.

[0193] Finally, convolutional classification is performed on the recovered features to generate a change detection result map. Specifically, large-size convolution processing, such as using a 7x7 convolution kernel, is applied to the recovered features to map them into a pixel-level change probability map. Large-size convolution kernels can further integrate local contextual information, generating a smoother and more accurate pixel-level change probability map, reducing false positives and false negatives. Based on the change probability map, by setting thresholds or post-processing, the final change detection result map is generated, clearly indicating the areas where location A changed between 2020 and 2022.

[0194] To ensure the effectiveness of this method, optimization is required through the training process. The initial image processing network is iteratively trained using a training sample set carrying pixel-level change labels. In each iteration, the training samples are input into the image processing network to obtain a predicted change detection map. Based on the predicted change detection map and the pixel-level change labels, the value of a joint loss function is calculated, such as a combination of the cross-entropy loss function and a normalized exponential loss function based on Lowaz extension. This loss function design helps balance the contributions of pixels from different categories, exhibiting robustness, especially in handling class-imbalanced remote sensing change detection tasks. Based on the value of the joint loss function, the network parameters of the image processing network are updated through backpropagation, enabling the network to more accurately identify surface changes.

[0195] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An image processing method applied to remote sensing image processing, characterized in that, The image processing method includes: Acquire the first and second remote sensing images of the same area acquired at different times; The first remote sensing image and the second remote sensing image are input into a shared weight dual-branch encoder to extract multi-scale dual-temporal feature maps; The multi-scale dual-temporal feature map is input into the dual-temporal change information fusion module for difference fusion to obtain the change fusion feature, wherein the dual-temporal change information fusion module has a built-in large convolutional kernel state space module. The change fusion features are input into the large receptive field feature aggregation module for feature recovery and aggregation to obtain the recovered features, wherein the large receptive field feature aggregation module includes the large convolutional kernel state space module. Perform convolutional classification on the recovered features to generate a change detection result map.

2. The image processing method as described in claim 1, characterized in that, The step of inputting the first remote sensing image and the second remote sensing image into a shared weighted dual-branch encoder to extract multi-scale dual-temporal feature maps includes: The first remote sensing image and the second remote sensing image are respectively input into the shared weight dual-branch encoder; The first remote sensing image is used to extract features through the shared weight dual-branch encoder to obtain first shallow features and first deep features; The second remote sensing image is used to extract features through the shared weight dual-branch encoder to obtain second shallow features and second deep features; The first shallow feature, the first deep feature, the second shallow feature, and the second deep feature are combined to form the multi-scale dual-temporal feature map.

3. The image processing method as described in claim 2, characterized in that, The shared weight dual-branch encoder is a twin feature extraction network that uses the same network structure and shares parameters; The first shallow feature and the second shallow feature have a first downsampling ratio relative to the first remote sensing image and the second remote sensing image; the first deep feature and the second deep feature have a second downsampling ratio relative to the first remote sensing image and the second remote sensing image; wherein the second downsampling ratio is greater than the first downsampling ratio.

4. The image processing method as described in claim 1, characterized in that, The multi-scale dual-temporal feature map is input into the dual-temporal change information fusion module for difference fusion to obtain change fusion features. The dual-temporal change information fusion module includes a built-in large convolutional kernel state space module, comprising: Based on the scale of the multi-scale dual-temporal feature map, the multi-scale dual-temporal feature map is divided into at least two scale-integrated feature pairs, each scale-integrated feature pair including a first temporal feature and a second temporal feature; For each of the same-scale feature pairs, the first temporal feature and the second temporal feature are concatenated along the channel dimension to obtain the concatenated feature; The spliced ​​features are normalized and processed by a multilayer perceptron to obtain intermediate features; The intermediate features are input into the large convolutional kernel state space module for spatial modeling to obtain deep aggregated features; Perform convolutional channel compression on the spliced ​​features to obtain the basic convolutional features; The deep aggregated features are fused with the basic convolutional features weighted by learnable coefficients, and the corresponding scale-dependent fused features are obtained through convolution processing. The change fusion features corresponding to each scale are combined into the change fusion features.

5. The image processing method as described in claim 1, characterized in that, The change fusion feature includes a first change fusion feature and a second change fusion feature, wherein the downsampling factor corresponding to the second change fusion feature is greater than the downsampling factor corresponding to the first change fusion feature; The step of inputting the fused feature into the large receptive field feature aggregation module for feature recovery and aggregation to obtain the recovered features includes: The second change fusion feature is input into the large receptive field feature aggregation module for processing and upsampling to obtain the second recovery feature; The first change fusion feature is fused with the second recovery feature, the fused feature is input into the large receptive field feature aggregation module for processing, and upsampling is performed to obtain the first recovery feature, which is then used as the recovery feature. The second change fusion feature is input into the large receptive field feature aggregation module for processing, including: The second variation fusion feature input to the large receptive field feature aggregation module is subjected to convolution, normalization, and activation processing to obtain compressed features; The compressed features are subjected to average pooling and linear transformation to obtain channel weights; The compressed features are input into the large convolutional kernel state space module to obtain context-enhanced features; The context enhancement features are multiplied element-wise with the channel weights and then convolution is performed to obtain the aggregated output features.

6. The image processing method as described in claim 1, characterized in that, The step of performing convolutional classification on the recovered features to generate a change detection result map includes: Perform large-size convolution processing on the restored features to map the restored features into a pixel-level probability map of change. The change detection result map is generated based on the change probability map; The kernel size for the large-size convolution processing is N*N.

7. The image processing method as described in claim 1, characterized in that, The image processing method further includes a training process, which includes: Obtain a training sample set carrying pixel-level change labels; The initial image processing network is iteratively trained using the training sample set. In each iteration, the training samples are input into the image processing network to obtain the predicted change detection result map. Based on the predicted change detection result map and the pixel-level change label, calculate the value of the joint loss function; Based on the value of the joint loss function, the network parameters of the image processing network are updated through backpropagation; The joint loss function is a combination of the cross-entropy loss function and the normalized exponential loss function based on the Lovarz extension.

8. The image processing method as described in claim 1, characterized in that, The large convolution kernel state space module includes a reparameterized large convolution branch and a two-dimensional state space scanning branch; The large convolutional kernel state space module is used to process the module input features to obtain the module output features. The processing includes: The module input features are input into the reparameterized large convolution branch, and multi-scale local spatial features are extracted through multiple depthwise separable convolutions with different dilation rates; The multi-scale local spatial features are input into the two-dimensional state space scanning branch, and the long-range dependency modeling of the multi-scale local spatial features is performed through a four-way selective scanning mechanism to obtain long-distance correlation features. The long-range correlation features are subjected to convolution, normalization, and activation processing, and residual fusion is performed with the module input features to obtain the module output features; During the training phase, the reparameterized large convolutional branch extracts multi-scale receptive field features in parallel through multiple branches. During the inference phase, the parameters of the multiple depthwise separable convolutions with different dilation rates are folded into a single equivalent large convolutional kernel. The equivalent receptive field size of the equivalent large convolutional kernel is equal to the difference between the kernel size and the dilation rate, multiplied by one.

9. A processing apparatus, characterized in that, The processing apparatus includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor, the image processing program being configured to implement the image processing method as described in any one of claims 1 to 8.

10. A computer program product, characterized in that, Includes the processing apparatus as described in claim 9.

Images