SAR image change detection method based on saliency detection and channel enhanced attention network

Abstract

The application discloses a SAR image change detection method based on saliency detection and channel enhanced attention network. Firstly, two difference maps are generated through a logarithmic ratio operator and saliency detection, and an enhanced saliency fusion difference map is obtained through multiplication fusion; then a clustering algorithm is used for pre-classification, and training samples and test samples are selected based on the classification results; finally, a CEAN network containing an EMA module and an ASPP module is constructed for feature learning and final classification. The application enhances the change region features through saliency detection, realizes accurate feature focusing through the EMA module, captures multi-scale context information through the ASPP module, effectively improves the accuracy and robustness of SAR image change detection, and is particularly suitable for change detection tasks in complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of SAR image change detection technology, specifically relating to a SAR image change detection method based on saliency detection and channel-enhanced attention network. This invention can be widely applied in computer vision and image processing fields such as remote sensing image analysis, environmental monitoring, and disaster assessment. Background Technology

[0002] Synthetic Aperture Radar (SAR) image change detection is an important technique for identifying surface changes by analyzing SAR images of the same area acquired at different times. However, due to the inherent multiplicative speckle noise in SAR images, as well as the complexity and multi-scale characteristics of the changed areas, traditional change detection methods often fail to achieve ideal results.

[0003] Existing SAR image change detection methods mainly face the following challenges: 1) Insufficient feature enhancement for significantly changed areas during the difference map generation process; 2) Limitations of traditional convolutional neural networks in capturing multi-scale contextual information; 3) Insufficient precision in the model's focus on the spatial features of changed areas; 4) Insufficient ability to preserve change edges and restore details in complex scenes. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a SAR image change detection method based on saliency detection and channel-enhanced attention network. By enhancing the features of the changed region through saliency detection and using the channel-enhanced attention network to achieve accurate multi-scale feature extraction, the accuracy and robustness of change detection are improved.

[0005] The present invention adopts the following technical solution:

[0006] A SAR image change detection method based on saliency detection and channel-enhanced attention network includes the following steps:

[0007] (1) Generate a log-ratio difference plot:

[0008] Input two dual-temporal SAR images and generate a log-ratio difference map using the log-ratio operator;

[0009] (2) Generate a significant difference plot:

[0010] Perform significance testing on the log-ratio difference plot to generate a significant difference plot;

[0011] (3) Generate a significant fusion difference map:

[0012] The logarithmic ratio difference plot and the significance difference plot are multiplied and fused to obtain the significance fused difference plot;

[0013] (4) Pre-classify the significant fusion difference map:

[0014] Clustering algorithms are used to pre-classify the saliency fusion difference map to obtain initial changed classes, unchanged classes, and uncertain classes;

[0015] (5) Select training samples and test samples:

[0016] From the original SAR image, neighborhoods around pixels belonging to the changed and unchanged classes are selected as training samples; neighborhoods around pixels belonging to the uncertain class are selected as test samples.

[0017] (6) Constructing the CEAN network model:

[0018] The CEAN network includes an EMA module and an ASPP module, wherein:

[0019] The EMA module precisely focuses on areas of change through a dynamic channel-spatial weighting mechanism;

[0020] The ASPP module captures multi-scale contextual information through multi-scale dilated convolution and global pooling.

[0021] (7) Train the CEAN network and perform final classification:

[0022] The CEAN network is trained using training samples. After training, the test samples are classified to obtain the final change detection results.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] First, this invention effectively enhances the feature representation of changing regions and suppresses the interference of background noise by fusing saliency detection with multiplication.

[0025] Secondly, the CEAN network used implements dynamic channel-spatial attention weighting through the EMA module, which can accurately focus on changing regions and improve the accuracy of feature selection.

[0026] Third, the ASPP module expands the receptive field without increasing the number of parameters by setting different porosity rates, and captures contextual information from local to global in parallel, effectively solving the problem of multi-scale change detection.

[0027] Fourth, this invention effectively solves the problems of insufficient training samples and class imbalance by using pre-classification and sample selection strategies, thereby improving the generalization ability of the model. Attached Figure Description

[0028] Figure 1 This is an overall flowchart of the present invention;

[0029] Figure 2 This is a diagram of the CEAN network structure in this invention;

[0030] Figure 3 This is a diagram of the ConvNext Block network structure in this invention;

[0031] Figure 4 This is a structural diagram of the EMA module in this invention;

[0032] Figure 5 This is a structural diagram of the ASPP module in this invention;

[0033] Figure 6 This is the Bern dataset in this embodiment of the invention, where (a) and (b) are two SAR images of the same area in Bern at different times, (c) is a reference map of the actual changes in Bern, and (d) is a map of the change detection effect of this invention.

[0034] Figure 7 This is the Farmland dataset in this embodiment of the invention, where (a) and (b) are two SAR images of the same area of ​​the farm at different times, (c) is a reference image of the actual changes in the farm, and (d) is a change detection effect image of the invention.

[0035] Figure 8 In the embodiments of the present invention, YellowRi v The erII dataset includes (a) and (b) two SAR images of the same area of ​​the Yellow River at different times, (c) a reference image of the actual changes in the Yellow River, and (d) a result of the change detection of this invention. Detailed Implementation

[0036] The present invention will now be described in further detail with reference to the accompanying drawings.

[0037] Reference Figure 1 The specific steps for implementing this invention are as follows:

[0038] Step 1: Generate a log-ratio difference plot.

[0039] Input two SAR images, I1 and I2, acquired at different times in the same region, and generate a difference map DI using the logarithmic ratio operator. LR :

[0040]

[0041] Step 2: Generate a significant difference plot.

[0042] Step 1: Obtain the logarithmic difference plot (DI) LR After smoothing with a Gaussian kernel to suppress noise interference, the smoothed difference map DI is obtained. S The calculation formula is as follows:

[0043] DIs (i, j) = G σ (i, j)*DI LR (i, j) (25)

[0044] Where * represents convolution operation, G σ (i, j) is the standard Gaussian kernel, defined as:

[0045]

[0046] From formulas (25) and (26), it can be seen that the larger σ is, the stronger the smoothing effect on the image;

[0047] Step 2: To highlight regions of significant change, the eight-neighbor discrete Laplace operator is used:

[0048]

[0049] Convolving the smoothed difference map yields a local contrast map C(i,j), defined as:

[0050] C(i,j)=|L*D s (i, j) | (28)

[0051] As can be seen from formula (27), the larger the local gradient at a certain pixel position, the larger its local contrast C(i,j), that is, the more likely the position is to be a region of change;

[0052] Step 3: To avoid the local contrast value range being too large and affecting subsequent fusion processing, the local contrast map C(i,j) is normalized to the interval [0,1] to obtain the saliency map DI. Sal :

[0053]

[0054] Among them, C min and C max ε represents the minimum and maximum values ​​of the local contrast map, and ε is a small constant to prevent the denominator from being zero. As can be seen from formula (29), the larger the local contrast, the closer its significance value is to 1, which means it is judged as a region of significant change.

[0055] Step 3: Generate a significant fusion difference map.

[0056] Step 1: During the fusion process, the enhancement weights are first constructed:

[0057] W(i,j)=1+DI Sal (i, j) (30)

[0058] The higher the significance, the larger the weight W(i,j), thus highlighting the significant region in the subsequent fusion.

[0059] Step 2: Final Significance Fusion Difference Plot (DI) F (i, j) is defined as:

[0060] DI F (i,j)=DI LR (i,j)·W(i,j) (31)

[0061] Substituting formula (30) into formula (31), it can be further written as:

[0062] DI F (i, j) = DI LR (i, j)·(1+DI) Sal (i, j)) (32)

[0063] Step 4: Pre-classify the saliency fusion difference map.

[0064] Fuzzy C-means clustering algorithm is used for DI F (i, j) is used for pre-classification, dividing the pixels into three categories:

[0065] Variation class (Ω) c ): Areas of significant change;

[0066] Unchanged class (Ω) u ): Stable background area;

[0067] Uncertainty class (Ω) i ): Boundary areas that are difficult to determine.

[0068] Step 5: Select training samples and test samples.

[0069] (5a) Select the variable Ω from the original SAR image. c and the unchanged class Ω u The N×N neighborhood around a pixel is used as a training sample;

[0070] (5b) Select Ω which belongs to the uncertainty class i The N×N neighborhood around the pixel is used as the test sample;

[0071] (5c) Adopt a sample balancing strategy to ensure that the number of training samples for the changed class and the unchanged class is balanced.

[0072] Step 6: Construct the CEAN network model.

[0073] The CEAN network mainly consists of a feature extraction backbone network, an EMA module, and an ASPP module.

[0074] (6a) The feature extraction backbone network adopts a lightweight convolutional neural network, which contains multiple convolutional layers and pooling layers;

[0075] (6b) EMA module structure as follows Figure 4 As shown, this is achieved through the following sub-steps:

[0076] For the input feature map Channel attention modeling and spatial attention modeling are performed separately to obtain the channel weights W. c With spatial weight W s Feature enhancement is achieved through channel-by-channel and spatial-by-spatial weighting, and its calculation steps include:

[0077] (1) Perform global average pooling on the feature map along the spatial dimension to obtain the channel description vector z. c :

[0078]

[0079] z c Input a channel attention module consisting of two fully connected layers to obtain the channel weights:

[0080] W c =σ(W2δ(W1z) c (34)

[0081] Where δ(·) is the ReLU activation function and σ(·) is the Sigmoid activation function;

[0082] (2) Perform max pooling and average pooling on the feature map in the channel dimension to obtain two spatial description maps:

[0083]

[0084] The two are concatenated and then input into the convolution operator to obtain the spatial weight matrix:

[0085] W s =σ(Conv 3×3 ([F avg ;F max (37)

[0086] (3) Adjust the channel weight W c With spatial weight W s After broadcasting to the same dimension, the feature maps are weighted to obtain enhanced feature maps:

[0087] F EMA (c, i, j) = F(c, i, j)·W c (c)·W s (i, j) (38)

[0088] (6c) The ASPP module structure is as follows: Figure 5 As shown, it includes the following steps:

[0089] Step 1: Input the input feature map F into five parallel branches for multi-scale feature extraction. The first branch performs a 1×1 convolution operation on the input feature map to obtain local non-dilated features.

[0090] F1 = Conv 1×1 (F) (39)

[0091] The second branch performs a 3×3 dilated convolution with a dilation rate r = 6 on the input feature map, resulting in features with a receptive field of 13×13:

[0092]

[0093] The third branch performs a 3×3 dilated convolution with a dilation rate r = 12 on the input feature map, resulting in features with a receptive field of 25×25.

[0094]

[0095] The fourth branch performs a 3×3 dilated convolution with a dilation rate r = 18 on the input feature map, resulting in features with a receptive field of 37×37.

[0096]

[0097] The fifth branch performs global average pooling on the input feature map:

[0098] F gap =GlobalAvgPool(F) (43)

[0099] Then, a 1×1 convolution is performed on the pooling result to obtain global context features, and bilinear interpolation is used to upsample the features to the spatial size of the input feature map, resulting in:

[0100] F5 = Upsample(Conv) 1×1 (F gap ), H, W) (44)

[0101] Step 2: Concatenate the output features of the five branches according to the channel dimension to form a multi-scale fused feature:

[0102] F concat =Concat(F1, F2, F3, F4, F5) (45)

[0103] Step 3: Perform a 1×1 convolution on the concatenated features to achieve the fusion and compression of multi-scale contextual information, obtaining the output of the ASPP module:

[0104] F out =Conv 1×1 (Fconcat (46)

[0105] Step 7: Train the CEAN network and perform the final classification.

[0106] (7a) Train the CEAN network using the training samples selected in step 5, employing the cross-entropy loss function and the Adam optimizer;

[0107] (7b) After the network training is completed, the test samples are input for forward propagation to obtain the probability of each pixel belonging to the change class;

[0108] (7c) Threshold segmentation is used to binarize the probability map to obtain the final change detection result map.

[0109] The simulation effects of this invention will be further described below in conjunction with simulation experiments:

[0110] 1. Simulation Environment

[0111] The hardware testing platform of this invention is: an Intel i5-9300H CPU with a main frequency of 2.40GHz and 16GB of memory; the software platform is: Windows 10 system and PyCharm platform.

[0112] 2. Simulation Content

[0113] The superiority of this invention is verified using three real SAR image datasets. In order to more objectively illustrate the effect of change detection, this invention provides five quantitative evaluation criteria for the detection results: number of false negatives (FN), number of false positives (FP), total number of errors (OE), percentage of correct classifications (PCC), and Kappa coefficient (KC). The results are compared with three existing methods, CWNN, DDNet, FFITN, and LANTNet, on real datasets.

[0114] The first dataset selected in this embodiment is the Bern dataset, such as... Figure 6 The two SAR images shown in (a) and (b) were taken by ERS-2 in April and May 1999, and are 301×301 pixels in size. Figure 6 (c) is a reference image for this dataset; the second dataset is the Farmland dataset, which was taken by the Radarsat-2 remote sensing satellite sensor launched by Canada in June 2008 and June 2009 at the Yellow River estuary in China. The spatial resolution is 8m, and the area has changed due to agricultural production activities. The image size is 306×291. Figure 7 (a) and 7(b) depict two images from June 2008 and June 2009, respectively. Figure 7(c) is a reference image for this dataset; the third dataset is the Yellow River II dataset, photographed in June 2008 and June 2009 respectively. For this dataset, a representative region of 257×289 pixels is extracted, as shown below. Figure 8 (a) and Figure 8 As shown in (b). Figure 8 (c) is a reference figure for this dataset.

[0115] 3. Simulation Results and Analysis

[0116] Table 1 presents the change detection comparison results for the Bern dataset. Compared with the other three methods, the method proposed in this invention has the lowest total error (OE). In addition, PCC and Kappa have the highest OE.

[0117] Table 2 presents the change detection comparison results for the Farmland dataset. Compared with the other three methods, the method proposed in this invention has the lowest number of missed detections (FP) and total error (OE). In addition, PCC and Kappa have the highest numbers.

[0118] Table 3 presents the change detection comparison results for the Yellow River II dataset. Compared with the other three methods, the method proposed in this invention has the lowest total error (OE), and the highest PCC and Kappa values.

[0119] Table 1 Comparison of Change Detection Results in the Bern Dataset

[0120] method FP FN OE PCC (%) Kappa CWNN 85 230 315 99.65 0.8528 DDNet 81 252 333 99.63 0.8425 FFINTNet 292 148 440 99.51 0.8182 LANTNet 96 193 289 99.68 0.8678 This invention 109 161 270 99.70 0.8789

[0121] Table 2 Comparison of Change Detection Results in the Farmland Dataset

[0122] method FP FN OE PCC (%) Kappa CWNN 848 572 1420 98.41 0.8602 DDNet 83 1611 1694 0.9810 0.8023 FFINTNet 235 935 1170 98.69 0.8742 LANTNet 720 841 1561 98.25 0.8409 This invention 257 795 1052 98.82 0.8886

[0123] Table 3 Comparison of Change Detection Results in the Yellow River II Dataset

[0124] method FP FN OE PCC (%) Kappa CWNN 1453 1510 2963 96.01 0.8651 DDNet 411 3527 3938 94.70 0.8023 FFINTNet 2576 826 3402 95.42 0.8529 LANTNet 326 3650 3976 94.65 0.8000 This invention 838 1577 2415 96.75 0.8879

[0125] Based on the data results in the three tables above, the SAR image change detection method based on saliency detection and channel-enhanced attention network proposed in this invention outperforms the other three methods on the three public datasets. The method proposed in this invention effectively suppresses the influence of speckle noise, and enhances the model's ability to capture features at different scales through the aggregation of multi-scale contextual information, thus obtaining reliable change detection results.

[0126] Advantages and effects of the present invention

[0127] Experiments on different SAR datasets have validated that the method of this invention effectively suppresses the false alarm rate while maintaining high detection accuracy, particularly excelling in edge preservation and detail recovery. Compared with existing mainstream methods, this invention significantly improves quantitative evaluation metrics including overall error rate (OE), percentage of correct classifications (PCC), and Kappa coefficient.

[0128] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A SAR image change detection method based on saliency detection and channel-enhanced attention network, characterized in that, Includes the following steps: (1) Input two dual-temporal SAR images and generate a log-ratio difference map using the log-ratio (LR) operator; (2) Perform a significance test on the logarithmic ratio difference plot to generate a significant difference plot; (3) Multiply and fuse the logarithmic ratio difference plot and the significance difference plot to obtain the significance fused difference plot; (4) The significant fusion difference map is pre-classified to obtain the changed class, the unchanged class, and the uncertain class; (5) Based on the pre-classification results, select training samples and test samples from the original images; Hierarchical Fuzzy C-Means (HFCM) clustering algorithm is used to pre-classify the discrete wavelet fusion difference map to obtain the initial classification result change class (Ω). c ), unchanged class (Ω) u ), uncertain class (Ω) i ). In obtaining the pre-classification results {Ω c Ω i Ω u After that, select the one belonging to Ω. c and Ω u The neighborhood around the pixel is used as the training sample; samples belonging to Ω are selected. i The neighborhood around the pixel is used as the test sample; (6) Construct a Channel-Enhanced Attention Network (CEAN) model, which includes an Efficient Multimodal Attention (EMA) module and an Atrous Spatial Pyramid Pooling (ASPP) module; (7) Use the training samples to train the CEAN network and classify the test samples to obtain the final change detection results.

2. The method according to claim 1, characterized in that: Using logarithmic ratio (Log - The log-ratio (LR) operator generates a log-ratio difference plot. The specific steps are as follows: For two raw SAR images I1 = {I1(i,j), 1≤i≤M, 1≤j≤N} and I2 = {I2(i,j), 1≤i≤M, 1≤j≤N} obtained at different times for the same geographic area, the difference map DI is plotted using the logarithmic ratio (LR) operator. LR Defined as:

3. The method according to claim 1, characterized in that: The method for performing significance testing on the log-ratio difference plot and generating the significance difference plot in step (2) is as follows: Step 1: Obtain the logarithmic difference plot (DI) LR After smoothing with a Gaussian kernel to suppress noise interference, the smoothed difference map DI is obtained. S The calculation formula is as follows: IN s (i, j)=G σ (i, j)*DI LR (i,j) (2) Where * represents convolution operation, G σ (i, j) is the standard Gaussian kernel, defined as: From formulas (2) and (3), it can be seen that the larger σ is, the stronger the smoothing effect on the image; Step 2: To highlight regions of significant change, the eight-neighbor discrete Laplace operator is used: Convolving the smoothed difference map yields a local contrast map C(i,j), defined as: C(i,j)=|L*D s (i,j)| (5) As can be seen from formula (4), the larger the local gradient at a certain pixel position, the larger its local contrast C(i,j), that is, the more likely the position is to be a region of change; Step 3: To avoid the local contrast value range being too large and affecting subsequent fusion processing, the local contrast map C(i,j) is normalized to the interval [0,1] to obtain the saliency map DI. Scl : Among them, C min and C max ε represents the minimum and maximum values ​​of the local contrast map, and ε is a small constant to prevent the denominator from being zero. As can be seen from formula (6), the larger the local contrast, the closer its significance value is to 1, which means it is judged as a region of significant change.

4. The method according to claim 1, characterized in that: The method for multiplying and fusing the logarithmic ratio difference plot and the significance difference plot in step (3) to obtain the significance fused difference plot is as follows: Step 1: From formulas (1) and (6), we can see that DI LR and DI Scl Then, during the fusion process, the enhancement weight term is first constructed: W(i,j)=1+DI Sal (i, j) (7) Among them, the higher the significance, the larger the weight W(I,j), thus highlighting the significant region in the subsequent fusion; Step 2: Final Significance Fusion Difference Plot (DI) F (i, j) is defined as: IN F (i, j)=DI LR (i,j) W(i,j) (8) Substituting formula (7) into formula (8), we can further write it as: IN F (i,j)=DI LR (i,j)·(1+DI Sal (i, j)) (9) 5. The method according to claim 1, characterized in that: The EMA module described in step (6) achieves feature enhancement through a dynamic channel-spatial weighting mechanism. The specific method is as follows: For the input feature map Channel attention modeling and spatial attention modeling are performed separately to obtain the channel weights W. c With spatial weight W s Feature enhancement is achieved through channel-by-channel and spatial-by-spatial weighting, and its calculation steps include: (1) Perform global average pooling on the feature map along the spatial dimension to obtain the channel description vector z. c : z c Input a channel attention module consisting of two fully connected layers to obtain the channel weights: W c =σ(W2δ(W1z c )) (11) Where δ(·) is the ReLU activation function and σ(·) is the Sigmoid activation function; (2) Perform max pooling and average pooling on the feature map in the channel dimension to obtain two spatial description maps: The two are concatenated and then input into the convolution operator to obtain the spatial weight matrix: W s =σ(Conv 3×3 ([F avg ;F max (14)(3) Adjust the channel weight W c With spatial weight W s After broadcasting to the same dimension, the feature maps are weighted to obtain enhanced feature maps: F EMA (c,i,j)=F(c,i,j)·W c (c)·W s (i,j) (15) 6. The method according to claim 1, characterized in that: The ASPP module described in step (6) captures multi-scale contextual information through multi-scale dilated convolution and global pooling. The specific method is as follows: Step 1: Input the input feature map F into five parallel branches for multi-scale feature extraction. The first branch performs a 1×1 convolution operation on the input feature map to obtain local non-dilated features. F1=Conv 1×1 (F) (16) The second branch performs a 3×3 dilated convolution with a dilation rate r = 6 on the input feature map, resulting in features with a receptive field of 13×13: The third branch performs a 3×3 dilated convolution with a dilation rate r = 12 on the input feature map, resulting in features with a receptive field of 25×25. The fourth branch performs a 3×3 dilated convolution with a dilation rate r = 18 on the input feature map, resulting in features with a receptive field of 37×37. The fifth branch performs global average pooling on the input feature map: F gap =GlobalAvgPool(F) (20) Then, a 1×1 convolution is performed on the pooling result to obtain global context features, and bilinear interpolation is used to upsample the feature map to the spatial size of the input feature map, resulting in: F5=Upsample(Conv 1×1 (F gap ),H,W) (21) Step 2: Concatenate the output features of the five branches according to the channel dimension to form a multi-scale fused feature: F concat =Concat(F1,F2,F3,F4,F5) (22) Step 3: Perform a 1×1 convolution on the concatenated features to achieve the fusion and compression of multi-scale contextual information, obtaining the output of the ASPP module: F out =Conv 1×1 (F concat ) (23)。

Images