SAR image change detection method based on saliency detection and attention CWNN

Abstract

The invention provides an SAR image change detection method based on saliency detection and attention CWNN. The SAR image change detection method comprises the following specific implementation steps: (1) obtaining a combined difference chart; (2) performing saliency detection on the combined difference image and generating a saliency image; (3) identifying and extracting key feature information; (4) pre-classifying the combined difference chart; (5) selecting a training sample and a test sample; and (6) using the trained CWNN model to carry out final classification on test samples. According to the SAR image change detection method, through significance detection and attention CWNN, the significant features of the change area can be accurately extracted, useful information is adaptively extracted through the network, and the accuracy and robustness of SAR image change detection are significantly improved.

Description

Technical Field

[0001] The present invention belongs to the field of SAR image change detection, and specifically relates to a SAR image change detection method based on saliency detection and attention CWNN. The present invention can be applied to computer vision, remote sensing image processing, pattern recognition and other fields for image fusion, change detection and target recognition. Background Art

[0002] Change detection technology is an important means of identifying surface changes by comparing images acquired at different times in the same area. SAR images provide an effective means of monitoring surface changes at night or in bad weather because they can penetrate clouds and lighting conditions. However, multiplicative speckle noise is introduced during the acquisition of SAR images, which can significantly affect the accuracy of change detection. In addition, SAR image change detection also faces a series of challenges such as image registration, radiation correction, and feature extraction.

[0003] At present, the research on change detection in SAR images mainly focuses on two aspects: one is to develop effective image preprocessing techniques to suppress speckle noise and improve image quality; the other is to design robust classification algorithms to accurately identify change areas. Traditional classification methods, such as Fuzzy C-Means (FCM) and K-Means Clustering (K-Means), are effective in some cases, but they usually require manual parameter setting and have poor adaptability to large-scale data sets. Summary of the invention

[0004] In order to overcome the limitations of the prior art, the present invention proposes a SAR image change detection method based on saliency detection and attention CWNN. The present invention not only improves the quality of the difference map, but also enhances the robustness of the classifier.

[0005] The specific steps of the present invention include the following:

[0006] (1) Obtain the combination difference map:

[0007] (1a) Input two dual-phase SAR images;

[0008] (1b) Using the Subtraction (S) operator, the Mean-ratio (MR) operator, and the Log-ratio (LR) operator to obtain a single difference map;

[0009] (1c) Combining the advantages of multiple operators, two of the difference maps are combined using a linear combination method to generate a combined difference map;

[0010] (2) Perform saliency detection on the combined difference map and generate a saliency map:

[0011] Perform saliency detection on the obtained combined difference map, extract salient areas and generate a saliency map;

[0012] (3) Identify and extract key feature information:

[0013] (3a) Introducing the CBAM module into the CWNN model;

[0014] (3b) Construct the network structure of the attention CWNN to further mine the deep features in the image and identify and extract key feature information;

[0015] (4) Pre-classify the combined difference map:

[0016] (4a) The hierarchical fuzzy C-means clustering (HFCM) algorithm is used to pre-classify the combined difference map to obtain the initial classification result {Ω c ,Ω i ,Ω u};

[0017] (4b) Through the saliency detection algorithm, {Ω c ,Ω i ,Ω u}Further screening;

[0018] (5) Obtain training samples and test samples:

[0019] (5a) After obtaining the pre-classification result {Ω c ,Ω i ,Ω u}, select the c and Ω u The neighborhood around the class pixel is used as a training sample;

[0020] (5b) After obtaining the pre-classification result {Ω c ,Ω i ,Ω u}, select the i The neighborhood around the class pixel is used as a test sample;

[0021] (6) Use the trained CWNN model to perform final classification on the test samples:

[0022] The CWNN model is trained using training samples. After the CWNN model training is completed, the test samples are input for final classification.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] First, the linear combination difference map method proposed in the present invention effectively combines the advantages of multiple operators and improves the reliability and accuracy of change detection results.

[0025] Second, saliency detection effectively suppresses the interference of background information, retains significant change features, and improves the accuracy of pre-classification.

[0026] Third, the CWNN network model combined with CBAM adopted in the present invention reduces the training time compared with the traditional convolutional network. At the same time, the network structure takes into account the rotational invariance and spatial hierarchy between features, thereby improving the robustness of the model and the accuracy of classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the implementation of the present invention;

[0028] Figure 2 is a ROC curve diagram of the difference map on the Berne data set of the present invention;

[0029] Figure 3 is a ROC curve diagram of the difference map on the Ottawa dataset of the present invention;

[0030] Figure 4 is a ROC curve diagram of the difference map on the San Francisco dataset of the present invention;

[0031] Figure 5 It is a difference graph combination process diagram of the present invention;

[0032] Figure 6 It is a CAM structure diagram of the present invention;

[0033] Figure 7 It is a SAM structure diagram of the present invention;

[0034] Figure 8 is the attention CWNN model diagram of the present invention;

[0035] Fig. 9 is the Bern dataset in the embodiment of the present invention, wherein (a) and (b) are two SAR images of the same area in Bern at different times, (c) is the Bern real change reference map, and (d) is the change detection effect map of the present invention;

[0036] Fig.10 is the Ottawa dataset in an embodiment of the present invention, wherein (a) and (b) are two SAR images of the same area of ​​Ottawa at different times, (c) is a real change reference map of Ottawa, and (d) is a change detection effect map of the present invention;

[0037] Fig.11: is a San Francisco dataset in an embodiment of the present invention, wherein (a) and (b) are two SAR images of the same area of ​​San Francisco at different times, (c) is a reference map of real changes in San Francisco, and (d) is a change detection effect map of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings.

[0039] Reference Figure 1 The specific steps of the present invention are as follows:

[0040] Step 1: Obtain the combined difference map.

[0041] Step 1: Input two SAR images I1 and I2 of the same area at different times, with a size of M × N. First, perform PPB filtering on I1 and I2 to obtain denoised images X1 and X2;

[0042] Step 2: Generate difference maps using the Subtraction (S) operator, the Mean-ratio (MR) operator, and the Log-ratio (LR) operator, and select the two difference maps D1 and D2 with larger AUC values;

[0043] The specific algorithms of the three operators and the combined difference graph are shown in formula (4), formula (5), formula (6) and formula (7):

[0044] D s (i,j)=|X1(i,j)-X2(i,j)| (4)

[0045]

[0046]

[0047] D0=αD1+(1-α)D2 (7)

[0048] Step 2: Perform saliency detection on the obtained combined difference map, extract salient areas and generate a saliency map.

[0049] Step 1: Divide the obtained difference map into image blocks of size m×m. i and X j denote pixel blocks centered at pixels i and j, respectively, and d c (X i , X j ) and d p (X i , X j ) represent X i and X jIn the Lab color space and position, the Euclidean distance combines the color distance and the position distance by introducing a distance weighting parameter λ. The dissimilarity measure between two pixel blocks is:

[0050]

[0051] The values ​​of m and λ are set to 7 and 3 respectively. From formula (8), we can see that when d c (X i , X j ) is larger, d p (X i , X j ) is smaller, d(X i , X j ) is larger, that is, the image block X i and X j The greater the difference, the greater the difference between i judged to be significant;

[0052] Step 2: To solve the problem that evaluating the dissimilarity of all pixel blocks in the difference map requires a lot of computation time, i When measuring the dissimilarity of i Calculate the dissimilarity with the k image blocks with the highest matching degree in its neighborhood, X i The saliency at a single scale τ is defined as:

[0053]

[0054] In step 3, in order to improve the contrast between salient and non-salient areas, a multi-scale saliency enhancement technique is used to evaluate the saliency of the target pixel. When the target pixel is different from the surrounding pixels at all scales, the pixel is judged to be salient. The specific calculation method is shown in formula (10):

[0055]

[0056] Where M represents the number of scales, Represents image block X i The significance on scale r, d r foci(i) represents X i The geometric distance to other image patches at scale r;

[0057] Step 4: After evaluating the significance of each pixel, the OTSU algorithm described by formula (11) is used for threshold segmentation. When the significance value of a pixel is higher than the threshold, the pixel is judged as significant and marked as 1; otherwise, the pixel is judged as non-significant and marked as 0.

[0058]

[0059] Among them, S b is the image value after being processed by the OTSU algorithm, is the original value of pixel i, and τ is the threshold automatically generated by the OTSU algorithm.

[0060] In step 5, the binary saliency image is multiplied by the fused difference map to obtain the saliency fused difference map.

[0061] Step 3: Build the network structure of the attention CWNN to further explore the deep features in the image and identify and extract key feature information:

[0062] Step 1: By introducing CBAM into the CWNN architecture, the deep features in the image are further mined. The network consists of two parts: CBAM and CWNN.

[0063] Step 2: In CBAM, the CAM module takes the image blocks in the training sample as input, first performs maximum pooling and average pooling operations to generate two one-dimensional feature maps, and then merges the two one-dimensional feature maps in the channel domain to integrate spatial information;

[0064] In the third step, MLP is used to reduce the dimension of the merged feature map. Finally, the channel attention map is output through element-by-element summation and Sigmoid function activation. The calculation process is shown as follows:

[0065] F c =σ(MLP(AvgPool(F))+MLP(MaxPool(F))) (12)

[0066] Step 4: The SAM module converts F c ′ is taken as input, and the input feature map is first subjected to channel-based average pooling and maximum pooling operations. Then, the two pooling operation results are concatenated in the channel dimension according to formula (14), and finally processed by convolution and Sigmoid activation function to output the spatial attention map F s ;

[0067]

[0068] F s =σ(f 7×7 ([AvgPool(F′ c );MaxPool(F′ c )])) (14)

[0069] Step 5, after getting F s Then, as shown in formula (15), F and F sPerform multiplication operation to obtain the weighted feature map F′ s According to formula (16), for F′ c and F′ s Perform element-by-element addition to obtain the output feature map F′, which is used as the input of the subsequent convolutional layer. After learning and training, the network outputs a binary change map containing only the numbers O and 1.

[0070]

[0071]

[0072] Step 4: pre-classify the combined difference map.

[0073] In the first step, the hierarchical fuzzy C-means clustering (HFCM) algorithm is used to pre-classify the combined difference map to obtain the initial classification result {Ω c ,Ω i ,Ω u};

[0074] Step 2: Use the saliency detection algorithm to c ,Ω i ,Ω u}Further screening.

[0075] Step 5: Obtain training samples and test samples.

[0076] Step 1: Get the pre-classification result {Ω c ,Ω i ,Ω u}, select the c and Ω u The neighborhood around the class pixel is used as a training sample, and a sample unification strategy is used to deal with the imbalance of positive and negative samples, selecting the same number of positive and negative samples for training the network;

[0077] Step 2: After obtaining the pre-classification result {Ω c ,Ω i ,Ω u}, select the i The neighborhood around the class pixel is used as the test sample.

[0078] Step 6: Use the trained CWNN model to perform final classification on the test samples.

[0079] The trained CWNN is used to classify the test samples and obtain the final SAR image change detection results.

[0080] The simulation effect of the present invention is further described below in conjunction with simulation experiments:

[0081] 1. Simulation environment

[0082] The hardware test platform of the embodiment of the present invention is: the processor is Intel i5-9300H CPU, the main frequency is 2.40GHz, the memory is 16GB, and the software platform is: Windows 10 system, Matlab 2019a platform.

[0083] 2. Simulation content

[0084] Three real SAR image data sets are used to verify the superiority of the present invention. At the same time, in order to more objectively illustrate the effect of change detection, the present invention provides five quantitative evaluation criteria for the detection results: number of missed detections (FN), number of false detections (FP), total number of errors (OE), percentage of correct classification (PCC) and Kappa coefficient (KC), and compares them with three existing methods, PCA-K, NRELM and CWNN, on real data sets.

[0085] The first data set selected in this embodiment is the Berne data set, such as Fig. 9 The two SAR images shown in (a) and (b) were taken by ERS-2 in April and May 1999, with an image size of 301 × 301 pixels. Fig. 9 (c) is the reference image of the dataset; the second dataset is the Ottawa dataset, which was taken by Radarsat-1 in May and August 1997 to capture the flood-affected areas of Ottawa, Canada. The image size is 290×350 pixels. Fig.10 (a) and 10(b) depict two images taken in May and August 1997, respectively. Fig.10 (c) is the reference image of the dataset; the third dataset is the San Francisco dataset, which is an image of the San Francisco area acquired by the ERS2 satellite SAR sensor in August 2003 and May 2004. The image size is 256×256 pixels. Fig.11 (a) and Fig.11 (b) Fig.11 (c) is the reference image of this dataset.

[0086] 3. Simulation results and analysis

[0087] Table 1 shows the comparison results of change detection on the Bern dataset. Compared with the other three methods, the method proposed in this paper has the lowest number of missed detections (FN) and overall error (OE). In addition, PCC and Kappa are the highest.

[0088] Table 2 shows the comparison results of change detection on the Ottawa dataset. Compared with the other three methods, the method proposed in this paper has the lowest number of missed detections (FN) and total error (OE). In addition, PCC and Kappa are the highest.

[0089] Table 3 shows the comparison results of change detection on the San Francisco dataset. Compared with the other three methods, the method proposed in the present invention has the lowest overall error (OE). In addition, it has the highest PCC and Kappa values.

[0090] Table 1 Comparison results of change detection on the Bern dataset

[0091] method FP FN OE PCC (%) Kappa PCA-K 69 263 332 99.63 0.8413 NRELM 214 140 354 99.61 0.8495 CWNN 82 239 321 99.65 0.8491 The present invention 165 115 280 99.69 0.8745

[0092] Table 2 Comparison results of change detection on the Ottawa dataset

[0093] method FP FN OE PCC (%) Kappa PCA-K 274 1170 2044 97.99 0.9214 NRELM 431 1451 1882 98.15 0.9285 CWNN 434 1291 1725 98.30 0.9375 The present invention 507 929 1436 98.59 0.9474

[0094] Table 3. Comparison results of change detection on San Francisco dataset

[0095] method FP FN OE PCC (%) Kappa PCA-K 802 345 1147 98.25 0.8738 NRELM 9787 12 9799 85.05 0.4262 CWNN 437 295 732 98.88 0.9170 The present invention 460 107 567 99.13 0.9325

[0096] From the data results in the above three tables, the SAR image change detection method based on saliency detection and attention CWNN proposed in the present invention has better detection effect on three public data sets than the other three methods. The method proposed in the present invention effectively retains the edge and detail information of the difference map, improves the contrast between the changed area and the unchanged area, and can greatly suppress the influence of background information on change detection.

Claims

1. A SAR image change detection method based on saliency detection and attention CWNN, comprising the following steps: (1) Obtain the combination difference map: (1a) Input two dual-phase SAR images; (1b) Using the Subtraction (S) operator, the Mean-ratio (MR) operator, and the Log-ratio (LR) operator to obtain a single difference map; (1c) Combining the advantages of multiple operators, two of the difference maps are combined using a linear combination method to generate a combined difference map; (2) Perform saliency detection on the combined difference map and generate a saliency map: Perform saliency detection on the obtained combined difference map, extract salient areas and generate a saliency map; (3) Identify and extract key feature information: (3a) Introducing the Convolutional Block Attention Module (CBAM) module into the Convolutional-Wavelet Neural Networks (CWNN) model; (3b) Construct the network structure of the attention CWNN to further mine the deep features in the image and identify and extract key feature information; (4) Pre-classify the combined difference map: (4a) The Hierarchical Fuzzy C-Means (HFCM) algorithm is used to pre-classify the combined difference map to obtain the initial classification result {Ω c ,Ω i ,Ω u }; (4b) Through the saliency detection algorithm, {Ω c ,Ω i ,Ω u }Further screening; (5) Obtain training samples and test samples: (5a) After obtaining the pre-classification result {Ω c ,Ω i ,Ω u }, select the c and Ω u The neighborhood around the class pixel is used as a training sample; (5b) After obtaining the pre-classification result {Ω c ,Ω i ,Ω u }, select the i The neighborhood around the class pixel is used as a test sample; (6) Use the trained CWNN model to perform final classification on the test samples: The CWNN model is trained using training samples. After the CWNN model training is completed, the test samples are input for final classification.

2. The SAR image change detection method based on saliency detection and attention CWNN according to claim 1 is characterized in that: The method for generating the combined difference map in step (1c) comprises the following specific steps: Step 1: Perform PPB filtering on two original SAR images I1 = {I1(i, j), 1≤i≤M, 1≤j≤N} and I2 = {I2(i, j), 1≤i≤M, 1≤j≤N} of the same geographical area obtained at different times, with a size of M×N, to obtain the corresponding denoised images X1 and X2; Step 2: Select two difference maps with larger AUC values ​​from the difference maps generated by the difference (S) operator, the mean ratio (MR) operator, and the logarithmic ratio (LR) operator, and set them as D1 and D2. Figures 2 to 4 show the ROC curves on the Berne dataset, the Ottawa dataset, and the San Francisco dataset, respectively. The area under the curve (AUC) represents the area under the receiver operating characteristic (ROC), and AW represents the adaptive weight difference map combination method. D1 and D2 are linearly combined by formula (5) to obtain the final combined difference map D0. Figure 5 shows the difference map combination process. The specific algorithms of the three operators and the combined difference graph are shown in formula (4), formula (5), formula (6) and formula (7): D s (i,j)=|X1(i,j)-X2(i,j)| (4) D0 = αD1 + (1-α)D2 (7).

3. The SAR image change detection method based on saliency detection and attention CWNN according to claim 1, characterized in that: The method for performing significance detection on the combined difference map and generating a significance map in step (2) is: Step 1: Divide the obtained difference map into image blocks of size m×m, let X i and X j denote pixel blocks centered at pixels i and j, respectively, and d c (X i , X j ) and d p (X i , X j ) represent X i and X j In the Lab color space and position, the Euclidean distance combines the color distance and the position distance by introducing a distance weighting parameter λ. The dissimilarity measure between two pixel blocks is: The values ​​of m and λ are set to 7 and 3 respectively. From formula (8), we can see that when d c (X i , X j ) is larger, d p (X i , X j ) is smaller, d(X i , X j ) is larger, that is, the image block X i and X j The greater the difference, the greater the difference between i judged to be significant; Step 2: To solve the problem that evaluating the dissimilarity of all pixel blocks in the difference map takes a lot of computation time, i When measuring the dissimilarity of i Calculate the dissimilarity with the k image blocks with the highest matching degree in its neighborhood, X i The saliency at a single scale τ is defined as: Step 3: In order to improve the contrast between salient and non-salient areas, a multi-scale saliency enhancement technique is used to evaluate the saliency of the target pixel. When the target pixel is different from the surrounding pixels at all scales, the pixel is judged to be salient. The specific calculation method is shown in formula (10): Where M represents the number of scales, Represents image block X i The significance on scale r, d r foci(i) represents X i The geometric distance to other image patches at scale r; Step 4: After evaluating the significance of each pixel, the Nobuyuki (OTSU) algorithm described in formula (11) is used for threshold segmentation. When the significance value of a pixel is higher than the threshold, the pixel is judged as significant and marked as 1; otherwise, the pixel is judged as non-significant and marked as 0. Among them, S b is the image value after being processed by the OTSU algorithm, is the original value of pixel i, τ is the threshold automatically generated by the OTSU algorithm; Step 5: Multiply the binary saliency image and the combined difference map to obtain a significant combined difference map; then use the hierarchical fuzzy C-means clustering algorithm to pre-classify the significant combined difference map, obtain pseudo labels, and construct training samples and test samples based on the pseudo labels.

4. The SAR image change detection method based on saliency detection and attention CWNN according to claim 1, characterized in that: The specific steps of constructing the network structure of the attention CWNN described in step (3b) to further mine the deep features in the image and identify and extract key feature information are as follows: Step 1: Build a CWNN based on CBAM. The network uses the attention mechanism to extract key features in the image. CBAM consists of two parts: Channel Attention Module (CAM) and Spatial Attention Module (SAM). CAM identifies important feature channels by analyzing the global information of feature maps. Figure 6 shows its structure. Step 2: After inputting the feature map, the average pooling and maximum pooling operations are first used to capture the diverse contextual information in the feature map. Then, this information is sent to a shared multi-layer perceptron (MLP) for processing. The MLP learns the complex nonlinear relationship between different channels and outputs a set of weights. After being activated by the Sigmoid function, a channel attention feature map is generated. Step 3: SAM follows CAM and further refines the focus of the model in the spatial dimension of the feature map. Figure 7 shows the specific structure of SAM. Two pooling operations are used to extract key information about the spatial position from each channel of the input feature. Then, the two feature maps obtained by the pooling operation are spliced ​​in the channel dimension. Secondly, the convolution operation is used to reduce the dimension of the spliced ​​feature map, and then the sigmoid function is activated to generate the spatial attention feature map. Finally, the CWNN network structure includes convolution layer, CBAM attention module, wavelet pooling layer and fully connected layer. Figure 8 shows its network structure.

5. The SAR image change detection method based on saliency detection and attention CWNN according to claim 1, characterized in that: The step (5a) described in obtaining the pre-classification result {Ω c ,Ω i ,Ω u }, select the c and Ω u The neighborhood around the class pixel is used as the training sample, and the sample unification strategy is adopted to deal with the imbalance problem of positive and negative samples, and the same number of positive and negative samples are selected to train the network.

Images