SAR image change detection method based on weighted fusion wavelet transform and improved residual network

Abstract

The invention provides an SAR (Synthetic Aperture Radar) image change detection method based on weighted fusion wavelet transform and an improved residual network. The specific implementation steps are as follows: (1) acquiring a weighted fusion difference chart; (2) discrete wavelet decomposition is carried out on the weighted fusion difference graph; (3) reconstructing and generating a discrete wavelet fusion difference chart; (4) identifying and extracting key feature information; (5) pre-classifying the discrete wavelet fusion difference chart; (6) selecting a training sample and a test sample; and (7) performing final classification on the test samples by using the trained SEPP-ResNet model. According to the method, through weighted fusion of wavelet transform and SEPP-ResNet, the understanding of SAR images can be enhanced, and multi-scale context information can be fully excavated, so that the network can realize more accurate and reliable change detection in a complex environment.

Description

Technical Field

[0001] This invention belongs to the field of SAR image change detection, specifically a SAR image change detection method based on weighted fusion wavelet transform and improved residual network. This invention can be applied to computer vision, remote sensing image processing, pattern recognition, and other fields for image fusion, change detection, and target recognition. Background Art

[0002] Image change detection is an important technology in remote sensing. It identifies changes in land cover by analyzing SAR images of the same area acquired at different time points. However, the acquisition process of SAR images introduces multiplicative speckle noise, which significantly affects the accuracy of change detection. Furthermore, SAR image change detection faces a number of challenges, including insufficient information utilization and poor global understanding.

[0003] Currently, research on change detection in SAR images focuses on two main areas: first, developing effective difference map generation techniques to suppress speckle noise and improve image quality; second, designing robust classification algorithms to accurately identify changed regions. Traditional methods, such as ratio-based, pixel-based, and object-based methods, while effective in some cases, often require manual parameter setting and have poor adaptability to large-scale datasets. Summary of the Invention

[0004] To overcome the limitations of existing technologies, this paper proposes a SAR image change detection method based on weighted fusion wavelet transform and improved residual network. This method not only improves the quality of difference maps but also enhances the robustness of the classification model.

[0005] The specific steps of implementing the present invention include the following:

[0006] (1) Obtain weighted fusion difference map:

[0007] (1a) Input two dual-phase SAR images;

[0008] (1b) Using the log-ratio (LR) operator and the mean-ratio (MR) operator to obtain two difference maps;

[0009] (1c) Combining the advantages of multiple operators, a weighted fusion method is used to fuse the two difference maps to generate a weighted fused difference map;

[0010] (2) Perform discrete wavelet decomposition on the weighted fusion difference map:

[0011] Perform discrete wavelet transform on the weighted fusion difference map to decompose it into low-frequency information (approximate part) and high-frequency information (detail part);

[0012] (3) Reconstruction and generation of discrete wavelet fusion difference map

[0013] The low-frequency information is fused using the weighted average method, while the high-frequency information is fused using the minimum variance criterion. Finally, the fused low-frequency and high-frequency information are recombined using the inverse discrete wavelet transform to generate a discrete wavelet fusion difference map.

[0014] (4) Identify and extract key feature information:

[0015] (4a) Introducing the SE (Squeeze-and-Excitation) attention mechanism and the Pyramid Pooling Module (PPM) into the Residual Network (ResNet) model;

[0016] (4b) Construct the SEPP-ResNet network structure to further extract key features and capture contextual information at different scales;

[0017] (5) Pre-classify the discrete wavelet fusion difference map:

[0018] The Hierarchical Fuzzy C-Means (HFCM) algorithm is used to pre-classify the discrete wavelet fusion difference map to obtain the initial classification result {Ω c ,Ω i ,Ω u};

[0019] (6) Obtain training samples and test samples:

[0020] (6a) After obtaining the pre-classification result {Ω c ,Ω i ,Ω u}, select the c and Ω u The neighborhood around the class pixel is used as a training sample;

[0021] (6b) After obtaining the pre-classification result {Ω c ,Ω i ,Ω u}, select the i The neighborhood around the class pixel is used as a test sample;

[0022] (7) Use the trained SEPP-ResNet model to perform final classification on the test samples:

[0023] The SEPP-ResNet model is trained using training samples. After the model training is completed, the test samples are input for final classification.

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] First, the weighted fusion difference map method proposed in the present invention effectively combines the advantages of multiple operators and improves the reliability and accuracy of change detection results.

[0026] Second, the discrete wavelet transform method makes full use of the low-frequency and high-frequency information of the image to enhance the difference map features and effectively suppress the interference of speckle noise.

[0027] Third, the SEPP-ResNet network model adopted in the present invention can pay more attention to key feature areas compared with traditional convolutional networks. At the same time, the network model can deeply extract contextual information of different scales, thereby improving the robustness of the model and the accuracy of classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is an implementation flow chart of the present invention;

[0029] Figure 2 It is a difference graph combination process diagram of the present invention;

[0030] Figure 3 This is a diagram of the process of reconstructing the fused difference map by discrete wavelet transform of the present invention;

[0031] Figure 4 This is the structure diagram of the SE attention mechanism of the present invention;

[0032] Figure 5 It is a PPM structure diagram of the present invention;

[0033] Figure 6 This is the SEPP-ResNet model diagram of the present invention;

[0034] Figure 7 This is the Bern dataset in an embodiment of the present invention, where (a) and (b) are two SAR images of the same area in Bern at different times, (c) is the Bern real change reference map, and (d) is the change detection effect map of the present invention;

[0035] Figure 8 is the Ottawa dataset in an embodiment of the present invention, where (a) and (b) are two SAR images of the same area of ​​Ottawa at different times, (c) is a real change reference map of Ottawa, and (d) is a change detection effect map of the present invention;

[0036] Figure 9This is the Sulzberger dataset in an embodiment of the present invention, where (a) and (b) are two SAR images of the same area of ​​Sulzberger at different times, (c) is a real change reference map of Sulzberger, and (d) is a change detection effect map of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below with reference to the accompanying drawings.

[0038] Reference Figure 1 , the specific steps of the present invention are as follows:

[0039] Step 1: Obtain the weighted fusion difference map.

[0040] Step 1: For two original SAR images I1 and I2 obtained at different times in the same geographical area, two difference maps are generated using the logarithmic ratio (LR) operator and the mean ratio (MR) operator. Let them be DI and MR respectively. LR and DI MR .

[0041] Step 2: DI is generated by combining the two and using weighted fusion WF , making comprehensive use of their respective advantages.

[0042] The specific algorithms of the two operators and weighted fusion difference graph are shown in formula (9), formula (10), and formula (11):

[0043]

[0044] DI WF (i,j)=αDI1(i,j)+βDI2(i,j) (11)

[0045] Where α and β represent the weight coefficients of LR and MR operators, respectively.

[0046] Step 2: Perform discrete wavelet decomposition on the weighted fusion difference map.

[0047] Perform discrete wavelet transform on the weighted fusion difference map to decompose it into low-frequency information (approximate part) and high-frequency information (detail part);

[0048] Step 3: Reconstruct and generate a discrete wavelet fusion difference map.

[0049] In the first step, for the decomposed weighted fusion difference map, the low-frequency coefficient part is fused using the weighted average method. The weight factor is determined by the absolute value of the two low-frequency coefficient matrices to ensure that the background is smoothed while retaining important information. The fusion rule of the low-frequency coefficient part is defined as:

[0050]

[0051] Among them, χ and γ represent and The weight factor, and Represent the low-frequency coefficient parts of the log-ratio difference map and the mean-ratio difference map, respectively.

[0052] The fusion coefficients χ and γ are defined as:

[0053]

[0054] in, It represents the absolute value of the average value of the low-frequency region coefficients obtained after discrete wavelet decomposition of the fused difference map. The variance of the two high-frequency coefficient matrices at the same position is calculated, and the coefficient with the smaller variance is selected as the fused coefficient to maintain the edge and texture information.

[0055] Step 2: For the decomposed weighted fusion difference map, the minimum variance criterion is used for the high-frequency coefficient part. The fusion rule for the high-frequency coefficient part is defined as:

[0056]

[0057] in, and Represent the high-frequency coefficient parts of the mean ratio difference map and the logarithmic ratio difference map, respectively. and Represent the variance of the two high-frequency coefficient matrices respectively, and the variance σ is defined as:

[0058]

[0059] Among them, D e (k) represents the number of nodes in the neighborhood window N i,j The high-frequency coefficient value at position k in the neighborhood window N is represented by u(k). i，j The average value of all high-frequency coefficients in N i，j It is a neighborhood window of size N×N, usually 3×3.

[0060] Step 3. Finally, the fused low-frequency and high-frequency coefficients are recombined into a new difference map using the inverse discrete wavelet transform (IDWT). The reconstructed discrete wavelet fusion difference map (DWTDI) effectively reduces the noise component in the image.

[0061] Step 4: Identify and extract key feature information.

[0062] Step 1: Input image X through F tr Transformed into image U, F tr is a standard convolution operator, defined as follows:

[0063]

[0064] in, is the input image, is the output image, V represents a set of learned filter kernels, V c represents the parameters of the c-th filter, Represents a 2D spatial kernel, and * represents a convolution operation.

[0065] In the second step, the H×W×C feature map containing global information is directly compressed into a 1×1×C feature vector Z. The channel features of the C feature maps are compressed into a single value. This makes the generated channel-level feature vector Z contain contextual information, alleviating the problem of channel dependency. It is defined as follows:

[0066]

[0067] Among them, Z c is the cth element of Z.

[0068] In the third step, the Excitation operation is used to fully capture the channel dependency. A gate mechanism consisting of two fully connected layers is used. The first fully connected layer compresses C channels into channels, and then pass through a Relu nonlinear activation layer. The second fully connected layer restores the channels to C channels, and then passes through the sigmoid activation function to obtain the weight s. The final dimension of s is 1×1×C, which is used to represent the weights of the C feature maps in the feature map U. r represents the compression ratio. The weight s is defined as follows:

[0069] s=F ex (z, W)=σ(g(z, W))=σ(W2δ(W1z)) (19)

[0070] Among them, F ex (·) represents the Excitation operation, and σ(·) represents the activation function.

[0071] In the fourth step, the obtained attention weight is weighted to the features of each channel, and each feature map in the feature map U is multiplied by the corresponding weight to obtain the final output of the SE attention mechanism. The output of each channel is defined as follows:

[0072]

[0073] Among them, F scale (·) represents the weighted operation, u c represents the channel eigenvalue, s c Represents the attention weight on each channel.

[0074] Step 5: After obtaining the feature map, add the PPM module to perform multi-scale feature fusion. Figure 5 Its structure is shown, and the feature map obtained after SE attention processing is Perform pooling operations of different scales (1×1, 2×2, 3×3 and 6×6) to obtain the pooled feature maps Among them, s is the pooling ratio, then each pooled feature map is upsampled to the same size H×W as the original feature map, and the upsampled feature map F′ s The definition is as follows:

[0075] F′ s =Upsample(F s , (H, W)) (s=1×1, 2×2, 3×3, 6×6) (21)

[0076] Among them, Upsample(·,·) represents the upsampling operation, F s Represents the feature map after pooling, and (H, W) represents the height and width of the feature map after upsampling.

[0077] Step 6: Concatenate all upsampled feature maps with the original input feature map according to the channel dimension to form the final multi-scale feature representation F out , defined as follows:

[0078] F out =Concat([F,F′ 1×1 , F′ 2×2 , F′ 3×3 , F′ 6×6 ]) (twenty two)

[0079] Among them, Concat([·]) represents the concatenation operation.

[0080] Step 5: Pre-classify the discrete wavelet fusion difference map.

[0081] In the first step, the Hierarchical Fuzzy C-means Clustering (HFCM) algorithm is used to pre-classify the discrete wavelet fusion difference map to obtain the initial classification result {Ω c ,Ω i ,Ω u};

[0082] Step 2, set the threshold again {Ω c ,Ω i ,Ω u}Further screening.

[0083] Step 6: Obtain training samples and test samples.

[0084] Step 1: Get the pre-classification result {Ω c ,Ω i ,Ω u}, select the c and Ω u The neighborhood around the class pixel is used as a training sample, and a sample unification strategy is used to deal with the imbalance of positive and negative samples, selecting the same number of positive and negative samples for training the network;

[0085] Step 2: After obtaining the pre-classification result {Ω c ,Ω i ,Ω u}, select the i The neighborhood around the class pixel is used as the test sample.

[0086] Step 7: Use the trained SEPP-ResNet model to perform final classification on the test samples.

[0087] The trained SEPP-ResNet is used to classify the test samples and obtain the final SAR image change detection results.

[0088] The simulation effect of the present invention is further described below in conjunction with simulation experiments:

[0089] 1. Simulation environment

[0090] The hardware test platform of the embodiment of the present invention is: the processor is an Intel i5-9300H CPU with a main frequency of 2.40 GHz, the memory is 16 GB, and the software platform is: Windows 10 system, Pycharm platform.

[0091] 2. Simulation content

[0092] Three real SAR image datasets are used to verify the superiority of the present invention. At the same time, in order to more objectively illustrate the effect of change detection, the present invention provides five quantitative evaluation criteria for the detection results: number of missed detections (FN), number of false detections (FP), number of overall errors (OE), percentage of correct classifications (PCC) and Kappa coefficient (KC), and compares them with three existing methods (FFITN, LANTNet and DDNet) on real datasets.

[0093] The first data set selected in this embodiment is the Berne data set, such as Figure 7 The two SAR images shown in (a) and (b) were taken by ERS-2 in April 1999 and May 1999, with an image size of 301 × 301 pixels. Figure 7(c) is the reference image of the dataset; the second dataset is the Ottawa dataset, which is the flood-affected area of ​​Ottawa, Canada, taken by Radarsat-1 in May and August 1997. The image size is 290×350 pixels. Figure 8 (a) and 8(b) depict two images taken in May and August 1997, respectively. Figure 8 (c) is the reference image of the dataset; the third dataset is the Sulzberger dataset, which is an image of the ice shelf breakup area caused by the tsunami acquired by the SAR sensor of the European Space Agency's ENVIAST satellite in April 1999 and April 2000. The image size is 256×256 pixels, as shown in Figure 9 (a) and Figure 9 As shown in (b), Figure 9 (c) is the reference image of this dataset.

[0094] 3. Simulation results and analysis

[0095] Table 1 shows the comparison results of change detection on the Bern dataset. Compared with the other three methods, the proposed method has the lowest overall error (OE). In addition, PCC and Kappa are the highest.

[0096] Table 2 shows the comparison results of change detection on the Ottawa dataset. Compared with the other three methods, the proposed method has the lowest number of missed detections (FP) and overall error (OE). In addition, it has the highest PCC and Kappa.

[0097] Table 3 shows the comparison results of change detection on the Sulzberger dataset. Compared with the other three methods, the proposed method has the lowest overall error (OE) and the highest PCC and Kappa values.

[0098] Table 1 Comparison results of change detection on the Bern dataset

[0099] method FP FN OE PCC (%) Kappa FFITN 292 148 440 99.51 0.8182 LANTNet 96 193 289 99.68 0.8678 DDNet 81 252 333 99.63 0.8425 The present invention 111 161 272 99.70 0.8781

[0100] Table 2 Comparison results of change detection on the Ottawa dataset

[0101] method FP FN OE PCC (%) Kappa FFITN 1052 853 1905 98.12 0.9299 LANTNet 2291 393 2684 97.36 0.9052 DDNet 1328 1106 2434 97.60 0.9104 The present invention 726 572 1298 98.72 0.9522

[0102] Table 3 Comparison results of change detection on the Sulzberger dataset

[0103] method FP FN OE PCC (%) Kappa FFITN 523 362 885 98.65 0.9568 LANTNet 569 1261 1830 97.21 0.9082 DDNet 183 1491 1674 97.45 0.9144 The present invention 330 449 779 98.81 0.9616

[0104] The data in the three tables above demonstrate that our proposed SAR image change detection method, based on weighted fusion wavelet transform and improved residual network, outperforms the other three methods on the three public datasets. This method effectively suppresses the influence of speckle noise and, by aggregating multi-scale contextual information, enhances the model's ability to capture features at different scales, resulting in reliable change detection results.

Claims

1. A SAR image change detection method based on weighted fusion wavelet transform and improved residual network, comprising the following steps: (1) Obtain weighted fusion difference map: (1a) Input two dual-phase SAR images; (1b) Using the log-ratio (LR) operator and the mean-ratio (MR) operator to obtain two difference maps; (1c) Combining the advantages of multiple operators, a weighted fusion method is used to fuse the two difference maps to generate a weighted fused difference map; (2) Perform discrete wavelet decomposition on the weighted fusion difference map: Perform discrete wavelet transform on the weighted fusion difference map to decompose it into low-frequency information (approximate part) and high-frequency information (detail part); (3) Reconstruction and generation of discrete wavelet fusion difference map The low-frequency information is fused using the weighted average method, while the high-frequency information is fused using the minimum variance criterion. Finally, the fused low-frequency and high-frequency information are recombined using the inverse discrete wavelet transform to generate a discrete wavelet fusion difference map. (4) Identify and extract key feature information: (4a) Introducing the SE (Squeeze-and-Excitation) attention mechanism and the Pyramid Pooling Module (PPM) into the Residual Network (ResNet) model; (4b) Construct the SEPP-ResNet network structure to further extract key features and capture contextual information at different scales; (5) Pre-classify the discrete wavelet fusion difference map: The Hierarchical Fuzzy C-Means (HFCM) algorithm is used to pre-classify the discrete wavelet fusion difference map to obtain the initial classification result {Ω c ,Ω i ,Ω u }; (6) Obtain training samples and test samples: (6a) After obtaining the pre-classification result {Ω c ,Ω i ,Ω u }, select the c and Ω u The neighborhood around the class pixel is used as a training sample; (6b) After obtaining the pre-classification result {Ω c ,Ω i ,Ω u }, select the i The neighborhood around the class pixel is used as a test sample; (7) Use the trained SEPP-ResNet model to perform final classification on the test samples: The SEPP-ResNet model is trained using training samples. After the model training is completed, the test samples are input for final classification.

2. The SAR image change detection method based on weighted fusion wavelet transform and improved residual network according to claim 1 is characterized by: The method for generating the weighted fusion difference map in step (1c) is as follows: Step 1: For two original SAR images I1 = {I1(i, j), 1≤i≤M, 1≤j≤N} and I2 = {I2(i, j), 1≤i≤M, 1≤j≤N} obtained at different times in the same geographical area, two difference maps are generated using the logarithmic ratio (LR) operator and the mean ratio (MR) operator. Let them be DI LR and DI MR . Step 2: Generate DI by combining the two and using weighted fusion wF , their respective advantages can be comprehensively utilized, retaining the effect of the LR operator in suppressing background information and the ability of the MR operator in enhancing the information of the changed area. Figure 2 shows the process of weighted fusion. The specific algorithms of the two operators and weighted fusion difference graph are shown in formula (1), formula (2), and formula (3): Where α and β represent the weight coefficients of LR and MR operators, respectively.

3. The SAR image change detection method based on weighted fusion wavelet transform and improved residual network according to claim 1 is characterized by: The method for reconstructing and generating the discrete wavelet fusion difference map described in step (3) is: Step 1: For the decomposed weighted fusion difference map, the low-frequency coefficient part is fused using the weighted average method. The weight factor is determined by the absolute value of the two low-frequency coefficient matrices to ensure that the background is smoothed while retaining important information. The fusion rule of the low-frequency coefficient part is defined as: Among them, χ and γ represent and The weight factor, and Represent the low-frequency coefficient parts of the log-ratio difference map and the mean-ratio difference map, respectively. The fusion coefficients χ and γ are defined as: in, It represents the absolute value of the average value of the low-frequency region coefficients obtained after discrete wavelet decomposition of the fused difference map. The variance of the two high-frequency coefficient matrices at the same position is calculated, and the coefficient with smaller variance is selected as the fused coefficient to maintain edge and texture information. Step 2: For the decomposed weighted fusion difference map, the minimum variance criterion is used for the high-frequency coefficient part. The fusion rule for the high-frequency coefficient part is defined as: in, and Represent the high-frequency coefficient parts of the mean ratio difference map and the logarithmic ratio difference map, respectively. and Represent the variance of the two high-frequency coefficient matrices respectively, and the variance σ is defined as: Among them, D e (k) represents the number of nodes in the neighborhood window N i，j The high-frequency coefficient value at position k in the neighborhood window N is represented by u(k). i，j The average value of all high-frequency coefficients in N i,j It is a neighborhood window of size N×N, usually 3×3. Step 3: Finally, the fused low-frequency and high-frequency coefficients are recombined into a new difference map using the inverse discrete wavelet transform (IDWT). The reconstructed discrete wavelet fused difference map (DWTDI) effectively reduces the noise component in the image. The process of reconstructing the fused difference map using discrete wavelet transform is shown in Figure 3.

4. The SAR image change detection method based on weighted fusion wavelet transform and improved residual network according to claim 1 is characterized in that: The specific steps for constructing the SEPP-ResNet network structure described in step (4b), further extracting key features, and capturing contextual information at different scales are as follows: Step 1: Build SEPP-ResNet, which uses the attention mechanism to extract key features of the image and the pyramid pooling module to perform multi-scale feature extraction. SEPP-ResNet mainly consists of two modules, namely SE attention mechanism and PPM. Figure 4 shows the structure diagram of the SE attention mechanism. Step 2: After inputting the feature map, average pooling is performed on each channel to extract global features. Then, the importance of the channel is calculated using a two-layer fully connected network, and weights are generated through the sigmoid function. Finally, these weights are applied to the original feature map, so that the network pays more attention to the key features of the changed area while suppressing irrelevant information. Step 3: After the SE attention mechanism, the feature map is fed into the PPM, and multi-scale contextual information is extracted through pooling operations of different scales (1×1, 2×2, 3×3, and 6×6). Subsequently, these pooled feature maps are upsampled back to the original size and concatenated with the SE-processed feature map in the channel dimension. Finally, the fused features are classified through global average pooling and a fully connected layer.

5. The SAR image change detection method based on weighted fusion wavelet transform and improved residual network according to claim 1 is characterized in that: The step (6a) described in obtaining the pre-classification result {Ω c ,Ω i ,Ω u }, select the c and Ω u The neighborhood around the class pixel is used as the training sample, and the sample unification strategy is adopted to deal with the imbalance problem of positive and negative samples. The same number of positive and negative samples are selected to train the network.

Images