SAR image building group extraction method based on comparative learning guidance

Abstract

The invention discloses an SAR image building group extraction method based on comparative learning guidance, and the method comprises the steps: constructing a deep network architecture composed of an encoder, a decoder, a header and a CCA constraint module, carrying out the dynamic division of a feature region through CCA statistical constraint, strengthening the consistency of building boundary features, and restraining a wrong segmentation phenomenon, thereby achieving the extraction of building groups. Therefore, accurate extraction and background completion of the building area in the SAR image are realized, and the constraint capability on pixel-level misclassification, the building segmentation precision and the model generalization capability are remarkably improved. According to the method, by introducing the CCA constraint, the consistent expression of the boundary features of the building is effectively enhanced, meanwhile, the interference features causing the incorrect division of the internal small regions are inhibited, and the segmentation effect of the building in the SAR image is improved.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation technology, specifically a method for extracting building clusters from SAR images based on contrastive learning guidance. Background Technology

[0002] SAR imagery, with its all-weather, all-time imaging capabilities, has become an important means of acquiring urban spatial information. However, due to the unique imaging mechanism of SAR images, they contain a large amount of noise, shadows, and textural complexity, which poses a significant challenge to building extraction.

[0003] Traditional SAR image building extraction methods primarily rely on manually designed feature extraction methods, such as edge detection and texture analysis. While these methods can extract building information to some extent, their dependence on fixed feature patterns makes them ill-suited for complex SAR image scenes and they exhibit poor robustness to noise and background interference. In recent years, with the rapid development of deep learning technology, image segmentation methods based on convolutional neural networks (CNNs) have gradually become mainstream. These methods can automatically extract building regions by learning multi-scale features of images. However, they still have some problems when processing SAR images, such as insufficient accuracy in extracting building boundary features, leading to missegmentation and limiting segmentation accuracy and model generalization ability. Furthermore, existing SAR image building extraction models based on encoder-decoder structures suffer from insufficient constraint of pixel-level missegmentation by the segmentation loss function. Summary of the Invention

[0004] This invention proposes a SAR image building cluster extraction method based on contrastive learning guidance to address the problem of insufficient constraint on pixel-level misclassification in existing technologies and improve segmentation accuracy.

[0005] The technical solution to achieve the purpose of this invention is: a method for extracting building clusters from SAR images based on contrastive learning guidance, comprising:

[0006] Construct a training dataset, which includes building SAR images and building SAR image segmentation results as ground truth;

[0007] Construct a deep convolutional neural network with an encoder-decoder architecture that integrates CCA constraints. The encoder-decoder architecture includes a semantic segmentation branch and a CCA constraint branch. The semantic segmentation branch includes an encoder, a decoder, and a segmentation head.

[0008] The semantic segmentation branch is trained using the training dataset, and the CCA constraint branch is trained using the encoder's encoding results and the output results of the semantic segmentation branch.

[0009] The composite loss of the encoder-decoder deep convolutional neural network is calculated by backpropagation, and the parameters of the semantic segmentation branch and the CCA constraint branch are updated synchronously by an adaptive optimizer until the encoder-decoder deep convolutional neural network converges.

[0010] By removing the CCA constraint branch in the deep convolutional neural network of the encoder-decoder architecture, a SAR image building cluster extraction model is obtained. The real-time acquired SAR images of buildings are input into the SAR image building cluster extraction model for prediction, and the building areas in the image are extracted.

[0011] Compared with the prior art, the significant advantages of this invention are:

[0012] 1) This invention dynamically divides feature regions by comparing segmentation results with real labels, introduces the statistical correlation of feature distribution into the training of segmentation network, and specifically enhances the consistent expression of features within building classes, while suppressing the interference features of misclassified regions, thus solving the fundamental problem that traditional segmentation loss functions are insufficient in constraining pixel-level misclassification.

[0013] 2) This invention uses the CCA constraint module to dynamically divide features into correctly segmented regions and incorrectly segmented regions of buildings, and generates constraint loss with the optimization objective of maximizing the canonical correlation coefficient, forcing the same type of correctly / incorrectly segmented features to converge in the statistical space. This statistical constraint method enables the model to learn more robust feature representations during training, thereby enhancing the model's adaptability to different scenarios and noise levels and improving its generalization ability. Attached Figure Description

[0014] Figure 1 This is a flowchart of the SAR image building cluster extraction method based on contrastive learning guided by the present invention.

[0015] Figure 2 This is a flowchart of the multi-scale feature map extraction process.

[0016] Figure 3 This is a schematic diagram of the CCA branch structure.

[0017] Figure 4 A visualization of the segmentation results.

[0018] Figure 5 A schematic diagram showing the segmentation results of easily misclassified regions before and after introducing the CCA branch. Detailed Implementation

[0019] To enable those skilled in the art to better understand the embodiments of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and implementation methods.

[0020] like Figure 1As shown, a method for extracting building clusters from SAR images based on contrastive learning is proposed. This method constructs a deep network architecture consisting of an encoder, decoder, segmentation module, and canonical correlation analysis (CCA) constraint module. Utilizing CCA statistical constraints, it dynamically divides feature regions, strengthens the consistency of building boundary features, and suppresses missegmentation. This achieves accurate extraction of building regions and background completion in SAR images, significantly improving the ability to constrain pixel-level missegmentation, the accuracy of building segmentation, and the model's generalization ability. The method includes the following steps:

[0021] S1. Construct a training dataset, which includes building SAR images and building SAR image segmentation results as ground truth. The specific process is as follows:

[0022] Each urban area was selected as the experimental study area, SARBud data was selected as the image data source, and the DLR 2019 global building segmentation results were selected as the ground truth image. The two images were registered to ensure coordinate system consistency, and images of size [missing information - likely a specific size or value] were used. The image-label dataset is obtained by cropping the overlapping parts of the sliding window. At this time, the image data is in single-channel PNG format and the label data is in binary PNG format, with 7000 images.

[0023] S2. Construct a deep convolutional neural network with an encoder-decoder architecture that incorporates CCA constraints. The encoder-decoder architecture includes a semantic segmentation branch and a canonical correlation analysis (CCA) constraint branch. The semantic segmentation branch includes an encoder, a decoder, and a segmentation head.

[0024] In a further embodiment, the encoder module employs the Vision Transformer (ViT) model to extract multi-scale features from the input SAR image and output a multi-scale feature map. The decoder module, based on the Transformer architecture, reconstructs the spatial resolution of the feature map through upsampling operations. Finally, the segmentation head module converts the decoded feature map into a binary segmentation mask, thereby achieving accurate segmentation of the input image.

[0025] S3. Train the semantic segmentation branch using the training dataset, and simultaneously train the CCA constraint branch using the encoder's encoding results and the semantic segmentation branch's output results. The specific process is as follows:

[0026] With resolution The SAR image I is input to the encoder, and after encoding, the encoder outputs a multi-scale feature map of size I. Multi-scale feature maps are generated. These feature maps are then input to the decoder, which upsamples and reconstructs location information. The reconstructed location information is output as the upsampled result and fed into the segmentation head to obtain a binary image of the same size as the original image, capable of distinguishing between building and non-building regions. .

[0027] The segmentation results are compared with the building labels in the original SAR image. The comparison yields four types of region masks, which are as follows:

[0028]

[0029]

[0030]

[0031]

[0032] In the formula, The image is a binary image, where =1 indicates that the corresponding pixel coordinates represent a building. =0 indicates that the corresponding coordinate position is the background; The results of the ground truth building SAR image segmentation are as follows, where =1 indicates that the actual corresponding pixel coordinates are of a building. =0 indicates that the actual corresponding coordinate position is the background; To correctly segment building areas using a mask; A mask that uses the actual background but is segmented into building areas; To correctly segment the background region mask; This is a mask for areas that actually contain buildings but are divided into background areas.

[0033] Then, using geometric mapping relationships, the nearest neighbor difference method is used to map these four masks to the multi-scale feature space, resulting in four region mapping masks with the same size as the multi-scale feature map. Based on the multi-scale feature map and the four region mapping masks, the feature map sets of each of the four regions are determined.

[0034] The size of the area mask is The size of the scaling mask is The scaling ratio is , For each pixel in the scaling mask Find the corresponding point in the region mask. The specific calculation method is as follows:

[0035]

[0036]

[0037] Where round represents the rounding operation, and the corresponding point in the region mask. The pixel values ​​are the scaling mask. Pixel values:

[0038]

[0039]

[0040]

[0041]

[0042] In the formula, To correctly segment building regions (TP) after scaling; This is a mask that, when scaled up, is actually the background but segmented into building areas (FP). To correctly segment the background region (TN) after scaling; This is a mask for the area that actually contains buildings but is divided into background areas (FN) after scaling.

[0043] By performing channel copying of the scaling mask in the third dimension, four types of region mapping masks with sizes (H / 4, W / 4, C) are obtained:

[0044]

[0045]

[0046]

[0047]

[0048] In the formula, Repeat is the channel copy operation, axis=2 means that the copy is performed in the third dimension, and repeats=C means that each element is repeated C times in the channel dimension; To correctly segment building regions (TPs) after mapping; This is a mask that, after mapping, is actually the background but segmented into building areas (FP); To correctly segment the background region (TN) mask after mapping; This is a mask for areas that actually contain buildings but are divided into background areas (FN) after mapping;

[0049] After randomly sampling the features from the correctly segmented building region (TP) feature map set and the correctly segmented background region (TN) feature map set, the TP sampled feature map set and the TN sampled feature map set are obtained. The TP sampled feature map set, the TN sampled feature map set, the FP feature map set and the FN feature map set are input into the CCA constraint branch for training.

[0050] In a further embodiment, the specific method for performing random sampling is as follows:

[0051] When randomly sampling from the feature map of correctly segmented building regions (TP) and the feature map of correctly segmented background regions (TN), the sampler procedure is first defined: Let... and These represent the sets of feature maps for correctly segmented building regions (TP) and correctly segmented background regions (TN), respectively. and Random sampling is used, and the random sampler is defined as follows: ,in It is a sample The probability distribution of the selected sample is a uniform distribution.

[0052] ,

[0053] Where N is the total number of samples, and the number of samples is... The TP sampling feature map can be obtained. TN sampling feature map ;

[0054] In a further embodiment, in the CCA constraint branch, the TP sampling feature map set is... With FN feature map set Perform canonical correlation analysis between each pair of elements to obtain the set of canonical correlation coefficients. ;

[0055] TN sampling feature map set With FP feature map set Perform canonical correlation analysis between any two elements in the set to obtain the set of canonical correlation coefficients. ;

[0056] Calculate the overall average canonical correlation coefficient The CCA constraint loss is obtained. The specific steps to obtain the canonical correlation coefficient set and CCA-constrained loss are as follows: (For the TP sampling feature map set) With FN feature map set The sizes of their respective samples are as follows: and ,Pick medium sample and take medium sample , and Canonical correlation analysis was conducted between the intervals, and a total of Next, we obtain the set of canonical correlation coefficients. ;

[0057] For the TN sampling feature map set With FP feature map set The sizes of their respective samples are as follows: and ,right and Canonical correlation analysis was performed between them, and a total of Next, we obtain the set of canonical correlation coefficients. ;

[0058] The average canonical correlation coefficient between FN and TP samples Similarly, the FP and TN samples are obtained by calculating the average canonical correlation coefficient. Overall average correlation coefficient and received CCA loss. .

[0059] S4. Calculate the composite loss of the encoder-decoder deep convolutional neural network through backpropagation, and use an adaptive optimizer to synchronously update the parameters of the semantic segmentation branch and the CCA constraint branch until the encoder-decoder deep convolutional neural network converges.

[0060] Calculate the segmentation results Building labels in the original SAR image Cross-entropy loss between A composite loss function is constructed by weighting and fusing the CCA constraint loss with the standard cross-entropy loss of the segmentation network. .

[0061] The composite loss function is:

[0062]

[0063] In the formula, These are dynamically adjusted weighting coefficients.

[0064]

[0065]

[0066] N is the number of samples. These are the building labels from the original SAR image. It is the predicted value of the SAR image after being processed by a deep convolutional neural network with an encoder-decoder architecture.

[0067] In a further embodiment, the adaptive optimizer synchronously updates the parameters of the encoder-decoder backbone network and the CCA constraint module, as follows:

[0068] The parameters are trained using a constructed encoder-decoder architecture that incorporates CCA constraints, fed with the training dataset. The optimizer employs the adaptive optimization algorithm Adam. The Adam optimizer formula is as follows:

[0069] initialization:

[0070]

[0071]

[0072]

[0073]

[0074]

[0075]

[0076] in Indicates the current parameter. Indicates the current gradient. This represents the first-order moment estimate. This represents the first-moment estimate after bias correction. This represents the first-order moment estimate. This represents the first-moment estimate after bias correction. Indicates the learning rate. Indicates the first-order decay rate. ϵ represents the second-order decay rate and the stability constant.

[0077] The optimizer's internal parameter momentum is set to 0.9, the initial learning rate is set, and an exponential decay learning strategy is used. The batch size and epochs for model training are set, and the training evaluation metric is set to MIoU. The MIoU calculation formula is as follows:

[0078]

[0079] S5. Remove the CCA constraint branch from the model, retaining only the encoder-decoder branch as the building semantic segmentation inference model. Input the SAR image I from which the building region needs to be extracted into the encoder-decoder branch. The encoder obtains multi-scale feature maps of the input image, and the decoder upsamples these feature maps to generate the final binary image for building segmentation, which is the building segmentation result of the input image. Specifically:

[0080] The model trained using this architecture is input to a SAR image for prediction, extracting built-up areas from the SAR image. By removing the CCA constraint loss from the model, the output of the decoder module is used as the model's prediction result. The trained model is tested using a test set, with the mean intersection-union ratio (MioU) as the testing metric. The model with the highest MioU is selected as the optimal model. The results are compared using a validation set; Table 1 shows the results. Figure 3 A visualization of the model segmentation results.

[0081] The SAR image I from which the building area needs to be extracted is input into the SAR image building group extraction model. The encoder obtains the multi-scale feature map of the input image, and the decoder upsamples the multi-scale feature map to generate the final binary image of building area segmentation, which is the building area segmentation result of the input image.

[0082] Figure 4 This demonstrates the significant improvement of the model in building extraction tasks after introducing CCA constraint branches. Compared to the model without CCA constraint branches, this model can more accurately identify building details and effectively avoid misclassification of details. Figure 5 In the diagram, white represents the segmentation results before the introduction of CCA constraints, while pink represents the newly added correctly extracted regions after the introduction of CCA constraint branches. The results show that the introduction of CCA constraint branches significantly improves the match between the model's extraction results and the original labels, substantially enhancing the accuracy and reliability of building extraction.

[0083] Recall(%) IoU(%) Original model 77.30 66.35 Introducing the CCA constraint branch model 79.56 67.97

[0084] Table 1

Claims

1. A method for extracting building clusters from SAR images based on contrastive learning guidance, characterized in that, include: Construct a training dataset, which includes building SAR images and building SAR image segmentation results as ground truth; Construct a deep convolutional neural network with an encoder-decoder architecture that integrates CCA constraints. The encoder-decoder architecture includes a semantic segmentation branch and a CCA constraint branch. The semantic segmentation branch includes an encoder, a decoder, and a segmentation head. The semantic segmentation branch is trained using the training dataset, and the CCA constraint branch is trained using the encoder's encoding results and the output results of the semantic segmentation branch. The composite loss of the encoder-decoder deep convolutional neural network is calculated by backpropagation, and the parameters of the semantic segmentation branch and the CCA constraint branch are updated synchronously by an adaptive optimizer until the encoder-decoder deep convolutional neural network converges. By removing the CCA constraint branch in the deep convolutional neural network of the encoder-decoder architecture, a SAR image building cluster extraction model is obtained. The real-time acquired SAR images of buildings are input into the SAR image building cluster extraction model for prediction, and the building areas in the image are extracted.

2. The method for extracting building clusters from SAR images based on contrastive learning as described in claim 1, characterized in that, The specific process of training the semantic segmentation branch using the training dataset, and simultaneously training the CCA constraint branch using the encoder's encoding results and the semantic segmentation branch's output results, is as follows: The building SAR images in the training dataset are input into the encoder to obtain multi-scale feature maps; Multi-scale feature maps are input to the decoder, which then upsamples and reconstructs the location information. The reconstructed location information is input into the segmentation head to obtain a binary image with the same size as the original image, which can distinguish between building and non-building areas. By comparing the segmentation results of the binary image with the ground-value building SAR image, four types of region masks are obtained. Using geometric mapping relationships, the nearest neighbor interpolation method is used to map four types of masks to a multi-scale feature space, resulting in four region mapping masks with the same size as the multi-scale feature map. Based on the multi-scale feature map and the four region mapping masks, the feature map sets of the four regions are determined. The feature map sets of the four regions are as follows: the feature map set of correctly segmented building regions, the feature map set of correctly segmented background regions, the feature map set of regions that are actually background but segmented into buildings, and the feature map set of regions that actually contain buildings but segmented into background. After randomly sampling the features from the set of feature maps of correctly segmented building areas and the set of feature maps of correctly segmented background areas, TP sampled feature map set and TN sampled feature map set are obtained respectively. The TP sampled feature map set, TN sampled feature map set, and feature map sets of actual background but segmented into building areas and actual building areas but segmented into background areas are input into the CCA constraint branch for training.

3. The method for extracting building clusters from SAR images based on contrastive learning as described in claim 2, characterized in that, The four types of region masks are as follows: ； ； ； ； In the formula, The image is a binary image, where =1 indicates that the corresponding pixel coordinates represent a building. =0 indicates that the corresponding coordinate position is the background; The results of the ground truth building SAR image segmentation are as follows, where =1 indicates that the actual corresponding pixel coordinates are of a building. =0 indicates that the actual corresponding coordinate position is the background. To correctly segment building area masks, A mask that uses the actual background but is segmented into building areas; To correctly segment the background region mask; This is a mask for areas that actually contain buildings but are divided into background areas.

4. The method for extracting building clusters from SAR images based on contrastive learning as described in claim 2, characterized in that, Using geometric mapping relationships, the nearest neighbor interpolation method is employed to map four types of region masks onto a multi-scale feature space, resulting in four region mapping masks with the same size as the multi-scale feature map. The specific steps are as follows: Let the size of the region mask be... The size of the scaling mask is The scaling ratio is , ; For each pixel in the scaling mask Find the corresponding point in the region mask. The specific calculation method is as follows: ； ； Where round represents the rounding operation, and the corresponding point in the region mask. The pixel values ​​are the scaling mask. Pixel values: ； ； ； ； In the formula, To correctly segment building areas after scaling; This is a mask that, when scaled up, is actually the background but segmented into building areas; To correctly segment the background area after scaling; This is a mask for a scaled-down area that actually contains buildings but is divided into a background area. By performing channel copying of the scaling mask in the third dimension, four types of region mapping masks with sizes (H / 4, W / 4, C) are obtained: ； ； ； ； In the formula, Repeat is the channel copy operation, axis=2 indicates that copying is performed in the third dimension, and repeatats=C indicates that each element is repeated C times in the channel dimension. To correctly segment building areas after mapping; This is a mask that, after mapping, actually represents the background but is segmented into building areas. To ensure correct segmentation of the background region after mapping; This is a mask for areas that actually contain buildings but are segmented into background areas after mapping.

5. The method for extracting building clusters from SAR images based on contrastive learning as described in claim 2, characterized in that, The specific formula for determining the feature map sets of the four regions based on the multi-scale feature maps and the four types of region mapping masks is as follows: ， ， ， ， In the formula, the Compress() function removes all pixels with a value of 0, resulting in a pixel with a size of [size missing]. The feature map set; To correctly segment the feature map set of building areas; A set of feature maps that are actually backgrounded but segmented into building areas; To correctly segment the background region feature map set; This is a set of feature maps representing areas with actual buildings but segmented into background areas. This is a multi-scale feature map.

6. The method for extracting building clusters from SAR images based on contrastive learning as described in claim 2, characterized in that, The set of feature maps of correctly segmented building areas and the set of feature maps for correctly segmenting background regions The specific method for random sampling of features in the data is as follows: The set of feature maps of correctly segmented building areas and the set of feature maps for correctly segmenting background regions Sampling is performed using a random sampler, which is... ,in It is a sample The probability distribution of the selected sample is a uniform distribution. ， Where N is the total number of samples, and the number of samples is... ; Obtain the TP sampling feature map set With TN sampling feature map set .

7. The method for extracting building clusters from SAR images based on contrastive learning as described in claim 2, characterized in that, Simultaneously, the encoding results of the encoder and the output results of the semantic segmentation branch are used to train the CCA constraint branch. The specific process is as follows: TP sampling feature map set A set of feature maps of areas with actual buildings but segmented into background areas. Perform canonical correlation analysis between each pair of elements to obtain the first set of canonical correlation coefficients. ; TN sampling feature map set A set of feature maps with actual backgrounds but segmented into building areas. Perform canonical correlation analysis between each pair of elements in the set to obtain the second canonical correlation coefficient set. ; Based on the first set of canonical correlation coefficients With the second set of canonical correlation coefficients To obtain the overall average canonical correlation coefficient The CCA constraint loss is obtained. .

8. The method for extracting building clusters from SAR images based on contrastive learning as described in claim 7, characterized in that, TP sampling feature map set A set of feature maps of areas with actual buildings but segmented into background areas. Perform canonical correlation analysis between each pair of elements to obtain the first set of canonical correlation coefficients. The specific steps are as follows: For TP sampling feature map set A set of feature maps of areas with actual buildings but segmented into background areas. The sizes of their respective samples are as follows: and ,Pick medium sample and take medium sample , and Canonical correlation analysis was conducted between the intervals, and a total of Next, the first set of canonical correlation coefficients was obtained. ; TN sampling feature map set A set of feature maps with actual backgrounds but segmented into building areas. Perform canonical correlation analysis between each pair of elements in the set to obtain the second canonical correlation coefficient set. The specific steps are as follows: For the TN sampling feature map set A set of feature maps with actual backgrounds but segmented into building areas. The sizes of their respective samples are as follows: and ,right and Canonical correlation analysis was performed between them, and a total of Next, the second set of canonical correlation coefficients was obtained. .

9. The method for extracting building clusters from SAR images based on contrastive learning as described in claim 8, characterized in that, Based on the first set of canonical correlation coefficients With the second set of canonical correlation coefficients To obtain the overall average canonical correlation coefficient The specific steps are as follows: Calculate the first average canonical correlation coefficient: ; Calculate the second average canonical correlation coefficient: ; Calculate the overall average correlation coefficient .

10. The method for extracting building clusters from SAR images based on contrastive learning as described in claim 9, characterized in that, The composite loss function of the encoder-decoder architecture deep convolutional neural network is: ； In the formula, The weighting coefficients are dynamically adjusted. For CCA constraint loss, The standard cross-entropy loss for the segmentation network is as follows: ； In the formula, N is the sample size. These are the building labels from the original SAR image. It is the predicted value of the SAR image after being processed by a deep convolutional neural network with an encoder-decoder architecture.

Images