A method, system, equipment, and medium for ultra-high resolution SAR building extraction based on attention and dilated convolution embedding UNet3Plus.
By introducing the UNet3Plus model with attention and dilated convolution embedding, the challenge of building extraction in ultra-high resolution SAR imagery is solved, achieving accurate identification and clear edge extraction of small-scale buildings, and improving the model's adaptability and segmentation performance.
Patent Information
- Application Number
- CN202410390026.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-02
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-04-02
AI Technical Summary
Existing SAR building extraction methods struggle to accurately identify building outlines and distinguish small-scale buildings at ultra-high resolutions. Furthermore, traditional methods have poor adaptability, while deep learning models rely heavily on high-quality labeled data and have significant limitations in feature learning.
The UNet3Plus model, based on attention and dilated convolution embedding, is adopted. By combining the global feature fusion module PAM and the comprehensive feature enhancement module A2R with attention mechanism and dilated convolution, the feature fusion and learning capabilities are improved, information loss is reduced, and the accuracy and robustness of building extraction are enhanced.
It improves the clarity and extraction effect of building boundaries in ultra-high resolution SAR images, reduces the missed detection of small targets, and provides an efficient and accurate building extraction solution.
Smart Images

Figure CN118212526B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent interpretation technology of SAR images, specifically involving a method, system, device and medium for ultra-high resolution SAR building extraction based on attention and dilated convolution embedding UNet3Plus. Background Technology
[0002] Synthetic Aperture Radar (SAR) is a crucial technology in remote sensing. It primarily utilizes the synthetic aperture principle of radar for remote sensing observation, offering advantages such as all-weather, all-day operation, and freedom from regional and lighting limitations, thus possessing broad application prospects. Buildings are a common feature type in SAR imagery. Accurate segmentation of SAR buildings can play a vital role in urban planning, rural revitalization, environmental monitoring, disaster management, and infrastructure management, assisting governments and decision-makers in obtaining timely information on urban and rural planning and construction, environmental changes and protection, and disaster impacts and responses.
[0003] SAR data reflects the microstructure and physical characteristics of ground features. The echo characteristics of buildings in SAR images are affected by factors such as surface materials, surrounding ground features (e.g., tall trees), building structure, and the shooting angle, exhibiting obvious mixed pixel characteristics. Furthermore, the inherent multiplicative noise in SAR images further complicates building segmentation. Especially in ultra-high resolution SAR imagery, small buildings are more susceptible to noise and shadows, appearing as inconspicuous features in the overall image. This makes it difficult to accurately extract details of small buildings or buildings in densely populated areas, resulting in significant omissions. The diversity of building shape, size, height, and material makes building features complex and variable, affecting the clarity and accuracy of building boundaries and the overall extraction effect.
[0004] Traditional SAR building extraction methods based on image processing and manually designed features, such as thresholding, edge detection, and region growing, can handle features like shape, texture, and grayscale to some extent. However, they have poor adaptability to different situations, are easily affected by noise and interference, and have low segmentation accuracy. Deep learning-based methods have stronger feature learning and representation capabilities, showing great potential in the SAR remote sensing field. However, powerful models generally have complex structures and a large number of parameters, requiring learning and extracting high-level features from large amounts of data. Obtaining high-quality SAR image label data is challenging, especially with ultra-high resolution data. Effective and crucial feature learning is key to determining segmentation performance. Furthermore, ultra-high resolution SAR images contain finer-grained surface information, meaning they contain more diverse multi-scale buildings and similar interference features. Some deep learning models still have limitations and are difficult to apply to ultra-high resolution SAR building images.
[0005] Patent application CN112101084A discloses an automatic extraction method for building earthquake damage information based on convolutional neural networks (CNNs) using polarimetric SAR (SPAN) images. This method utilizes CNNs to extract information about hillsides and ridges, and extracts texture features from SPAN images of polarimetric SAR data. After fusing the information, the hillside and ridge portions are removed, and the removed textures are added together to increase the grayscale of the building areas. Next, texture features (correlation) are extracted from the polarimetric feature images of the polarimetric SAR data, and again, CNNs are used to extract intact buildings from the superimposed features of anisotropy and correlation, classifying the remaining building areas as collapsed buildings. This method does not belong to the binary segmentation of buildings in SAR images and relies on the texture and polarimetric features of SPAN images, as well as the texture addition feature fusion method. Therefore, there is room for improvement in terms of feature representation ability, feature fusion method, and computational efficiency.
[0006] Patent application CN114594476A discloses a method for extracting building regions using π / 4 reduced polarimetric synthetic aperture radar (SAR). This method preprocesses π / 4 reduced polarimetric SAR image data, calculates reduced polarimetric features from the preprocessed SAR image, and then calculates water body extractors, orthogonal building region extractors, and inclined building region extractors based on these features. Finally, by selecting typical regions and performing statistical analysis, thresholds for each extractor are obtained, and the extracted building regions are obtained through threshold segmentation. However, this method involves cumbersome modeling steps, and its segmentation performance can be further optimized.
[0007] In summary, existing SAR building extraction methods have limitations in feature representation, adaptation to ultra-high resolution data, and segmentation performance, particularly in accurately identifying building outlines and distinguishing small-scale buildings. Summary of the Invention
[0008] To overcome the shortcomings of the prior art, the present invention aims to provide a method, system, device, and medium for ultra-high resolution SAR building extraction based on attention and dilated convolution embedding UNet3Plus. By introducing attention mechanisms and dilated convolution characteristics into UNet3Plus, it can effectively fuse features and control attention across feature maps of different scales and resolutions, improving the model's ability to learn complex features and extract building structures of different sizes. Utilizing UNet3Plus's multi-level, multi-scale feature fusion capabilities improves the accuracy and robustness of building segmentation. By repeatedly utilizing multi-level, multi-scale features from different stages of the encoder, it minimizes information loss during feature extraction, thereby increasing the ability to accurately extract details of small buildings or densely populated areas, and improving the clarity of building boundaries and the overall building extraction effect in ultra-high resolution SAR images.
[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0010] A method for extracting ultra-high resolution SAR buildings based on attention and dilated convolution embedding UNet3Plus includes the following steps:
[0011] Step 1, collect samples:
[0012] Collect ultra-high resolution SAR image data, including building types and non-building types of different sizes and shapes. The SAR image data contains corresponding pixel-level labels. Process the SAR image data. If it is a large scene, perform data cropping at an appropriate scale to form a small image of size H*W. The processed SAR image data that includes both building types and non-building types is randomly divided into training set, validation set and test set according to the proportion.
[0013] Step 2, Data Preprocessing:
[0014] Transform the training set, validation set, and test set obtained in step 1 into a dataset sample type suitable for the MindSpore deep learning framework;
[0015] Step 3, construct a deep network model with attention and dilated convolutional embeddings in UNet3Plus:
[0016] The basic UNet3Plus consists of an encoder E, a decoder D, and full-scale skip connections. The encoder E is used to extract multi-level semantic features of the image, including low-level texture information and high-level semantic information. The decoder D is used to synthesize the multi-level semantic features obtained by the encoder E and gradually restores the high-resolution feature map through full-scale skip connections and upsampling process, and finally generates a high-resolution segmentation result.
[0017] After reusing the multi-level semantic features of encoder E for scale alignment, the feature map is dynamically adjusted by learning the feature weights at each position through an attention mechanism. Simultaneously, dilated convolution is used to adjust the receptive field size of decoder E, forming encoder E. * This ultimately forms a deep network model with attention and dilated convolution embeddings into UNet3Plus;
[0018] Step 4, Training and Monitoring the Network:
[0019] Using the training and validation sets processed in step 2, train the deep network model with attention and dilated convolution embedding UNet3Plus constructed in step 3. By monitoring the effect of the validation set during training, adjust the hyperparameters of the training using the validation set, evaluate the performance of the model, save the model parameters with better performance, until the training termination condition is met, and obtain the trained deep network model with attention and dilated convolution embedding UNet3Plus.
[0020] Step 5, Prediction Results:
[0021] The trained deep network model with attention and dilated convolution embedded in UNet3Plus obtained in step 4 is used to predict the results of the test set after step 2. When predicting the results, the threshold is set to perform binarization, and finally the building / non-building segmentation prediction results in the SAR image are obtained.
[0022] Step 6, Performance Evaluation and Visualization:
[0023] Using the building / non-building segmentation prediction results obtained in step 5 and the corresponding SAR test set image labels, the evaluation index and overall evaluation index of building / non-building categories under ultra-high resolution are calculated to evaluate the network performance. The building / non-building segmentation prediction results obtained in step 5 are visualized to form a prediction result map.
[0024] The specific process of step 2 includes:
[0025] Step 2.1: For the training set, validation set and test set obtained in Step 1, generate a list of corresponding data files, and encode them into MindRecord type data suitable for the MindSpore framework according to the list of data files.
[0026] Step 2.2: Using the list of data files generated in Step 2.1, calculate the mean and variance of the training set samples. During training and validation testing, the training set, validation set, and test set are all standardized.
[0027] Step 2.3 involves decoding the MindRecord type data obtained in Step 2.1, standardizing the decoded image data using the mean and variance obtained in Step 2.2, and performing dimensional expansion, format conversion, random shuffling, and batch processing on the decoded image data and label data to obtain iterable dataset samples, including training set, validation set, and test set.
[0028] The specific process of step 3 includes:
[0029] Step 3.1, construct the basic UNet3Plus model: the encoder E consists of 5 coding blocks, each of which contains 2 convolutional layers; the decoder D consists of 4 decoding blocks, each of which contains 1 convolutional layer; the full-scale skip connections adjust the feature map size by upsampling and downsampling and adjust the number of channels using a 3*3 convolutional structure.
[0030] Step 3.2, construct the Global Feature Fusion Module (PAM): Perform average pooling and max pooling on the output feature maps of each coding block of encoder E. For the first coding block, perform 2x pooling followed by 8x pooling. For the remaining coding blocks, perform 8x pooling and 4x pooling sequentially to form... The blended feature F after concatenating all features and aligning them f , will mix features F f By adjusting the weight ratios of features at different levels within the attention module, the discriminative and representational abilities of the features are enhanced, resulting in a fused global feature set.
[0031] Step 3.3, Construct the comprehensive feature enhancement module A 2 R: Comprehensive Feature Enhancement Module A 2 R includes the basic 3x3 convolution operation CoBR and the 3x3 dilated convolution operation ACoBR. x , where x represents the dilation rate and the attention-diffusing convolution operation A. 2 CoBR x , x represents the dilation rate and attention convolution operation ACoBR; in the comprehensive feature enhancement module A 2In R, the input features are first subjected to a basic 3x3 convolution operation, CoBR: this includes 3x3 convolution, batch normalization, and ReLU activation. After the basic convolution operation, the number of channels is doubled, yielding shallow features F0. These shallow features F0 are then fed into the next convolution operation, CoBR, to obtain features F1. Subsequently, the features are passed through 3x3 dilated convolution operations, ACoBR, with dilation rates of 2, 4, and 8. x x represents the expansion rate. At this point, the number of channels remains unchanged, yielding features F2, F3, and F4. During cascading, the number of channels doubles. Specifically, F3 and F4 are concatenated and then fed into A. 2 CoBR 4 , obtain features After being spliced with F2, it is sent to A. 2 CoBR 2 , obtain features After concatenation with F1, the data is fed into an attention-based convolutional operation (ACoBR) to obtain deep features. Finally, the shallow features F0 and the deep features are combined. The enhanced comprehensive feature F is obtained by summing the residuals. z Comprehensive Feature Enhancement Module A 2 The characteristic connectivity formula for R is expressed as:
[0032]
[0033]
[0034]
[0035]
[0036]
[0037] Among them, F -1 For the comprehensive feature enhancement module A 2 Input features of R, attention-based dilated convolution operation A 2 CoBR x This indicates that an attention mechanism is added after batch normalization, where x represents the dilation rate, corresponding to 4 and 2 respectively; similarly, the attention convolution operation AcoBR adds an attention mechanism after batch normalization. This indicates a feature concatenation operation; + indicates the addition of the residuals of corresponding elements;
[0038] Step 3.4: Replace the last layer coding block structure of the basic encoder E with the comprehensive feature enhancement module A constructed in step 3.3. 2 R, using the comprehensive feature enhancement module A 2R is used to connect encoder E and decoder D to capture broader contextual information; a 1*1 convolution is used to fuse the global features output by the global feature fusion module PAM in step 3.2. The shape is adjusted to match the feature shape output by the fourth layer coding block structure, and then added to the feature residual output by the fourth layer coding block structure to obtain feature F. -1 , feature F -1 Input into Integrated Feature Enhancement Module A 2 R forms all the connections, constituting encoder E. * And a complete network model.
[0039] The training and monitoring process in step 4 includes:
[0040] Step 4.1: During training, the model outputs predicted logical values, which are then transformed into predicted probability values after a Sigmoid operation. The network loss L is calculated using the labels corresponding to the iterable training set obtained in Step 2.3.
[0041] Step 4.2: Use the network loss L obtained in Step 4.1 and backpropagation to update the learnable parameters of the deep network model with attention and dilated convolutional embeddings UNet3Plus constructed in Step 3;
[0042] Step 4.3: After the number of training iterations meets the requirements, evaluate the model's performance using the iterable validation set obtained in Step 2.3, save the model parameters with better performance, and also save the model parameters after every fixed number of training iterations.
[0043] Step 4.4: Repeat steps 4.1 to 4.3 until the training termination condition is met; by monitoring the training process, adjusting the training process and hyperparameter settings, obtain the trained deep network model with attention and dilated convolution embeddings into UNet3Plus.
[0044] The input image has a shape of [N,1,H,W]. The channel and size information of the feature maps before and after passing through different modules in step 3 are as follows:
[0045] Table 1. Size and shape of feature maps before and after different modules.
[0046]
[0047]
[0048] The network loss L in step 4.1 is the binary cross-entropy loss L. BCE And soft Dice loss L SD The mixed loss form, with individual losses described in detail below:
[0049] Binary cross-entropy loss LBCE :
[0050]
[0051] Where N is the number of samples, y i p represents the true label of the i-th sample, where the label of the binary split is either 0 or 1. i is the probability that the i-th sample predicted by the model is of the positive class, and log represents the natural logarithm;
[0052] Soft Dice loss L SD :
[0053]
[0054] Where N is the number of samples, y i This is the true label of the i-th sample. The label for the binary split is either 0 or 1, p i It is the segmentation probability of the i-th sample predicted by the model;
[0055] The network loss L is described in detail below:
[0056]
[0057] Where α and β are L BCE and L SD The combination coefficients are two constants.
[0058] This invention also provides an ultra-high resolution SAR building extraction system based on attention and dilated convolution embedding UNet3Plus, comprising:
[0059] Sample acquisition module: used to acquire ultra-high resolution SAR image data, including building types and non-building types of different sizes and shapes. The SAR image data contains corresponding pixel-level labels. The SAR image data is processed. If it is a large scene, the data is cropped at an appropriate scale to form a small image of size H*W. The processed SAR image data that contains both building types and non-building types is randomly divided into training set, validation set and test set according to the proportion.
[0060] Data preprocessing module: used to transform the acquired training set, validation set and test set into dataset sample types suitable for the MindSpore deep learning framework;
[0061] The deep network model building module is used to build a deep network model with attention and dilated convolution embedding UNet3Plus. The basic UNet3Plus consists of encoder E, decoder D, and full-scale skip connections. Encoder E is used to extract multi-level semantic features of the image, including low-level texture information and high-level semantic information. Decoder D is used to synthesize the multi-level semantic features obtained by encoder E and gradually restores the high-resolution feature map through full-scale skip connections and upsampling process, finally generating a high-resolution segmentation result.
[0062] After reusing the multi-level semantic features of encoder E for scale alignment, the feature map is dynamically adjusted by learning the feature weights at each position through an attention mechanism. Simultaneously, dilated convolution is used to adjust the receptive field size of decoder E, forming encoder E. * This ultimately forms a deep network model with attention and dilated convolution embeddings into UNet3Plus;
[0063] Network Training and Monitoring Module: This module is used to train the constructed deep network model with attention and dilated convolution embeddings UNet3Plus using the processed training and validation sets. By monitoring the effect of the validation set during the training process, the hyperparameters of the training are adjusted using the validation set, the performance of the model is evaluated, and the parameters of the model with better performance are saved until the training termination condition is met, thus obtaining the trained deep network model with attention and dilated convolution embeddings UNet3Plus.
[0064] The result prediction module is used to predict the results of the processed test set by using a deep network model that is trained with attention and dilated convolution embedded in UNet3Plus. When predicting the results, the model is binarized by setting a threshold, and finally the building / non-building segmentation prediction results in the SAR image are obtained.
[0065] Performance evaluation and visualization module: This module is used to calculate evaluation metrics and overall evaluation metrics for building / non-building categories at ultra-high resolution using the acquired building / non-building segmentation prediction results and the corresponding SAR test set image labels, to evaluate network performance, and to visualize the acquired building / non-building segmentation prediction results to form a prediction result map.
[0066] This invention also provides an ultra-high resolution SAR building extraction device based on attention and dilated convolution embedding UNet3Plus, comprising:
[0067] Memory: A computer-readable device that stores the computer program of the above-mentioned ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus;
[0068] Processor: Used to implement the ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus when executing the computer program.
[0069] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the aforementioned ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus.
[0070] Compared with the prior art, the present invention has the following advantages:
[0071] 1. This invention targets buildings as a special ground feature in ultra-high resolution SAR imagery. It creates a dataset by organizing and segmenting the data. Considering the limitations of some models in adapting to ultra-high resolution data and improving segmentation performance, this invention is based on the UNet3Plus framework. Leveraging its relatively small parameter count and data encoding / decoding capabilities, it reduces reliance on high-quality label data while maintaining good segmentation performance.
[0072] 2. This invention designs a Global Feature Fusion (PAM) module. The PAM module further reduces information loss by repeatedly utilizing multi-level, multi-scale features from different stages of the encoder, and enhances the accuracy of extracting details of small buildings or densely populated areas from ultra-high resolution SAR images by filtering key information through an attention mechanism and strengthening the weight ratio of important information.
[0073] 3. This invention designs a comprehensive feature enhancement module A. 2 SAR images have complex ground feature characteristics and inherent noise, which increases the difficulty of building segmentation. To address this problem, the present invention designs a comprehensive feature enhancement module A. 2 R introduces channel-space attention mechanisms and dilated convolutions at different scales, enabling effective feature fusion and attention control of feature maps at different scales and resolutions at ultra-high resolution. This reduces the number of buildings missed and improves the accuracy and robustness of building extraction.
[0074] In summary, the method of this invention has significant advantages over existing technologies. By introducing an attention mechanism and dilated convolution embedding UNet3Plus, it improves the edge extraction of buildings in ultra-high resolution SAR images and reduces the missed detection of small targets, effectively addressing many challenges in SAR building extraction, including the handling of inconspicuous building characteristics, ultra-high resolution data processing, and accurate extraction of building details. Its relatively lightweight network structure, global feature fusion, and effective feature enhancement methods give this invention high portability and generalization ability, providing an efficient and accurate solution for building extraction tasks in ultra-high resolution SAR images. Attached Figure Description
[0075] Figure 1 This is a flowchart illustrating the method steps of the present invention.
[0076] Figure 2 This is a flowchart of the method for extracting buildings from ultra-high resolution SAR images according to the present invention.
[0077] Figure 3 This is a sample example of the dataset used in this invention.
[0078] Figure 4 This is a diagram showing the overall structure of the deep network model constructed using attention and dilated convolution embeddings in UNet3Plus, as described in this invention.
[0079] Figure 5 This is a comparison chart of partial test set results from embodiments of the present invention; wherein, Figure 5 (a) is the ground truth diagram. Figure 5 (b) shows the prediction results for Model_2. Figure 5 (c) is a graph showing the prediction results of the present invention.
[0080] Figure 6 This is an example of the visualization of the prediction map and label map in an embodiment of the present invention. Detailed Implementation
[0081] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.
[0082] See Figure 1 and Figure 2 A method for extracting ultra-high resolution SAR buildings based on attention and dilated convolution embedding UNet3Plus includes the following steps:
[0083] Step 1, Sample Collection:
[0084] Ultra-high resolution SAR image data is acquired, including building types and non-building types of different sizes and shapes. The SAR image data contains corresponding pixel-level labels. This SAR image data is processed; if it is a large-scale image, it is cropped at an appropriate scale to form a smaller image of size H*W. In this embodiment, to ensure the consistency of sample distribution in the training, validation, and test sets after the small image division, each set contains various data sample types. Furthermore, if the sample data is of the overlapping cropping type, it should be ensured that no data samples from the training set appear in the validation and test sets. Finally, the processed SAR image data containing both building and non-building types is randomly divided into training, validation, and test sets according to a set ratio.
[0085] The data collected in this embodiment is airborne ultra-high resolution SAR imagery with a size of 512*512. The imagery contains building types of different sizes and shapes, especially a rich variety of small-scale building types. The training set, validation set, and test set are randomly divided in a 6:2:2 ratio, while ensuring the consistency of data distribution in different sets.
[0086] Step 2, Data Preprocessing:
[0087] The ultra-high resolution SAR building image dataset obtained in step 1 is preprocessed. Based on the divided dataset containing training, validation and test sets, it is transformed into a dataset sample type suitable for the MindSpore deep learning framework.
[0088] The specific process of step 2 includes:
[0089] Step 2.1: For the training set, validation set and test set of the ultra-high resolution SAR building / non-building images obtained in Step 1, generate a list of corresponding data files. Based on the list of data files, encode them into MindRecord type data suitable for the MindSpore framework.
[0090] Step 2.2: Using the list of data files generated in Step 2.1, calculate the mean and variance of the training set samples. During training and validation testing, the training set, validation set, and test set are all standardized, and the corresponding pixel-level label data is not processed in any way.
[0091] Step 2.3 involves decoding the MindRecord type data obtained in Step 2.1, standardizing the decoded image data using the mean and variance obtained in Step 2.2, and performing dimensional expansion, format conversion, random shuffling, and batch processing on the decoded image data and label data to obtain an iterable dataset sample, including a training set, a validation set, and a test set.
[0092] Given that the sample data is an ultra-high resolution SAR image, the imaging of buildings is significantly affected by the shooting angle, i.e., the distance direction. Different shooting angles result in the shadows of buildings appearing in different directions. Data augmentation strategies commonly used in the field of natural imagery, such as rotation and flipping, will change the building features of the real scene and are no longer suitable for ultra-high resolution SAR building images. Therefore, no data augmentation operations of any kind will be performed on this data.
[0093] In this embodiment, the same standardization, dimensionality expansion, format conversion, random shuffling, and batch processing operations are used when preprocessing the training set, validation set, and test set of the dataset.
[0094] Step 3, construct a deep network model with attention and dilated convolutional embeddings in UNet3Plus:
[0095] The basic UNet3Plus consists of an encoder E, a decoder D, and full-scale skip connections. The encoder E is used to extract multi-level semantic features of the image, including low-level texture information and high-level semantic information. The decoder D is used to synthesize the multi-level semantic features obtained by the encoder E and gradually restores the high-resolution feature map through full-scale skip connections and upsampling process, and finally generates a high-resolution segmentation result.
[0096] After reusing the multi-level semantic features of encoder E for scale alignment, the feature map is dynamically adjusted by learning the feature weights at each position through an attention mechanism. Simultaneously, dilated convolution is used to adjust the receptive field size of decoder E, forming encoder E. * This ultimately forms a deep network model with attention and dilated convolution embeddings into UNet3Plus;
[0097] Figure 4 This is an overall structure diagram of the deep network model with attention and dilated convolution embedded in UNet3Plus constructed in step 3 of this embodiment.
[0098] The specific process of step 3 includes:
[0099] Step 3.1, construct the basic UNet3Plus model: the encoder E consists of 5 coding blocks, each of which contains 2 convolutional layers; the decoder D consists of 4 decoding blocks, each of which contains 1 convolutional layer; the full-scale skip connections adjust the feature map size by upsampling and downsampling and adjust the number of channels using a 3*3 convolutional structure.
[0100] Step 3.2, construct the Global Feature Fusion Module (PAM): Perform average pooling and max pooling on the output feature maps of each coding block of encoder E. Specifically, the output features of the first coding block are first pooled by 2x, then pooled by 8x, and the remaining coding blocks are pooled by 8x and then by 4x in sequence, forming... The blended feature F after concatenating all features and aligning them f This preserves more comprehensive information from the encoding stage. The mixed features F f By adjusting the weight ratios of features at different levels within the attention module, the discriminative and representational abilities of the features are further enhanced, resulting in a fused global feature set.
[0101] Step 3.3, Construct the comprehensive feature enhancement module A 2 R: The fifth-layer coding block plays a crucial role in connecting the encoder E and the decoder D. It refines the basic fifth-layer coding block structure through dilated convolution and attention mechanisms; Integrated feature enhancement module A 2 R includes the basic 3x3 convolution operation CoBR and the 3x3 dilated convolution operation ACoBR. x , where x represents the dilation rate and the attention-diffusing convolution operation A. 2 CoBR x 'x' represents the dilation rate and the attention convolution operation ACoBR. In the comprehensive feature enhancement module A... 2 In R, the input features are first subjected to a basic 3x3 convolution operation, CoBR: this includes 3x3 convolution, batch normalization, and ReLU activation. After the basic convolution operation, the number of channels is doubled, yielding shallow features F0. These shallow features F0 are then fed into the next convolution operation, CoBR, to obtain features F1. Subsequently, the features are passed through 3x3 dilated convolution operations, ACoBR, with dilation rates of 2, 4, and 8. x x represents the expansion rate. At this point, the number of channels remains unchanged, yielding features F2, F3, and F4. During cascading, the number of channels doubles. Specifically, F3 and F4 are concatenated and then fed into A. 2 CoBR 4 , obtain features After being spliced with F2, it is sent to A. 2 CoBR 2 , obtain features After concatenation with F1, the data is fed into an attention-based convolutional operation (ACoBR) to obtain deep features. Finally, the shallow features F0 and the deep features are combined. The enhanced comprehensive feature F is obtained by summing the residuals. z Comprehensive Feature Enhancement Module A 2 The characteristic connectivity formula for R is expressed as:
[0102]
[0103]
[0104]
[0105]
[0106]
[0107] Among them, F -1 For the comprehensive feature enhancement module A 2 Input features of R, attention-based dilated convolution operation A 2 CoBR x This indicates that an attention mechanism is added after batch normalization, where x represents the dilation rate, corresponding to 4 and 2 respectively; similarly, the attention convolution operation ACoBR adds an attention mechanism after batch normalization. This indicates a feature concatenation operation; + indicates the addition of the residuals of corresponding elements;
[0108] Step 3.4: Replace the last layer coding block structure of the basic encoder E with the comprehensive feature enhancement module A constructed in step 3.3. 2 R, using the comprehensive feature enhancement module A 2 R is used to connect encoder E and decoder D to capture broader contextual information; a 1*1 convolution is used to fuse the global features output by the global feature fusion module PAM in step 3.2. The shape is adjusted to match the feature shape output by the fourth layer coding block structure, and then added to the feature residual output by the fourth layer coding block structure to obtain feature F. -1 , feature F -1 Input into Integrated Feature Enhancement Module A 2 R forms all the connections, constituting encoder E. * And a complete network model.
[0109] The initial size of the sample data in this embodiment is [N, 1, 512, 512]. Table 2 shows the channel and size information of the feature maps before and after different modules in the model.
[0110] Table 2. Size and shape of feature maps before and after different modules.
[0111]
[0112]
[0113] Step 4, Network Training and Monitoring:
[0114] Using the training and validation sets from the ultra-high resolution SAR building image dataset processed in step 2, train the deep network model with attention and dilated convolutional embedding UNet3Plus constructed in step 3. By monitoring the effect of the validation set during training, adjust the training hyperparameters using the validation set, evaluate the model's performance, and save the model parameters with better performance until the training termination condition is met, thus obtaining the trained deep network model with attention and dilated convolutional embedding UNet3Plus.
[0115] The specific training process in step 4 includes:
[0116] Step 4.1: During training, the model outputs predicted logical values, which are then transformed into predicted probability values after a Sigmoid operation. The network loss L is calculated using the labels corresponding to the iterable training set obtained in Step 2.3.
[0117] Step 4.2: Use the network loss L obtained in Step 4.1 and backpropagation to update the learnable parameters of the deep network model with attention and dilated convolutional embeddings UNet3Plus constructed in Step 3;
[0118] Step 4.3: After the number of training iterations meets the requirements, evaluate the model's performance using the iterable validation set obtained in Step 2.3, save the model parameters with better performance, and also save the model parameters after every fixed number of training iterations.
[0119] Step 4.4: Repeat steps 4.1 to 4.3 until the training termination condition is met. By monitoring the training process, adjusting the training process and hyperparameter settings, obtain the trained deep network model with attention and dilated convolutional embeddings in UNet3Plus.
[0120] The network loss L in step 4.1 includes the binary cross-entropy loss L. BCE And soft Dice loss L SD The details are as follows:
[0121] Binary cross-entropy loss L BCE :
[0122]
[0123] Where N is the number of samples, y i The true label (0 or 1) of the i-th sample is used, and the label of the binary split is either 0 or 1, p i is the probability that the i-th sample predicted by the model is of the positive class, and log represents the natural logarithm;
[0124] Soft Dice loss L SD :
[0125]
[0126] Where N is the number of samples, y i p is the true label (0 or 1) of the i-th sample. i It is the segmentation probability of the i-th sample predicted by the model.
[0127] Total network loss L:
[0128]
[0129] Where α and β are the binary cross-entropy loss L BCE And soft Dice loss L SD The combination coefficients are two constants. In this embodiment, the two constant coefficients are 0.5 and 1, respectively.
[0130] Furthermore, this embodiment sets the maximum number of training iterations to 500, and employs two strategies: retaining the model parameters with the best performance metrics and saving the model parameters every 20 epochs. The optimization method and hyperparameters are adjusted based on the monitoring metrics of the training and validation sets. The best-performing model weights are obtained with an Adam optimizer of β1 = 0.9, β2 = 0.999, a base learning rate of 0.01, and a batch size of 16.
[0131] Step 5, Result Prediction:
[0132] The trained deep network model with attention and dilated convolution embedded in UNet3Plus obtained in step 4 is used to predict the results of the test set after step 2. When predicting the results, the threshold is set to perform binarization, and finally the building / non-building segmentation prediction results in the SAR image are obtained.
[0133] Step 6, Performance Evaluation and Visualization:
[0134] Using the building / non-building segmentation prediction results obtained in step 5 and the corresponding SAR test set image labels, evaluation metrics and overall evaluation metrics for building / non-building categories at ultra-high resolution are calculated, including precision, recall, F1 score, soft dice coefficient, overall accuracy (OA), Kappa coefficient, intersection-over-union ratio (IoU), mean intersection-over-union ratio (MIoU), and frequency-weighted intersection-over-union ratio (FWIoU) to evaluate network performance. The building / non-building segmentation prediction results obtained in step 5 are then visualized to form a prediction result map.
[0135] Figure 5 The image shows a comparison of the prediction results after visualization on a portion of the test set in an embodiment of the present invention. Figure 6 This is an example of the visualization of the prediction map and label map in an embodiment of the present invention.
[0136] The effects of the present invention will be further explained below with reference to simulation experiments.
[0137] 1. Simulation Experiment Conditions and Methods:
[0138] The hardware testing platform for the simulation experiment of this invention is: CPU 48 cores 192G; NPU is Ascend 910 Premium A32G;
[0139] The software platform for the simulation experiment of this invention is as follows: Operating system: EulerOS 2.8.3 (SP8); MindSpore version: MindSpore 2.1.0; Online platform: ModelArts; Notebook integrated development environment; Python 3.7.10; Hardware device: Ascend 910;
[0140] Experimental Methods: A method for building extraction from single-polarization SAR imagery in complex scenes.
[0141] 2. Simulation content and results:
[0142] Figure 3 This is a sample example of the dataset used in this invention.
[0143] Figure 4 This is a diagram showing the overall structure of the deep network model constructed using attention and dilated convolution embeddings in UNet3Plus, as described in this invention.
[0144] Figure 5 This is a comparison chart of partial test set results from embodiments of the present invention, wherein... Figure 5 (a) is the ground truth diagram. Figure 5 (b) shows the prediction results for Model_2. Figure 5 (c) is a graph showing the prediction results of the present invention.
[0145] The detailed parameters of the data used in the simulation experiment of this invention are shown in Table 3 below.
[0146] Table 3 Detailed parameters of the data used in this invention
[0147]
[0148] The images are randomly divided into training, validation, and test sets according to a set ratio, while ensuring the consistency of data distribution and the non-overlapping nature of the data sets. Figure 3 This shows a sample example of this dataset.
[0149] The relevant evaluation indicators and their definitions used in the simulation experiment for the building are as follows:
[0150] (1) Precision: The proportion of correctly predicted positive out of all positive predictions, defined as follows:
[0151]
[0152] Where i represents a positive example, j represents a negative example, and k represents the number of land cover categories, which is 2 in this simulation; p ii This represents the total number of pixels in true category i, i.e., the true instances TP; p ij This represents the total number of pixels whose true class is j and whose true class is i, i.e., false positives (FP).
[0153] (2) Recall: The proportion of all positive samples that are correctly predicted as positive, defined as follows:
[0154]
[0155] Where i represents a positive example, j represents a negative example, and k represents the number of land cover categories, which is 2 in this simulation; p ii This represents the total number of pixels of true class i, i.e., the true instances TP; p ji The total number of true class i and false negatives FN represents the total number of pixels identified as class j.
[0156] (3) F1 Score: Also called F1 value. Based on the harmonic mean of recall and precision, it is defined as follows:
[0157]
[0158] Where P is precision and R is recall;
[0159] (4) Soft Dice coefficient: also known as Soft Dice or Soft Dice similarity. It measures the similarity between the segmentation results predicted by the model and the true labels, and is defined as follows:
[0160]
[0161] Among them, P i G represents the predicted segmentation result. i This represents the true label result, k represents the number of pixels in the sample, and ε is a very small number used to avoid the case where the denominator is zero.
[0162] (5) Overall Accuracy OA: The proportion of correctly marked pixels out of the total number of pixels, defined as follows:
[0163]
[0164] Where i represents a positive example, j represents a negative example, and k represents the number of land cover categories, which is 2 in this simulation; p ii This represents the total number of pixels in the true category i, i.e., the true instance TP;
[0165] (6) Kappa coefficient: penalizes the "bias" of the model to obtain a more impartial model, defined as follows:
[0166]
[0167]
[0168] Among them, P o The proportion of correctly classified samples in each class to the total sample is equivalent to OA, a. i b represents the number of true samples in the i-th class. i This represents the number of samples predicted to be of class i, M is the total number of samples, and k represents the number of land cover categories, which is 2 in this simulation;
[0169] (7) Intersection over Union (IoU): The ratio of the intersection to the union of each predicted result and the true value, defined as follows:
[0170]
[0171] Where i represents a positive example, j represents a negative example, and k represents the number of land cover categories, which is 2 in this simulation; p ii This represents the total number of pixels of true class i, i.e., the true instances TP; p ij p represents the total number of pixels whose true class is j and whose identifier is i, i.e., false positives (FP). ji The total number of true class i and false negatives FN represents the total number of pixels identified as class j.
[0172] (8) Mean Intersection over Union (MIoU): The average sum of the ratios of the intersection and union of each type of predicted result and the true value, defined as follows:
[0173]
[0174] Where i represents a positive example, j represents a negative example, and k represents the number of land cover categories, which is 2 in this simulation; p ii This represents the total number of pixels of true class i, i.e., the true instances TP; p ji The number of true negatives (FN) represents the total number of pixels in class i and labeled as class j; ij This represents the total number of pixels whose true class is j and whose true class is i, i.e., false positives (FP).
[0175] (9) Frequency-weighted intersection-union ratio (FWIoU): An improvement on MIoU, it sets weights based on the frequency of occurrence of each category, as defined below:
[0176]
[0177] Where i represents a positive example, j represents a negative example, and k represents the number of land cover categories, which is 2 in this simulation; p ii This represents the total number of pixels of true class i, i.e., the true instances TP; p ji The number of true negatives (FN) represents the total number of pixels in class i and labeled as class j; ij This represents the total number of pixels whose true class is j and whose true class is i, i.e., false positives (FP).
[0178] The quantitative evaluation results of the simulation experiments of this invention on the test set are shown in Tables 4 and 5. Model_1 represents the method proposed in this invention, and its specific structure is as follows: Figure 4 As shown. Model_2 is a supervised method based on a convolutional neural network architecture. Both models use image patches from the same SAR imagery as training data to demonstrate the effectiveness of the invention. Figure 5 The results of a portion of the test set from the simulation experiment are shown in a comparative chart.
[0179] Table 4. Quantitative simulation evaluation results of building segmentation on the test set.
[0180]
[0181] Table 5 shows some overall quantitative simulation evaluation results of this invention on the test set.
[0182] Evaluation indicators Method of the present invention OA 0.9581 MIoU 0.8075 FWIoU 0.9260
[0183] 3. Analysis of experimental results:
[0184] Through in-depth analysis of the experimental results of ultra-high resolution SAR building segmentation, this invention demonstrates significant advantages. Firstly, by comparing some quantitative evaluation results in Table 4, the method of this invention (Model_1) shows a significant improvement over the supervised method (Model_2) based on a convolutional neural network architecture in multiple evaluation metrics. For example, the model of this invention achieves higher values in metrics such as Precision, Recall, F1 Score, Soft Dice, IoU, and Kappa. Precision, Recall, F1 Score, Soft Dice, and IoU describe the segmentation performance for the specific category of buildings. Recall is improved to 0.8407, effectively reducing the omission of buildings. The remaining metrics, Precision, F1 Score, Soft Dice, and IoU, are all improved by 5%-8% compared to Model_2. Simultaneously, the Kappa coefficient is improved by more than 6%, resulting in a more impartial and accurate model. These results indicate that the method of this invention achieves a more significant performance improvement in building segmentation tasks.
[0185] Further observation of the overall quantitative evaluation results in Table 5 reveals that the present invention achieved satisfactory performance on the test set. Its OA reached 0.9581, MIoU reached 0.8075, and FWIoU reached 0.9260, demonstrating excellent overall performance. These results show that the method of the present invention has high accuracy and robustness in handling ultra-high resolution SAR building segmentation tasks, effectively extracting building outlines and accurately classifying them.
[0186] By comparison Figure 5 The experimental results visualization is shown. Figure 6 The s-figure used in this invention; wherein, Figure 5 (a) is the ground truth diagram. Figure 5 (b) shows the prediction results for Model_2. Figure 5 (c) This image shows the prediction results of the present invention. The superiority of the present invention's method (Model_1) in building segmentation is highlighted in red, indicating the main differences. In the first set of prediction results, the model of the present invention can more accurately capture the boundaries and details of buildings, significantly improving the prediction effect of building boundaries. In the second set of prediction results, the present invention has a stronger ability to distinguish buildings and similar features in the image, achieving a more accurate segmentation effect. In the third set of prediction results, the present invention effectively improves the extraction effect of small-sized buildings. In contrast, the prediction results of Model_2 have a certain degree of blurring, error, and missed detection. This indicates that the present invention's method can more effectively process ultra-high resolution SAR images and improve the accuracy and clarity of segmentation of buildings of different scales and shapes.
[0187] In summary, the ultra-high resolution SAR building segmentation method based on attention and dilated convolution embedding UNet3Plus demonstrated significant advantages in experimental results analysis. By achieving high performance indicators in both quantitative evaluation and visualization analysis, this invention provides a reliable solution for ultra-high resolution SAR image building segmentation tasks.
[0188] This invention also provides an ultra-high resolution SAR building extraction system based on attention and dilated convolution embedding UNet3Plus, comprising:
[0189] Sample acquisition module: used to acquire ultra-high resolution SAR image data in step 1, including building types and non-building types of different sizes and shapes. The SAR image data contains corresponding pixel-level labels. The SAR image data is processed. If it is a large scene, the data is cropped at an appropriate scale to form a small image of size H*W. The processed SAR image data that contains both building types and non-building types is randomly divided into training set, validation set and test set according to the proportion.
[0190] Data preprocessing module: This module is used to perform data preprocessing on the ultra-high resolution SAR building image dataset obtained in step 1 in step 2, and to convert the ultra-high resolution SAR building image dataset obtained in step 1 into a dataset sample type suitable for the MindSpore deep learning framework.
[0191] The deep network model building module is used to implement the deep network model of attention and dilated convolution embedding UNet3Plus in step 3. The basic UNet3Plus consists of encoder E, decoder D and full-scale skip connections. Encoder E is used to extract multi-level semantic features of the image, including low-level texture information and high-level semantic information. Decoder D is used to synthesize the multi-level semantic features obtained by encoder E and gradually restore the high-resolution feature map through full-scale skip connections and upsampling process, and finally generate high-resolution segmentation results.
[0192] After reusing the multi-level semantic features of encoder E for scale alignment, the feature map is dynamically adjusted by learning the feature weights at each position through an attention mechanism. Simultaneously, dilated convolution is used to adjust the receptive field size of decoder E, forming encoder E. * This ultimately forms a deep network model with attention and dilated convolution embeddings into UNet3Plus;
[0193] The network training and monitoring module is used in step 4 to train the deep network model with attention and dilated convolution embedding UNet3Plus constructed in step 3 by using the training and validation sets in the ultra-high resolution SAR building image dataset processed in step 2. By monitoring the effect of the validation set during training, the hyperparameters of the training are adjusted using the validation set, the performance of the model is evaluated, and the model parameters with better performance are saved until the training termination condition is met, and the trained deep network model with attention and dilated convolution embedding UNet3Plus is obtained.
[0194] The result prediction module is used to predict the results of the test set after step 2 using the deep network model with attention and dilated convolution embedded in UNet3Plus obtained in step 4 in step 5. When predicting the results, the binarization is performed by setting a threshold, and finally the building / non-building segmentation prediction results in the SAR image are obtained.
[0195] Performance evaluation and visualization module: This module is used in step 6 to calculate the evaluation metrics and overall evaluation metrics for building / non-building categories at ultra-high resolution using the building / non-building segmentation prediction results obtained in step 5 and the corresponding SAR test set image labels. These metrics include precision, recall, F1 score, SoftDice coefficient, overall accuracy (OA), Kappa coefficient, intersection-over-union ratio (IoU), average intersection-over-union ratio (MIoU), and frequency weight intersection-over-union ratio (FWIoU). The module evaluates network performance and visualizes the building / non-building segmentation prediction results obtained in step 5, forming a prediction result map.
[0196] This invention also provides an ultra-high resolution SAR building extraction device based on attention and dilated convolution embedding UNet3Plus, comprising:
[0197] Memory: A computer-readable device that stores the computer program of the above-mentioned ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus;
[0198] Processor: Used to implement the ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus when executing the computer program.
[0199] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the aforementioned ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus.
Claims
1. A method for ultra-high resolution SAR building extraction based on attention and dilated convolution embedding UNet3Plus, characterized by: Includes the following steps: Step 1, collect samples: Collect ultra-high resolution SAR image data, including building types and non-building types of different sizes and shapes. The SAR image data contains corresponding pixel-level labels. Process the SAR image data. If it is a large scene, perform data cropping at an appropriate scale to form a small image of size H*W. The processed SAR image data that includes both building types and non-building types is randomly divided into training set, validation set and test set according to the proportion. Step 2, Data Preprocessing: Transform the training set, validation set, and test set obtained in step 1 into a dataset sample type suitable for the MindSpore deep learning framework; Step 3, construct a deep network model with attention and dilated convolutional embeddings in UNet3Plus: The basic UNet3Plus consists of an encoder E, a decoder D, and full-scale skip connections. The encoder E is used to extract multi-level semantic features of the image, including low-level texture information and high-level semantic information. Decoder D is used to synthesize the multi-level semantic features acquired by encoder E, and gradually recovers them to a high-resolution feature map through full-scale skip connections and upsampling processes, ultimately generating a high-resolution segmentation result; After reusing the multi-level semantic features of encoder E for scale alignment, the feature map is dynamically adjusted by learning the feature weights at each position through an attention mechanism. Simultaneously, dilated convolution is used to adjust the receptive field size of decoder E, forming encoder E. * This ultimately forms a deep network model with attention and dilated convolution embeddings into UNet3Plus; where: Constructing the basic UNet3Plus model: Encoder E consists of 5 layers of coding blocks, and decoder D consists of 4 layers of decoding blocks; Construct a global feature fusion module (PAM): perform average pooling and max pooling on the output feature maps of each coding block of encoder E; Constructing a comprehensive feature enhancement module A 2 R: Comprehensive Feature Enhancement Module A 2 R includes basic 3x3 convolution operations. 3x3 dilated convolution operation , This represents the dilation rate and attention-diffusing convolution operation. , Representing dilation rate and attention convolution operation ; Replace the last layer of the basic encoder E with the comprehensive feature enhancement module A. 2 R, using the comprehensive feature enhancement module A 2 R is used to connect encoder E and decoder D, capturing broader contextual information; Step 4, Training and Monitoring the Network: Using the training and validation sets processed in step 2, train the deep network model with attention and dilated convolution embedding UNet3Plus constructed in step 3. By monitoring the effect of the validation set during training, adjust the hyperparameters of the training using the validation set, evaluate the performance of the model, save the model parameters with better performance, until the training termination condition is met, and obtain the trained deep network model with attention and dilated convolution embedding UNet3Plus. Step 5, Prediction Results: The trained deep network model with attention and dilated convolution embedded in UNet3Plus obtained in step 4 is used to predict the results of the test set after step 2. When predicting the results, the threshold is set to perform binarization, and finally the building / non-building segmentation prediction results in the SAR image are obtained. Step 6, Performance Evaluation and Visualization: Using the building / non-building segmentation prediction results obtained in step 5 and the corresponding SAR test set image labels, the evaluation index and overall evaluation index of building / non-building categories under ultra-high resolution are calculated to evaluate the network performance. The building / non-building segmentation prediction results obtained in step 5 are visualized to form a prediction result map.
2. The ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus according to claim 1, characterized in that: The specific process of step 2 includes: Step 2.1: For the training set, validation set and test set obtained in Step 1, generate a list of corresponding data files, and encode them into MindRecord type data suitable for the MindSpore framework according to the list of data files. Step 2.2: Using the list of data files generated in Step 2.1, calculate the mean and variance of the training set samples. During training and validation testing, the training set, validation set, and test set are all standardized. Step 2.3 involves decoding the MindRecord type data obtained in Step 2.1, standardizing the decoded image data using the mean and variance obtained in Step 2.2, and performing dimensional expansion, format conversion, random shuffling, and batch processing on the decoded image data and label data to obtain iterable dataset samples, including training set, validation set, and test set.
3. The ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus according to claim 1, characterized in that: The specific process of step 3 includes: Step 3.1, construct the basic UNet3Plus model: the encoder E consists of 5 coding blocks, each of which contains 2 convolutional layers; the decoder D consists of 4 decoding blocks, each of which contains 1 convolutional layer; the full-scale skip connections adjust the feature map size by upsampling and downsampling and adjust the number of channels using a 3*3 convolutional structure. Step 3.2, construct the Global Feature Fusion Module (PAM): Perform average pooling and max pooling on the output feature maps of each coding block of encoder E. For the first coding block, perform 2x pooling followed by 8x pooling. For the remaining coding blocks, perform 8x pooling and 4x pooling sequentially to form... , …, , ; Blending all aligned features into a single composite feature , will mix features By adjusting the weight ratios of features at different levels within the attention module, the discriminative and representational abilities of the features are enhanced, resulting in a fused global feature set. ; Step 3.3, Construct the comprehensive feature enhancement module A 2 R: Comprehensive Feature Enhancement Module A 2 R includes basic 3x3 convolution operations. 3x3 dilated convolution operation , This represents the dilation rate and attention-diffusing convolution operation. , Representing dilation rate and attention convolution operation In the comprehensive feature enhancement module A 2 In R, the input features are first subjected to a basic 3x3 convolution operation. This includes 3x3 convolution, batch normalization, and ReLU activation operations. After the basic convolution operation, the number of channels is doubled to obtain shallow features. Shallow features Pass it to the next convolution operation , to obtain features Then, it undergoes 3x3 dilated convolution operations with dilation rates of 2, 4, and 8 in sequence. , This represents the expansion rate; at this point, the number of channels remains unchanged, and the features are obtained. , and When cascading and splicing, the number of channels doubles, among which, and After splicing, it is sent in , obtain features ; and After splicing, it is sent in , obtain features ; and After splicing, the data is fed into the attention convolution operation. To obtain deep features Finally, shallow features and deep features The summation of residuals yields the enhanced comprehensive features. Comprehensive Feature Enhancement Module A 2 The characteristic connectivity formula for R is expressed as: in, For the comprehensive feature enhancement module A 2 Input features of R, attention-based dilated convolution operation This indicates that an attention mechanism is added after batch normalization. This represents the dilation rate, corresponding to 4 and 2; similarly, the attention convolution operation... To incorporate an attention mechanism after batch normalization; This indicates a feature concatenation operation; This indicates the sum of the residuals of corresponding elements; Step 3.4: Replace the last layer coding block structure of the basic encoder E with the comprehensive feature enhancement module A constructed in step 3.
3. 2 R, using the comprehensive feature enhancement module A 2 R is used to connect encoder E and decoder D to capture broader contextual information; a 1*1 convolution is used to fuse the global features output by the global feature fusion module PAM in step 3.
2. The shape is adjusted to match the feature shape output by the fourth layer coding block structure, and then added to the feature residual output by the fourth layer coding block structure to obtain the feature. , will feature Input into Integrated Feature Enhancement Module A 2 R forms all the connections, constituting encoder E. * And a complete network model.
4. The ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus according to claim 1, characterized in that: The training and monitoring process in step 4 includes: Step 4.1: During training, the model outputs predicted logical values, which are then transformed into predicted probability values after a Sigmoid operation. The network loss L is calculated using the labels corresponding to the iterable training set obtained in Step 2.
3. Step 4.2: Use the network loss L obtained in Step 4.1 and backpropagation to update the learnable parameters of the deep network model with attention and dilated convolutional embeddings UNet3Plus constructed in Step 3; Step 4.3: After the number of training iterations meets the requirements, evaluate the model's performance using the iterable validation set obtained in Step 2.3, save the model parameters with better performance, and also save the model parameters after every fixed number of training iterations. Step 4.4: Repeat steps 4.1 to 4.3 until the training termination condition is met; by monitoring the training process, adjust the training process and hyperparameter settings to obtain the trained network model with attention and dilated convolution embeddings in UNet3Plus.
5. The ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus according to claim 4, characterized in that: The network loss L in step 4.1 is the binary cross-entropy loss. And soft Dice loss L SD The mixed loss form, with individual losses described in detail below: Binary cross-entropy loss : in, It is the sample size. Indicates the first The true label of each sample, the label for binary splitting is 0 or 1. It is the model's prediction of the first The probability that a sample is a positive class. Represents the natural logarithm; Soft Dice loss L SD : in, It is the sample size. It is the first The true label of each sample, the label for binary splitting is 0 or 1. It is the model's prediction of the first The segmentation probability of each sample; The network loss L is described in detail below: in, and for and L SD The combination coefficients are two constants.
6. The ultra-high resolution SAR building extraction system based on attention and dilated convolution embedding UNet3Plus as described in any one of claims 1 to 5, characterized in that: include: Sample acquisition module: used to acquire ultra-high resolution SAR image data, including building types and non-building types of different sizes and shapes. The SAR image data contains corresponding pixel-level labels. The SAR image data is processed. If it is a large scene, the data is cropped at an appropriate scale to form a small image of size H*W. The processed SAR image data that contains both building types and non-building types is randomly divided into training set, validation set and test set according to the proportion. Data preprocessing module: used to transform the acquired training set, validation set and test set into dataset sample types suitable for the MindSpore deep learning framework; Deep network model building module: used to build deep network models with attention and dilated convolution embedding UNet3Plus. The basic UNet3Plus consists of encoder E, decoder D and full-scale skip connections. Encoder E is used to extract multi-level semantic features of the image, including low-level texture information and high-level semantic information. Decoder D is used to synthesize the multi-level semantic features acquired by encoder E, and gradually recovers them to a high-resolution feature map through full-scale skip connections and upsampling processes, ultimately generating a high-resolution segmentation result; After reusing the multi-level semantic features of encoder E for scale alignment, the feature map is dynamically adjusted by learning the feature weights at each position through an attention mechanism. Simultaneously, dilated convolution is used to adjust the receptive field size of decoder E, forming encoder E. * This ultimately forms a deep network model with attention and dilated convolution embeddings into UNet3Plus; Network Training and Monitoring Module: This module is used to train the constructed deep network model with attention and dilated convolution embeddings UNet3Plus using the processed training and validation sets. By monitoring the effect of the validation set during the training process, the hyperparameters of the training are adjusted using the validation set, the performance of the model is evaluated, and the parameters of the model with better performance are saved until the training termination condition is met, thus obtaining the trained deep network model with attention and dilated convolution embeddings UNet3Plus. The result prediction module is used to predict the results of the processed test set by using a deep network model that is trained with attention and dilated convolution embedded in UNet3Plus. When predicting the results, the model is binarized by setting a threshold, and finally the building / non-building segmentation prediction results in the SAR image are obtained. Performance evaluation and visualization module: This module is used to calculate evaluation metrics and overall evaluation metrics for building / non-building categories at ultra-high resolution using the acquired building / non-building segmentation prediction results and the corresponding SAR test set image labels, to evaluate network performance, and to visualize the acquired building / non-building segmentation prediction results to form a prediction result map.
7. A high-resolution SAR building extraction device based on attention and dilated convolution embedded in UNet3Plus, characterized in that: include: Memory: A computer program for the ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus as described in any one of claims 1-5, and is a computer-readable device; Processor: Used to implement the ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus as described in any one of claims 1-5 when executing the computer program.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the ultra-high resolution SAR building extraction method based on attention and dilated convolution embedding UNet3Plus as described in any one of claims 1-5.
Citation Information
Patent Citations
Automatic polarimetric SAR building earthquake damage information extraction method based on convolutional neural network
CN112101084A
Pi / 4 compact polarized synthetic aperture radar building area extraction method
CN114594476A
SAR image complex building extraction method and system based on selection attention network
CN111860233A
High-resolution remote sensing image building extraction method based on deep learning
CN114973011A
