A remote sensing image semantic segmentation method based on weighted combination loss

Abstract

This invention discloses a semantic segmentation method for remote sensing images based on weighted combined loss, belonging to the fields of remote sensing image processing and computer vision technology. The method includes: acquiring remote sensing image data to be segmented and preprocessing it; inputting the preprocessed image into an improved U-Net network model; performing multi-level feature extraction through an encoder, and multi-scale feature extraction and fusion through a hollow spatial pyramid pooling module; performing feature reconstruction and segmentation prediction through a decoder; calculating the difference between the predicted result and the true label using a weighted combined loss function, which is composed of cross-entropy loss, Dice loss, and Focal loss combined according to a preset weight ratio; updating the model parameters according to the loss value until convergence. This invention effectively solves the problems of class imbalance, insufficient boundary segmentation accuracy, and difficulty in small target recognition in semantic segmentation of remote sensing images, significantly improving segmentation accuracy and model robustness.

Description

Technical Field

[0001] This invention belongs to the fields of remote sensing image processing and computer vision technology, and particularly relates to a semantic segmentation method for remote sensing images based on weighted combination loss. Background Technology

[0002] With the rapid development of Earth observation technology, the spatial, temporal, and spectral resolutions of remote sensing sensors have continuously improved, resulting in remote sensing image data characterized by large scale, rapid updates, and rich information. Semantic segmentation, as a typical pixel-level dense prediction task, classifies each pixel in an image to achieve a refined representation of ground feature information, and has significant application value in fields such as urban planning, land use monitoring, disaster assessment, and ecological environmental protection.

[0003] Traditional remote sensing image interpretation methods mainly rely on manual interpretation or rule-based image processing techniques, such as thresholding, edge detection, and region growing. These methods typically require manual feature design and parameter adjustment, and while they are effective in simple scenarios, they generally suffer from low efficiency, high subjectivity, and poor generalization ability. Especially in high-resolution remote sensing imagery, due to the complexity of ground features, significant scale differences, and severe background interference, traditional methods struggle to meet practical application requirements.

[0004] With the development of deep learning technology, Convolutional Neural Networks (CNNs) have achieved remarkable results in the field of computer vision and are gradually being introduced into remote sensing image processing tasks. The U-Net model, as a typical encoder-decoder structure, effectively fuses shallow detail information with deep semantic information through a skip connection mechanism, demonstrating outstanding performance in detail recovery. The DeepLabV3+ model, through dilated convolution and dilated spatial pyramid pooling (ASPP) modules, achieves multi-scale feature extraction and possesses stronger semantic expressive capabilities in complex scenes.

[0005] Regarding loss functions, existing techniques primarily employ cross-entropy loss as the main loss function for semantic segmentation tasks. Cross-entropy loss guides model parameter optimization by calculating the difference between the predicted probability distribution and the true labels. However, in remote sensing image semantic segmentation tasks, due to the extremely uneven distribution of land cover categories—background pixels typically occupying the majority while target category pixels such as roads and buildings account for a smaller proportion—this category imbalance causes the model to overemphasize the majority class during training, neglecting the minority class, thus affecting the segmentation accuracy of small targets.

[0006] Therefore, there is an urgent need for a semantic segmentation method for remote sensing images that can effectively solve class imbalance, improve boundary segmentation accuracy, and enhance the ability to identify small targets. Summary of the Invention

[0007] The purpose of this invention is to provide a remote sensing image semantic segmentation method based on weighted combination loss, which solves the problems of class imbalance, insufficient boundary segmentation accuracy, and difficulty in small target identification in the existing remote sensing image semantic segmentation technology.

[0008] To achieve the above objectives, this invention provides a remote sensing image semantic segmentation method based on weighted combination loss, comprising the following steps: S1. Obtain the remote sensing image data to be segmented, and preprocess the remote sensing image data, including normalization and data augmentation operations. S2. Input the preprocessed remote sensing image data into the improved U-Net network model for feature extraction; S3. Multi-level feature extraction is performed on the input remote sensing image data through the encoder to obtain feature maps at different scales; S4. Multi-scale processing of feature maps of different scales output by the encoder is performed through the hollow space pyramid pooling module. S5. The multi-scale features are fused to obtain a fused feature map; S6. The decoder upsamples and reconstructs the fused feature map, and combines the skip connection features of the corresponding layer in the encoder to generate a segmentation prediction result. S7. Calculate the difference between the segmentation prediction result and the true label using a weighted combined loss function. The weighted combined loss function is composed of cross-entropy loss, Dice loss and Focal loss combined according to a preset weight ratio. S8. Update the network model parameters using the backpropagation algorithm based on the value of the weighted combined loss function.

[0009] Preferably, the improved U-Net network model in S2 includes an encoder, a decoder, and a void space pyramid pooling module disposed between the encoder and the decoder.

[0010] Preferably, the encoder contains four downsampling layers, each of which includes two 3×3 convolutional layers and one 2×2 max pooling layer, with the number of feature channels gradually increasing from 64 to 512.

[0011] Preferably, the decoder includes four upsampling layers. Each upsampling layer first doubles the size of the feature map through an upsampling operation, then fuses it with the features of the corresponding layer in the encoder through skip connections, and finally reconstructs the features through two 3×3 convolutional layers.

[0012] Preferably, the S4 dilated spatial pyramid pooling module includes multiple parallel branches, each employing dilated convolutions with different dilation rates to extract multi-scale features, as detailed below: First branch: Extracts local features using 1×1 convolution; The second branch: features are extracted using a 3×3 dilated convolution with a dilation rate of 6; The third branch: Features are extracted using 3×3 dilated convolution with a dilation rate of 12; Fourth branch: Features are extracted using 3×3 dilated convolution with a dilation rate of 18; Fifth branch: Perform global average pooling and then upsample back to the original size.

[0013] Preferably, S7 specifically includes: S71. Introduce Dice loss and Focal loss; S72. Construct a combined loss function based on the cross-entropy loss function.

[0014] Preferably, the specific details of introducing Dice loss and Focal loss in S71 are as follows: DiceLoss is introduced as an improvement technique to enhance the model's ability to focus on small target regions. DiceLoss is constructed based on the Dice coefficient, which is defined as follows: ; In the formula, For the model to the first The predicted probability of each pixel; For the corresponding real tags; The DiceLoss expression is constructed as follows: ; FocalLoss is introduced to enhance the model's ability to learn from difficult samples; the FocalLoss expression is constructed as follows: ; In the formula, This represents the model's predicted probability of the true class. Category weights; It is a regulating factor.

[0015] Preferably, in S72, a combined loss function is constructed based on the cross-entropy loss function, with the following expression: ; In the formula, The cross-entropy loss function; , and These are the weighting coefficients.

[0016] Preferably, the method further includes a boundary-weighted loss calculation step, the details of which are as follows: Boundary regions are extracted from real labels, and the boundary regions are defined as the pixels at the boundary of different land cover category labels; A first weight is assigned to boundary pixels, and a second weight is assigned to non-boundary pixels, wherein the first weight is greater than the second weight; Multiplying the weights by the weighted combined loss function yields the final boundary weighted loss.

[0017] Preferably, the remote sensing image semantic segmentation system based on the above method includes: The data preprocessing module is used to normalize and augment remote sensing images; The feature extraction module adopts an improved U-Net network model, including an encoder, a decoder, and a hole-space pyramid pooling module; A multi-scale feature fusion module is used to fuse feature information at different scales; The segmentation prediction module is used to generate pixel-level classification results; The loss calculation module uses a weighted combined loss function to calculate the difference between the prediction results and the true labels; The model optimization module is used to update network parameters based on the loss value.

[0018] Therefore, the present invention employs the aforementioned remote sensing image semantic segmentation method based on weighted combination loss, which has the following beneficial effects: (1) The combined loss function significantly improves the model’s ability to identify different categories. Through the synergistic effect of cross-entropy loss, Dice loss and Focal loss, the class imbalance problem is effectively solved. (2) Due to the Focal loss’s focus on hard-to-classify samples and the Dice loss’s optimization of region consistency, the model’s segmentation accuracy in the boundary region is significantly improved. Quantitative evaluation shows that the F1-score of the boundary region is improved, the boundary is visually clearer and smoother, and jagged edges and breakage phenomena are reduced.

[0019] (3) The combined loss function makes the model training process smoother through the synergistic effect of the three loss functions; the cross-entropy loss provides a stable gradient benchmark, the Dice loss avoids the gradient vanishing problem in the later stage of training, and the Focal loss accelerates the learning of hard-to-classify samples. (4) Due to the introduction of the ASPP module, the model’s adaptability to targets of different scales is enhanced; when tested on remote sensing images of different resolutions, the performance of the model decreases significantly less than that of the baseline model; especially when dealing with scenes with large scale changes, the model can accurately identify large buildings and small ancillary facilities at the same time, showing good scale adaptability.

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0021] Figure 1 This is a flowchart of a remote sensing image semantic segmentation method based on weighted combination loss according to the present invention; Figure 2 This is a schematic diagram of the overall technical architecture of the present invention; Figure 3 This is a diagram of the improved U-Net+ASPP structure of the present invention; Figure 4 This is a schematic diagram of the ASPP module structure of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0023] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as a process, method, system, product, or server that includes a series of steps or units, not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0024] The following is combined Figure 1 Figure 1 illustrates the embodiments of the present invention in detail.

[0025] Example 1 This embodiment provides a semantic segmentation method for remote sensing images based on weighted combination loss, which is suitable for automatic feature extraction scenarios in high-resolution remote sensing images, such as urban building segmentation and road network extraction.

[0026] The specific process is as follows: Figure 1 As shown, the method in this embodiment includes the following steps: Step S1: Data preprocessing; The input remote sensing images are normalized, mapping pixel values ​​from [0,255] to the [0,1] interval. Data augmentation is performed on the training data, including random rotation (angle range from -30° to +30°), random flipping (horizontal and vertical flipping), and random cropping (cropping size of 512×512 pixels), to increase data diversity.

[0027] Step S2: Feature extraction; The preprocessed image is input into the improved U-Net network model. For example... Figure 3 As shown, the model includes an encoder, a decoder, and a Spatial Pyramid Pooling (ASPP) module positioned between the encoder and the decoder.

[0028] The encoder contains four downsampling layers, each consisting of two 3×3 convolutional layers (each followed by a BatchNorm and ReLU activation function) and a 2×2 max-pooling layer. The number of feature channels gradually increases from 64 to 128, 256, and 512. Through downsampling, the spatial resolution of the image gradually decreases, while the level of abstraction of semantic information gradually increases.

[0029] Step S3: Multi-scale feature extraction; The high-level features output from the encoder (512 channels, spatial resolution 1 / 16 of the input image) are input into the ASPP module. For example... Figure 4 As shown, the ASPP module contains 5 parallel branches: The first branch extracts local features using a 1×1 convolution, with 256 output channels. The second branch extracts features using a 3×3 dilated convolution with a dilation rate of 6, also with 256 output channels. The third branch extracts features using a 3×3 dilated convolution with a dilation rate of 12, with 256 output channels. The fourth branch extracts features using a 3×3 dilated convolution with a dilation rate of 18, with 256 output channels. The fifth branch performs global average pooling, then adjusts the number of channels to 256 using a 1×1 convolution, and finally upsamples back to the original size using bilinear interpolation.

[0030] By using dilated convolutions with different dilation rates, the ASPP module can effectively expand the receptive field and capture multi-scale contextual information without increasing the number of parameters.

[0031] Step S4: Feature fusion; The five branch features output by the ASPP module are concatenated along the channel dimension to obtain a feature map with 1280 channels. Then, a 1×1 convolutional layer is used to reduce the number of channels to 256, resulting in the fused multi-scale features.

[0032] Step S5: Feature decoding; The fused features are then input into the U-Net decoder. The decoder consists of four upsampling layers. Each upsampling layer first doubles the size of the feature map using transposed convolution or bilinear interpolation, then concatenates it with the features from the corresponding layer in the encoder via skip connections. Finally, it reconstructs the features using two 3×3 convolutional layers. Through skip connections, the decoder can fuse shallow detail information from the encoder, improving the accuracy of segmentation boundaries.

[0033] Step S6: Segmentation prediction; The decoder's final output feature map (64 channels) is mapped to the class space through a 1×1 convolutional layer. For a C-class classification problem, the output has C channels, with each channel representing the predicted probability of the corresponding class. The Softmax activation function is then used to obtain the probability distribution of each pixel belonging to each class.

[0034] Step S7: Loss Calculation; The difference between the predicted results and the true labels is calculated using a weighted combined loss function.

[0035] The specific details of introducing Dice loss and Focal loss are as follows: DiceLoss is introduced as an improvement technique to enhance the model's ability to focus on small target regions. DiceLoss is constructed based on the Dice coefficient, which is defined as follows: ; In the formula, For the model to the first The predicted probability of each pixel; For the corresponding real tags; The DiceLoss expression is constructed as follows: ; FocalLoss is introduced to enhance the model's ability to learn from difficult samples; the FocalLoss expression is constructed as follows: ; In the formula, This represents the model's predicted probability of the true class. Category weights; It is a regulating factor.

[0036] Based on the cross-entropy loss function, a combined loss function is constructed, as shown below: ; In the formula, The cross-entropy loss function; , and These are the weighting coefficients.

[0037] Step S8: Model optimization; The Adam optimizer was used with an initial learning rate of 0.001, a batch size of 8, and a training duration of 100 epochs. A learning rate decay strategy was employed, reducing the learning rate to 0.1 every 30 epochs. Based on the calculated total loss, the model parameters were updated using the backpropagation algorithm, and training was iterated multiple times until convergence.

[0038] Example 2 This embodiment focuses on the automatic building extraction task from high-resolution urban remote sensing imagery, and optimizes the method in Embodiment 1 accordingly. The overall technical architecture is as follows: Figure 2 As shown.

[0039] Technical configuration adjustments: (1) The weight configuration of the combined loss function is adjusted to =0.8, =1.2, =0.3. Increase the weights of the Dice loss to improve the overlap of building areas.

[0040] (2) The void ratio of ASPP modules is adjusted to 4, 8, and 12 to better adapt to medium-sized buildings.

[0041] (3) Introduce a boundary refinement module in the decoder. After each upsampling layer, add a boundary refinement branch. This branch extracts boundary features through two 3×3 convolutional layers and then adds them element-wise with the features of the main branch.

[0042] Boundary-weighted loss: To further improve the accuracy of boundary segmentation, this embodiment adds a boundary weighting term to the combined loss function: (1) Extract the boundary region from the real label. The boundary is defined as the intersection pixel of the building label and the background label, which is achieved by the morphological gradient operator.

[0043] (2) Assign a weight of 2.0 to boundary pixels and a weight of 2.0 to non-boundary pixels. for 1.0.

[0044] (3) Multiply the weights by the combined loss function to obtain the final boundary weighted loss.

[0045] Post-processing optimization: Post-processing optimization of the probability map output by the model: (1) Morphological opening operation: using 3×3 structuring elements to remove small noise areas; (2) Morphological closing operation: using 3×3 structuring elements to fill the voids inside the building; (3) Connected component analysis: removing areas with an area of ​​less than 100 pixels.

[0046] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for semantic segmentation of remote sensing images based on weighted combination loss, characterized in that, Includes the following steps: S1. Obtain the remote sensing image data to be segmented, and preprocess the remote sensing image data, including normalization and data augmentation operations. S2. Input the preprocessed remote sensing image data into the improved U-Net network model for feature extraction; S3. Multi-level feature extraction is performed on the input remote sensing image data through the encoder to obtain feature maps at different scales; S4. Multi-scale processing of feature maps of different scales output by the encoder is performed through the hollow space pyramid pooling module. S5. The multi-scale features are fused to obtain a fused feature map; S6. The decoder upsamples and reconstructs the fused feature map, and combines the skip connection features of the corresponding layer in the encoder to generate a segmentation prediction result. S7. Calculate the difference between the segmentation prediction result and the true label using a weighted combined loss function. The weighted combined loss function is composed of cross-entropy loss, Dice loss and Focal loss combined according to a preset weight ratio. S8. Update the network model parameters using the backpropagation algorithm based on the value of the weighted combined loss function. 2.The method of claim 1, wherein, The improved U-Net network model in S2 includes an encoder, a decoder, and a void space pyramid pooling module positioned between the encoder and decoder. 3.The method of claim 2, wherein, The encoder contains four downsampling layers, each of which includes two 3×3 convolutional layers and one 2×2 max pooling layer, with the number of feature channels gradually increasing from 64 to 512.

4. The method of claim 3, wherein, The decoder contains four upsampling layers. Each upsampling layer first doubles the size of the feature map through an upsampling operation, then fuses it with the features of the corresponding layer in the encoder through skip connections, and finally reconstructs the features through two 3×3 convolutional layers.

5. The remote sensing image semantic segmentation method based on weighted combination loss according to claim 4, characterized in that, The S4 dilated spatial pyramid pooling module includes multiple parallel branches, each employing dilated convolutions with different dilation rates to extract multi-scale features. The specific details are as follows: First branch: Extracts local features using 1×1 convolution; The second branch: features are extracted using a 3×3 dilated convolution with a dilation rate of 6; The third branch: Features are extracted using 3×3 dilated convolution with a dilation rate of 12; Fourth branch: Features are extracted using 3×3 dilated convolution with a dilation rate of 18; Fifth branch: Perform global average pooling and then upsample back to the original size.

6. The remote sensing image semantic segmentation method based on weighted combination loss according to claim 4, characterized in that, S7 specifically includes: S71. Introduce Dice loss and Focal loss; S72. Construct a combined loss function based on the cross-entropy loss function.

7. The remote sensing image semantic segmentation method based on weighted combination loss according to claim 6, characterized in that, The specific details of introducing Dice loss and Focal loss in S71 are as follows: DiceLoss is introduced as an improvement technique to enhance the model's ability to focus on small target regions. DiceLoss is constructed based on the Dice coefficient, which is defined as follows: ； In the formula, For the model to the first The predicted probability of each pixel; For the corresponding real tags; The DiceLoss expression is constructed as follows: ； FocalLoss is introduced to enhance the model's ability to learn from difficult samples; the FocalLoss expression is constructed as follows: ； In the formula, This represents the model's predicted probability of the true class. Category weights; It is a regulating factor.

8. The remote sensing image semantic segmentation method based on weighted combination loss according to claim 7, characterized in that, In S72, a combined loss function is constructed based on the cross-entropy loss function, as expressed below: ； In the formula, The cross-entropy loss function; , and These are the weighting coefficients.

9. The remote sensing image semantic segmentation method based on weighted combination loss according to claim 8, characterized in that, The method also includes a boundary-weighted loss calculation step, the details of which are as follows: Boundary regions are extracted from real labels, and the boundary regions are defined as the pixels at the boundary of different land cover category labels; A first weight is assigned to boundary pixels, and a second weight is assigned to non-boundary pixels, wherein the first weight is greater than the second weight; Multiplying the weights by the weighted combined loss function yields the final boundary weighted loss.

10. The remote sensing image semantic segmentation method based on weighted combination loss according to claim 9, characterized in that, The remote sensing image semantic segmentation system based on the above method includes: The data preprocessing module is used to normalize and augment remote sensing images; The feature extraction module adopts an improved U-Net network model, including an encoder, a decoder, and a hole-space pyramid pooling module; A multi-scale feature fusion module is used to fuse feature information at different scales; The segmentation prediction module is used to generate pixel-level classification results; The loss calculation module uses a weighted combined loss function to calculate the difference between the prediction results and the true labels; The model optimization module is used to update network parameters based on the loss value.

Images