An image semantic segmentation model and a segmentation method

Abstract

The application discloses an image semantic segmentation model which is composed of a designed mixed attention focusing method, an attention rectification residual module (ARRM) and a mixed feature integration module (MFFM), is trained in a deep supervision mode as a whole, is reasonably provided with deformable convolution, is combined with a constructed multi-scale spatial attention module (MSP) and a double-pooling attention module (DPA) to be jointly optimized, and the problem of small target feature segmentation difficulty is solved. In order to reflect the excellent performance of the model on a downstream task, the training between different data sets is completed in a transfer learning mode, and the application range of the model is expanded. Finally, grouping convolution is added in the backbone network, the calculation cost is greatly reduced, and the problem of high training cost of the segmentation model is solved under the premise of guaranteeing the segmentation effect.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an image semantic segmentation model and segmentation method. Background Technology

[0002] Image segmentation is a key technology in image processing and an important component of computer vision. It enables higher-level analysis and understanding of images. It subdivides an image into different sub-regions, a pixel-level image resolution process. Currently, it is mainly divided into semantic segmentation, instance segmentation, and panoptic segmentation, distinguished by classifying target entities into different categories, distinct entities, and combinations thereof.

[0003] Semantic segmentation is the most fundamental and important concept in image segmentation. It groups pixels of the same type together, accurately classifying pixels and has wide applications in autonomous driving, drone autonomous navigation, medical image processing, satellite remote sensing image processing, and other digital image processing fields. Early methods, primarily traditional, achieved accurate image segmentation, mainly including threshold-based, edge-based, and region-based segmentation, as well as algorithms combining specific theoretical tools, such as morphological segmentation, hybrid genetic algorithms, and wavelet analysis and transformation-based segmentation techniques. There were also methods incorporating machine learning, such as FCM clustering and region level sets. While these traditional image segmentation methods can achieve a certain level of accuracy, they still rely on prior knowledge, have poor robustness to complex target segmentation, and weak fine-grained information extraction capabilities, making them unsuitable for real-world scenarios.

[0004] In recent years, deep learning has developed rapidly and achieved excellent performance in the field of image segmentation. Thanks to the fast and efficient execution capabilities and strong generalization performance of deep learning models, high-precision segmentation can be achieved while ensuring time and space efficiency. Classical Fully Convolutional Networks (FCNs) are designed to better determine the precise category of each pixel in an image during pixel-level segmentation and to increase the receptive field. However, due to repeated convolution stacking, they lose focus on image details. Subsequently, an increasing number of improved networks based on FCNs have emerged, such as U-Net based on an encoder-decoder structure, the DeeplabV3 series of networks based on dilated pyramid pooling (ASPP), and TransUNet, a combination of Transformer and U-Net. However, these models have some drawbacks: the U-Net series of networks achieves cross-layer information preservation through residual skip connections, integrating high and low layer feature information and breaking the situation of information loss and zero interaction between different layers, but it also introduces more information redundancy and noise. This useless information greatly reduces the model's segmentation ability; the DeeplabV3 series of networks expands the receptive field by adding dilated convolution pyramid pooling (ASPP), but its segmentation effect is poor for some small objects. For different segmentation tasks, dilated convolution introduces unnecessary receptive fields, lacks universality, and has a high overall computational cost. While Transformer-based segmentation networks are mostly combined with U-Net to achieve good results, their long training time, high computational cost, and large data requirements make them unsuitable for the biomedical image segmentation field where datasets are scarce. In summary, segmentation tasks still face some limitations in the field of application technology.

[0005] (1) Most segmentation algorithms are limited to single-domain applications. The same segmentation model does not have good universality for different segmentation targets, such as remote sensing images, automatic matting, autonomous driving, and biomedical image segmentation. It cannot simultaneously present good segmentation performance for different modal 2D, 3D and other image information.

[0006] (2) The segmentation effect of targets with large differences in shape distribution and size is significantly different. For example, when segmenting remote sensing images, the segmentation effect can be good for streets and buildings of different sizes, but the segmentation effect is poor for small targets such as cars and trees in the street, which are small in outline. They cannot be well located, and the segmentation at the boundary of the target is more blurred and difficult to process.

[0007] (3) Due to the requirements of segmentation accuracy, segmentation networks are generally deep and wide with complex internal structures, resulting in a huge number of model parameters and computational load, and extremely low efficiency in the inference process. For certain specific scenarios such as unmanned driving and satellite remote sensing observation, it is necessary to segment and evaluate targets in real time, which places high demands on the model's time and space costs and inference speed. Most existing segmentation models cannot achieve a good balance between time efficiency and segmentation accuracy. Summary of the Invention

[0008] In view of the shortcomings of the prior art, the technical problem to be solved by the present invention is: how to provide an image semantic segmentation model with low training cost, wide applicability and good segmentation performance.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0010] An image semantic segmentation model includes a feature extraction module, a feature fusion module, and a deep supervised training module; features are extracted, fused, and supervised at different levels to improve the model's performance and generalization ability.

[0011] The feature extraction module consists of six STDC backbone convolutional layers, including stages 1 to 6. Each stage includes multiple basic modules, different attention modules, and skip connections of varying sizes. The last two stages also incorporate an attention-corrected residual (ARRM) module for feature attention correction and filtering. The design approach is to use different attention mechanisms for different feature layers, rather than simply integrating channel and spatial attention modules. For low-level features, a multi-scale spatial attention mechanism (MSP) is used to extract spatial information through three different pooling layers. For high-level features, a channel attention mechanism is used to achieve targeted filtering of semantic information, and skip connections of different sizes are used to preserve features, thereby optimizing the results.

[0012] The feature fusion module inputs the stage 3 features and the integrated stage 5 features into the hybrid feature integration MFFM module for fusion, realizing the extraction and combination of high-level semantic information and low-level spatial information, thereby improving segmentation performance. In addition, considering that some segmentation targets have blurred boundaries, this model starts from the perspective of strengthening the ability to extract boundary information and adaptively adjusting the model's receptive field. Deformable convolution is introduced into the hybrid feature integration MFFM module. Deformable convolution fine-tunes the direction of the pixels after ordinary convolution, realizing the adaptive expansion of the convolution kernel. Experimental results show that the addition of deformable convolution further optimizes the overlap between the predicted boundary and the label, but excessive use will lead to negative optimization.

[0013] Inspired by UNet++ and UNet3+, the deep supervised training module, through comparative experiments, uses three levels of features for upsampling: stage 5 features, reshaped stage 6 features, and stage 3 and stage 5 features fused by the MFFM module through feature integration. These three features are used as input to the segmentation head, and a weighted average is used to obtain the final output. This deep supervision method helps to solve the gradient vanishing problem during training, while improving the accuracy and stability of segmentation.

[0014] The MSP module performs AvgPooling, Strip-Pooling, and MaxPooling on the input features along the channel dimension, respectively, to obtain rich spatial feature information of the segmentation target through these three pooling paths. The results from these three pooling paths are then stacked across channels, and the number of channels is adjusted to 1 using convolution. A sigmoid activation function is used for normalization to obtain spatial attention weights. Finally, the spatial attention weights are multiplied by the original feature matrix, and skip connections are used to preserve the original information. The addition of strip pooling allows the module to obtain horizontal and vertical contextual information around the target features. The strip-shaped pooling kernel enables the model to effectively establish long-distance dependencies between discrete regions, extracting richer information than traditional square pooling kernels. The calculation of the MSP module is as follows:

[0015] (1)

[0016] (2)

[0017] in Represents the Sigmoid activation function; Concat represents the channel concatenation operation; Conv 3×3 Represents a convolutional block consisting of 3x3 kernels; StripPool represents strip pooling; F input Represents the original feature map of the input;

[0018] The Attention-Corrected Residual ARRM module includes a DoublePooling-Attention module and residual skip connections. First, the input feature map is reduced in dimensionality through a 3×3 convolution, then passed through two parallel MaxPooling and AvgPooling layers, followed by a shared MLP layer to compress and expand the channels (ratio=4). Finally, the two outputs are element-wise summed, and after passing through a BN layer and a sigmoid activation function, the corresponding attention-corrected weight matrix is ​​obtained. This matrix is ​​then multiplied by the original feature matrix and skip-connected to it to obtain the output. This module uses two parallel pooling operations to focus on both prominent and relatively average global information, improving the model's feature extraction ability from two perspectives. Simultaneously, skip connections preserve information and prevent information loss. The calculation of the Attention-Corrected Residual ARRM module is as follows:

[0019] (3)

[0020] (4)

[0021] (5)

[0022] (6)

[0023] (7)

[0024] in This represents the original feature map as input. This represents the new feature after adjusting the number of channels through convolution. MaxPool and AvgPool represent max pooling and average pooling operations, respectively. MLP represents a shared multilayer perceptron, BN represents batch normalization, and ReLU and... Representing different activation functions, and These represent element-wise summation and dot product, respectively; DPA is a dual-pooling attention module; F MaxPool F represents the feature map obtained after max pooling. AvgPool This represents the feature map obtained after the average pooling operation.

[0025] The hybrid feature integration MFFM module includes a hybrid attention mechanism containing MSP and SE modules and deformable convolution. The input consists of low-level features and high-level semantic features. Considering the differences between low-level and high-level features, different attention mechanisms are adopted for processing. The low-level features are highly correlated with small feature points and boundary contour information. Its high resolution and small number of channels make it suitable for models to focus on important regions and spatial location information in the feature map of each channel. Low-level feature processing requires passing the input low-level features through the MSP module. High-level information features have a large number of channels and a small resolution, making them suitable for models to focus on more important channels and filter out unimportant noise. High-level information processing uses an SE module. Residual skip connections are also used to preserve information. After attention filtering, the features of the two paths are each adaptively adjusted using a deformable convolution to optimize boundary feature extraction. Then, channels are concatenated. Due to the low resolution of high-level features, they need to be upsampled to the same resolution. The concatenated features are then enhanced with non-linearity through a 1x1 convolutional block, and then concatenated again with the processed high-level features to strengthen semantic guidance. The concatenated feature matrix is ​​then extracted using a deformable convolution, and finally summed with the low-level features filtered by the MSP module before output. The calculation of the MFFM module for integrating mixed features is as follows:

[0026] (8)

[0027] (9) (10)

[0028] (11)

[0029] (12)

[0030] DConv stands for deformable convolution operation, UpS for upsampling operation, MSP for MSP module, and SE for compression and activation operation. and These are the low-level and high-level feature maps of the MFFM module, which integrate the input mixed features. and These represent the low-level and high-level features after being filtered by the hybrid attention mechanism, respectively. and This is the result of fusing features at two different scales; This is the final output feature map.

[0031] An image semantic segmentation method includes the following steps:

[0032] (a) Select image datasets with the same features as the images to be processed for batch data preprocessing. Set the corresponding window width and window level according to the segmentation target to increase the contrast of the target area. Then convert the image data into PNG format image data.

[0033] (b) Divide the image dataset obtained in step (a) into a training set, a validation set, and a test set without any overlap;

[0034] (c) Perform data augmentation on the image data in the training set, including random rotation, random horizontal flipping and random vertical flipping, to increase data diversity and prevent the model from overfitting.

[0035] (d) To address the challenges of segmenting targets with significant scale variations in different tasks (such as houses and pedestrians in remote sensing imagery) and the presence of minute targets (such as biomedical lesions) in some tasks, resulting in highly variable and complex target regions, a loss function was designed to improve the model's segmentation accuracy. This loss function employs a hybrid of DiceLoss and BCELoss, where α and β are the weight hyperparameters of DiceLoss and BCELoss, respectively, and their sum is 1. In the experiment, these values ​​were set to [0.5, 0.5] to ensure more balanced and stable model training. The expression for DiceLoss is:

[0036] (13)

[0037] The loss expression for BCELoss is:

[0038] (14)

[0039] The expression for the mixed loss of DiceLoss and BCELoss is:

[0040] (15)

[0041] N represents the total number of pixels, and C represents the total number of categories. This represents the class c in the binary GroundTruth, where the i-th pixel corresponds to the class value. This represents the predicted probability value for the corresponding category; Represents the smoothing exponent and equal to ;

[0042] (e) Obtain the image semantic segmentation model described above;

[0043] (f) The image semantic segmentation model is trained based on the loss function. The model weights are updated by backpropagation of gradients during the training process of the neural network. The model hyperparameter settings are judged based on the segmentation effect on the validation set after training. The saved model weights are updated. Finally, the model training is completed, and the image semantic segmentation effect is evaluated on the test set. During the training process of the image semantic segmentation model, the multiple predicted feature images output by the deep supervised training module are processed by formula (15) and then mean-averaged to obtain the total loss expression:

[0044] (16)

[0045] In the experiment, N=3 was set. At this time, the model achieved the best segmentation effect when the three deep supervised branches were used as outputs. Considering that the three stages of output feature maps correspond to similar levels and the feature information they contain is not significantly different, the average value was directly calculated when adding weights. After the image semantic segmentation model is trained, semantic segmentation can be performed on the image data to be processed.

[0046] Compared with existing technologies, this invention has the following advantages: Compared with traditional segmentation algorithms, this invention does not require prior knowledge provided by humans. Relying on the powerful learning ability of neural networks, the model exhibits robust performance and superior segmentation results on different segmentation tasks. Compared with other deep learning segmentation models, this invention rationally designs the CNN network, enabling the model to adaptively learn segmentation targets with different receptive field sizes, breaking the limitations of the receptive field in traditional segmentation networks. Combining various regularization methods enhances the model's feature extraction ability for segmentation difficulties such as small target features and blurred boundary features, resulting in better segmentation performance compared to other excellent segmentation models. It also performs well on different tasks. Furthermore, it is superior in terms of time and space costs and inference speed, meeting the requirements for lightweight design. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the image semantic segmentation model of the present invention;

[0048] Figure 2 This is a schematic diagram of the MSP module in this invention;

[0049] Figure 3 This is a schematic diagram of the ARRM module in this invention;

[0050] Figure 4 This is a schematic diagram of the MFFM module in this invention;

[0051] Figure 5 This is a schematic diagram of the deep supervised training module in this invention;

[0052] Figure 6This is a flowchart of the image semantic segmentation method of the present invention;

[0053] Figure 7 Visual comparison chart of different model segmentation masks provided by this invention;

[0054] Figure 8 This invention provides a visual comparison chart of features extracted using supervision branches at different depths. Detailed Implementation

[0055] like Figure 1 As shown, the image semantic segmentation model in this specific embodiment includes a feature extraction module, a feature fusion module, and a deep supervised training module;

[0056] The feature extraction module consists of six STDC backbone convolutional layers, including stages 1 to 6. Each stage includes multiple basic modules, different attention modules, and skip connections of varying sizes. The last two stages also incorporate the Attention Correction Residual (ARRM) module for feature attention correction and selection. For low-level features, the Multi-Scale Spatial Attention (MSP) module is used to extract spatial information through three different pooling layers. For high-level features, a channel attention mechanism is used to achieve targeted selection of semantic information, and skip connections of different sizes are used to preserve features, thereby optimizing the results.

[0057] The feature fusion module inputs the stage 3 features and the integrated stage 5 features into the hybrid feature integration MFFM module for fusion, realizing the extraction and combination of high-level semantic information and low-level spatial information, improving segmentation performance. Deformable convolution is introduced in the hybrid feature integration MFFM module. Deformable convolution fine-tunes the direction of the pixels after ordinary convolution, realizing the adaptive expansion of the convolution kernel.

[0058] The deep supervised training module uses three levels of features for upsampling: stage 5 features, reshaped stage 6 features, and stage 3 and stage 5 features fused by the MFFM module through feature integration. These three features are used as input to the segmentation head, and a weighted average is used to obtain the final output.

[0059] like Figure 2 As shown, the MSP module performs AvgPooling, Strip-Pooling, and MaxPooling on the input features along the channel dimension, respectively, to obtain rich feature information of the segmentation target in the spatial dimension through the three pooling paths. Then, the results obtained from the three pooling paths are stacked along the channels, and the number of channels is adjusted to 1 through convolution. The spatial attention weights are obtained by normalization using the sigmoid activation function. Finally, the spatial attention weights are multiplied by the original feature matrix. The calculation of the MSP module is as follows:

[0060] (1)

[0061] (2)

[0062] in Represents the Sigmoid activation function; Concat represents the channel concatenation operation; Conv 3×3 Represents a convolutional block consisting of 3x3 kernels; StripPool represents strip pooling; F input Represents the original feature map of the input;

[0063] like Figure 3 As shown, the attention-corrected residual ARRM module includes a DoublePooling-Attention module and residual skip connections. First, the input feature map is reduced in dimensionality through a 3×3 convolution. After two parallel MaxPooling and AvgPooling layers, it passes through a shared MLP layer to compress and expand the channels. Finally, the two outputs are added element-wise, and after passing through a BN layer and a sigmoid activation function, the corresponding attention-corrected weight matrix is ​​obtained. This matrix is ​​then multiplied by the original feature matrix and skip-connected to it to obtain the output. The calculation of the attention-corrected residual ARRM module is as follows:

[0064] (3)

[0065] (4)

[0066] (5)

[0067] (6)

[0068] (7)

[0069] in This represents the original feature map as input. This represents the new feature after adjusting the number of channels through convolution. MaxPool and AvgPool represent max pooling and average pooling operations, respectively. MLP represents a shared multilayer perceptron, BN represents batch normalization, and ReLU and... Representing different activation functions, and These represent element-wise summation and dot product, respectively; DPA is a dual-pooling attention module; F MaxPool F represents the feature map obtained after max pooling. AvgPool This represents the feature map obtained after the average pooling operation.

[0070] like Figure 4As shown, the hybrid feature integration MFFM module includes a hybrid attention mechanism containing MSP and SE modules and deformable convolutions. The input consists of low-level features and high-level semantic features. Low-level feature processing involves passing the input low-level features through the MSP module; high-level information processing uses an SE module. After attention filtering, the features from both paths are adaptively adjusted using a deformable convolution to optimize boundary feature extraction. Then, channel concatenation is performed. The concatenated features are further enhanced with a 1x1 convolutional block for non-linearity, and then concatenated again with the processed high-level features to strengthen semantic guidance. The concatenated feature matrix is ​​then extracted using a deformable convolution. Finally, it is summed with the low-level features filtered by the MSP module before output. The calculation of the hybrid feature integration MFFM module is as follows:

[0071] (8)

[0072] (9) (10)

[0073] (11)

[0074] (12)

[0075] DConv stands for deformable convolution operation, UpS for upsampling operation, MSP for MSP module, and SE for compression and activation operation. and These are the low-level and high-level feature maps of the MFFM module, which integrate the input mixed features. and These represent the low-level and high-level features after being filtered by the hybrid attention mechanism, respectively. and This is the result of fusing features at two different scales; This is the final output feature map.

[0076] This model uses deep supervision for feature regularization during training. Supervision is applied after the hidden layers in both stages of the network and in the final output layer, which enables the intermediate layers of the neural network to be trained well. At the same time, the skip connection in different stages of this model alleviates the gradient vanishing problem and reduces the training difficulty of deep supervision.

[0077] According to the appendix Figure 5The visualized feature map results show significant differences in features across different stages. The feature maps generated by the three selected deep supervised branches (layers 4, 5, and 6) exhibit relatively balanced changes after training regularization, and the model's ROI pixel regions are similar. Overall, the model demonstrates the hierarchical feature representation of the neural network in the form of deep supervised branches, integrating feature maps of different scales to fully utilize high, medium, and low-level features, ensuring a more robust optimization process and better expressive power.

[0078] like Figure 6 As shown, an image semantic segmentation method includes the following steps:

[0079] (a) Select image datasets with the same features as the images to be processed for batch data preprocessing, set the corresponding window width and window level according to the segmentation target, and then convert the image data into PNG format image data;

[0080] (b) Divide the image dataset obtained in step (a) into a training set, a validation set, and a test set without any overlap;

[0081] (c) Perform data augmentation on the image data in the training set, including random rotation, random horizontal flipping, and random vertical flipping;

[0082] (d) Design a loss function to improve the segmentation accuracy of the model. The loss function adopts a hybrid loss of DiceLoss and BCELoss, where α and β are the weight hyperparameters of DiceLoss and BCELoss, respectively, and their sum is 1. The loss expression of DiceLoss is:

[0083] (13)

[0084] The loss expression for BCELoss is:

[0085] (14)

[0086] The expression for the mixed loss of DiceLoss and BCELoss is:

[0087] (15)

[0088] N represents the total number of pixels, and C represents the total number of categories. This represents the class c in the binary GroundTruth, where the i-th pixel corresponds to the class value. This represents the predicted probability value for the corresponding category; Represents the smoothing exponent and equal to ;

[0089] (e) Obtain the image semantic segmentation model described above;

[0090] (f) The image semantic segmentation model is trained based on the loss function. The model weights are updated by backpropagation of gradients during the training process of the neural network. The model hyperparameter settings are judged based on the segmentation effect on the validation set after training. The saved model weights are updated. Finally, the model training is completed, and the image semantic segmentation effect is evaluated on the test set. During the training process of the image semantic segmentation model, the multiple predicted feature images output by the deep supervised training module are processed by formula (15) and then mean-averaged to obtain the total loss expression:

[0091] (16)

[0092] Once the image semantic segmentation model is trained, semantic segmentation can be performed on the image data to be processed.

[0093] The calculation principles of the segmentation evaluation metrics involved in training the image semantic segmentation model are as follows: DSC (Dicesimilarity coefficient) and IoU (Intersection over union) reflect the degree of overlap between the predicted image and the label. The larger the value, the better the segmentation effect. HD (Hausdorff distance) is bidirectional HD, which means finding the minimum distance between the two maximum distances calculated from two directions for set X and set Y. The smaller the value, the smaller the boundary difference between the two. SE (Sensitivity) reflects the model's ability to distinguish positive samples. Also, the larger the value, the better.

[0094] (17)

[0095] (18)

[0096] (19)

[0097] (20)

[0098] (twenty one)

[0099] In the formula, sets X and Y represent predictions and labels, respectively; TP represents positive samples predicted as positive by the model, TN represents negative samples predicted as negative by the model; FP represents negative samples predicted as positive by the model; FN represents positive samples predicted as negative by the model; and d(x,y) represents the distance from x to y. The test results of all models trained on the dataset are distributed with mean and standard deviation, and all metrics exceed the 95% confidence level.

[0100] This embodiment uses CT lesion segmentation of intracranial hemorrhage (ICH) as an example. After data preprocessing, 2090 spontaneous hemorrhage CT slices and 728 traumatic hemorrhage CT slices were obtained, and the data were divided into training, validation, and test sets in an 8:1:1 ratio, resulting in two datasets, A and B. After data augmentation, the model was trained. The following metrics were used to evaluate the segmentation model's performance during validation and testing: Dice coefficient (DSC), Intersection over Union (IoU), HD distance (HD), and sensitivity (SE).

[0101] The image semantic segmentation model (DFMA-Seg algorithm) of this invention is compared with several excellent segmentation algorithms. The segmentation results on the spontaneous ICH dataset are shown in Table 1, and the segmentation results on the traumatic ICH dataset are shown in Table 2. The visualization comparison effect is as follows. Figure 7 Furthermore, comparative experiments were conducted on deep supervised branches, revealing the optimal number of branch layers (N=3) for the DFMA-Seg algorithm during training. The comparison results are shown in Table 3, and the features of deep supervised branches are visualized as follows: Figure 8 As shown in Table 4, a comparison is made from the perspectives of computational time and space costs. Four metrics are used for evaluation: number of parameters, memory usage, computational cost (FLOPS), and inference time. All experiments demonstrate that the DFMA-Seg algorithm exhibits the best segmentation performance in this implementation, with the lowest computational cost and memory usage, and a fast inference speed, thus meeting the lightweight requirements.

[0102]

[0103] Table 1. Model comparison results on dataset A

[0104]

[0105] Table 2. Model comparison results on dataset B

[0106]

[0107] Table 3 Comparison Results of Deep Supervision Branches (Deepsup)

[0108]

[0109] Table 4 Comparison of Time and Space Costs

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described with reference to preferred embodiments, those skilled in the art should understand that various changes in form and detail can be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. An image semantic segmentation method, characterized in that: Includes the following steps: (a) Select image datasets with the same features as the images to be processed for batch data preprocessing, set the corresponding window width and window level according to the segmentation target, and then convert the image data into PNG format image data; (b) Divide the image dataset obtained in step (a) into a training set, a validation set, and a test set without any overlap; (c) Perform data augmentation on the image data in the training set, including random rotation, random horizontal flipping, and random vertical flipping; (d) Design a loss function to improve the segmentation accuracy of the model. The loss function adopts a hybrid loss of DiceLoss and BCELoss, where α and β are the weight hyperparameters of DiceLoss and BCELoss, respectively, and their sum is 1. The loss expression of DiceLoss is: (13) The loss expression for BCELoss is: (14) The expression for the mixed loss of DiceLoss and BCELoss is: (15) N represents the total number of pixels, and C represents the total number of categories. This represents the class c in the binary GroundTruth, where the i-th pixel corresponds to the class value. This represents the predicted probability value for the corresponding category; Represents the smoothing exponent and equal to ; (e) Obtain an image semantic segmentation model, including a feature extraction module, a feature fusion module, and a deep supervised training module; The feature extraction module consists of six STDC backbone convolutional layers, including stages 1 to 6. Each stage includes multiple basic modules, different attention modules, and skip connections of varying sizes. The last two stages also incorporate an attention-corrected residual (ARRM) module for feature attention correction and selection. For low-level features, a multi-scale spatial attention mechanism (MSP) module is used to extract spatial information through three different pooling layers. High-level features utilize a channel attention mechanism to achieve targeted filtering of semantic information, and feature preservation is achieved through skip connections of different sizes, thereby optimizing the results; The feature fusion module inputs the stage 3 features and the integrated stage 5 features into the hybrid feature integration MFFM module for fusion, realizing the extraction and combination of high-level semantic information and low-level spatial information, improving segmentation performance. Deformable convolution is introduced in the hybrid feature integration MFFM module. Deformable convolution fine-tunes the direction of the pixels after ordinary convolution, realizing the adaptive expansion of the convolution kernel. The deep supervised training module uses three levels of features for upsampling: stage 5 features, reshaped stage 6 features, and stage 3 and stage 5 features fused by the MFFM module through feature integration. These three features are used as input to the segmentation head, and a weighted average is used to obtain the final output. The MSP module performs AvgPooling, Strip-Pooling, and MaxPooling on the input features along the channel dimension, respectively, to obtain rich spatial feature information of the segmentation target through these three pooling paths. The results from these three pooling paths are then stacked across the channels, and the number of channels is adjusted to 1 using convolution. The spatial attention weights are then normalized using the sigmoid activation function. Finally, the spatial attention weights are multiplied by the original feature matrix. The calculation of the MSP module is as follows: (1) (2) in This represents the Sigmoid activation function; Concat represents the channel concatenation operation; F input Represents the original feature map of the input; The Attention-Corrected Residual ARRM module includes a DoublePooling-Attention module and residual skip connections. First, the input feature map is reduced in dimensionality through a 3×3 convolution. After two parallel MaxPooling and AvgPooling layers, it passes through a shared MLP layer to compress and expand the channels. Finally, the two outputs are element-wise added, and after passing through a BN layer and a sigmoid activation function, the corresponding attention-corrected weight matrix is ​​obtained. This matrix is ​​then multiplied by the original feature matrix and skip-connected to it to obtain the output. The calculation of the Attention-Corrected Residual ARRM module is as follows: (3) (4) (5) (6) (7) in This represents the original feature map as input. This represents the new feature after adjusting the number of channels through convolution. MaxPool and AvgPool represent max pooling and average pooling operations, respectively. MLP represents a shared multilayer perceptron, BN represents batch normalization, and ReLU and... Representing different activation functions, and These represent element-wise summation and dot product, respectively; DPA is a dual-pooling attention module; F MaxPool F represents the feature map obtained after max pooling; AvgPool The MFFM module, representing the feature map obtained after average pooling, incorporates a hybrid attention mechanism (MSP and SE modules) and deformable convolutions. The input consists of low-level features and high-level semantic features. Low-level feature processing involves passing the input low-level features through the MSP module; high-level information processing uses an SE module. After attention filtering, the features from both paths are adaptively adjusted using a deformable convolution to optimize boundary feature extraction. Channel concatenation is then performed, and the concatenated features are further enhanced with a 1x1 convolutional block for non-linearity. This concatenation is then combined with the processed high-level features to strengthen semantic guidance. The concatenated feature matrix is ​​then extracted using a deformable convolution, and finally summed with the low-level features filtered by the MSP module before output. The calculation of the MFFM module is as follows: (8) (9) (10) (11) (12) DConv stands for deformable convolution operation, UpS for upsampling operation, MSP for MSP module, and SE for compression and activation operation. and These are the low-level and high-level feature maps of the MFFM module, which integrate the input mixed features. and These represent the low-level and high-level features after being filtered by the hybrid attention mechanism, respectively. and This is the result of fusing features at two different scales; This is the final output feature map; (f) The image semantic segmentation model is trained based on the loss function. The model weights are updated by gradient backpropagation during the training process of the neural network. The model hyperparameter settings are judged based on the segmentation effect on the validation set after training. The saved model weights are updated. Finally, the image semantic segmentation effect is evaluated on the test set after the model training is completed. During the training of the image semantic segmentation model, the multiple predicted feature images output by the deep supervised training module are processed by formula (15) and then averaged to obtain the total loss expression: (16) Once the image semantic segmentation model is trained, semantic segmentation can be performed on the image data to be processed.

Images