Image segmentation method based on generalized balanced loss
By introducing a generalized class-balanced loss function and a SegNext network, the problem of imbalanced samples in image segmentation is solved, improving the segmentation accuracy of small sample regions and the overall image segmentation performance, and enhancing the model's robustness to imbalanced data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2024-06-14
- Publication Date
- 2026-07-24
AI Technical Summary
Existing image segmentation methods cannot effectively improve the segmentation accuracy of a few classes when dealing with imbalanced datasets, especially long-tailed data distributions, resulting in insufficient overall segmentation accuracy, particularly in face segmentation and few-sample segmentation scenarios.
A generalized class-balanced loss function (GBal-CE Loss) is introduced. By calculating the pixel and instance proportion of each class, the parameters γ and β are dynamically adjusted to balance the loss contribution of different classes, enhance the attention to small sample regions, and combine it with the SegNext network for image segmentation.
It improves the overall accuracy and robustness of image segmentation, especially the segmentation accuracy of small sample regions under long-tailed data distribution, and enhances the model's ability to handle imbalanced data.
Smart Images
Figure CN118735942B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image segmentation technology, specifically relating to an image segmentation method based on generalized class balance loss. Background Technology
[0002] Image segmentation is a core problem in computer vision. It involves subdividing a digital image into multiple sets of pixels to simplify or alter its representation, making it easier to analyze. Image segmentation refers to dividing an image into several non-overlapping regions based on features such as grayscale, color, spatial texture, and geometric shape. These features should exhibit consistency or similarity within the same region, while showing significant differences between different regions. Simply put, it's about separating the target from the background in an image. For grayscale images, pixels within a region generally have similar grayscale values, while grayscale discontinuities are typically observed at the region boundaries. Image segmentation has a wide range of applications, including natural scene understanding, remote sensing image processing, medical image analysis, and more specific applications such as face recognition and few-shot learning.
[0003] In natural scene segmentation, the challenge lies in parsing the rich and complex backgrounds and diverse object categories within images. Remote sensing image segmentation focuses on identifying surface features from satellite or aerial images, which typically involves processing large-scale scenes and analyzing multispectral data. Medical image segmentation, on the other hand, focuses on accurately identifying key medical information such as anatomical structures and lesions from images acquired through imaging techniques such as CT and MRI. Its accuracy is directly related to the effectiveness of diagnosis and treatment.
[0004] In these fields, especially in face segmentation and few-shot segmentation scenarios, the problem of imbalanced data distribution is particularly prominent. Face segmentation typically requires accurate differentiation of facial features, which involves identifying and dividing different facial regions, such as eyes, nose, and mouth, from the entire image for further analysis and processing. However, in the real world, specific facial expressions, poses, or occlusions can cause sample imbalance. Similarly, the challenge of few-shot segmentation lies in how to learn effective feature representations from a very small number of samples to achieve accurate identification of rare categories.
[0005] However, most existing image segmentation methods rely on standard loss functions, such as cross-entropy loss, which perform well on balanced datasets. However, these methods often fail to provide satisfactory results when dealing with imbalanced or long-tailed data distributions because they do not adequately account for the significant differences in the number of samples per class. Particularly in segmentation tasks with a small number of classes, these standard loss functions often lead to overfitting the model to the majority class while neglecting the minority class, resulting in insufficient segmentation accuracy. Furthermore, traditional methods often encounter similar limitations when dealing with complex scenarios such as face segmentation, as the sample distribution in these scenarios may be more extreme and imbalanced.
[0006] The patent application with publication number CN111210443B, entitled "A Deformable Convolution Hybrid Task Concatenated Semantic Segmentation Method Based on Embedding Balance", has the following shortcomings: it ignores the sample imbalance problem in existing image segmentation techniques, especially the image segmentation challenge under long-tailed data distribution, as well as the insufficient accuracy of traditional image segmentation methods based on cross-entropy loss function when processing small sample regions.
[0007] While existing image segmentation techniques have made significant progress in many fields, they still face some key challenges. One of the most significant problems is imbalanced data distribution, particularly its poor performance when dealing with long-tailed data distributions. This distribution means that some classes have far more samples than others, causing models to tend to optimize the majority class during training while neglecting the minority class, thus reducing overall segmentation accuracy and generalization ability.
[0008] Furthermore, traditional image segmentation methods based on cross-entropy loss also have significant drawbacks. Cross-entropy loss is a commonly used loss function that measures the difference between the predicted probability distribution and the true label. While cross-entropy loss works well in many cases, it often fails to provide sufficient guidance when dealing with imbalanced datasets, leading to insufficient learning of a few classes and consequently affecting the model's segmentation performance for those classes. Summary of the Invention
[0009] To overcome the shortcomings of existing technologies, this invention aims to provide an image segmentation method based on generalized class-balanced loss. This method addresses the problem of imbalanced samples and improves the segmentation accuracy and robustness of the model under long-tailed data distributions. By introducing a generalized class-balanced loss function (GBal-CE Loss) and utilizing different adjustment factors, the loss contributions between different classes are effectively balanced. This not only overcomes the limitations of traditional image segmentation methods based on cross-entropy loss functions when facing imbalanced samples, but also further improves the overall image segmentation accuracy by accurately segmenting small sample regions. This achieves a significant enhancement in the model's ability to handle imbalanced data distributions while maintaining overall performance.
[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0011] The image segmentation method based on generalized class balance loss includes the following steps;
[0012] Step 1: Input the image into the segmentation network; extract image features through the encoder in the segmentation network, and restore the image features to the original resolution through the decoder to generate the predicted segmented image;
[0013] Step 2: Using the real segmented image as the ground truth label, calculate the generalized class balance loss (GBal-CE Loss) between the predicted segmented image and the real segmented image;
[0014] Step 3: Perform backpropagation using the generalized class balance loss and update the network parameters.
[0015] In step 1, the segmentation network uses SegNext, which is an encoder-decoder architecture, including an encoder and a decoder. The encoder uses a pyramid architecture, and the decoder uses a lightweight Hammerger module.
[0016] The decoder receives features from the last three stages of the encoder to preserve high-level semantic information; a lightweight Hamburger module is used to further model the global context; the decoder gradually recovers the spatial resolution of the image through a series of upsampling layers, each followed by a convolutional layer and an activation function to gradually refine the feature map, and finally the decoder output has the same resolution as the input image, generating the predicted segmented image.
[0017] The pyramid structure in step 1 consists of four stages, with the spatial resolution decreasing sequentially in each stage. Similar to the Vision Transformer (ViT), it uses a multi-scale convolutional attention (MSCA) module. The multi-scale convolutional attention module combines depth-wise convolution to aggregate local information, multi-branch depth-wise convolution to capture multi-scale contextual information, and ordinary convolution to model correlation in the channel dimension.
[0018] The lightweight Hammerger module is used to aggregate features from the last three stages of the encoder for global context modeling, while the SegNext decoder only aggregates features from stages 2 to 4.
[0019] The spatial resolution of each stage of the pyramid structure decreases sequentially. In each stage, a downsampling block and a building block are used. The downsampling block and the building block (MSCA module) in each stage work together to gradually extract high-level features of the image. The downsampling block reduces the spatial resolution, so that the subsequent MSCA module can process higher-level features. In the pyramid structure, the building block of each stage processes features of different resolutions, thereby capturing multi-scale contextual information throughout the network.
[0020] The downsampling block includes a convolutional layer with a stride of 2 and a kernel size of 3×3, followed by a batch normalization layer;
[0021] The building block employs a multi-scale convolutional attention (MSCA) module, which uses convolutional outputs as attention weights to reweight the inputs, thereby enhancing the model's representational capabilities.
[0022] MSCA consists of three parts: a deep convolution for aggregating local information, a multi-branch deep strip convolution to capture multi-scale context, and a 1×1 convolution for modeling the relationship between different channels.
[0023] The output of the 1×1 convolution is directly used as the input of the attention weight reweighted MSCA. A hierarchical structure is adopted, and a batch normalization layer is used in each building block to improve segmentation performance.
[0024] In step 1, the decoder receives features from the last three stages of the encoder to preserve high-level semantic information; a lightweight Hamburger module is used to further model the global context; the decoder gradually recovers the spatial resolution of the image through a series of upsampling layers, with each upsampling layer followed by a convolutional layer and an activation function to gradually refine the feature map. Finally, the decoder output has the same resolution as the input image, generating the predicted segmented image.
[0025] Furthermore, regarding the parameter settings of the encoder and decoder, the encoder parameters include the weights and biases of each convolutional layer (including depthwise convolution, strip convolution, and 1×1 convolution) and the parameters of the batch normalization layers (scaling and offset parameters for mean and variance). The decoder parameters include the weights and biases of each upsampling layer, the parameters of each convolutional layer and activation function, and the parameters in the Hamburger module. SegNext's MSCAN backbone network is an extended version of the Vision Attention Network (VAN). Each module uses batch normalization instead of layer normalization. Different MSCAN backbones are constructed by stacking MSCAN modules. SegNext is used to aggregate information from the bottom up, extract multi-scale contextual information from local to global, and achieve adaptability in spatial and channel dimensions.
[0026] Step 2 involves comparing the predicted segmented image and the ground truth segmented image at the pixel level to determine the predicted and ground truth classes for each pixel. The number of pixels in each class is counted, and the class imbalance coefficient is calculated. For each class, its pixel percentage is calculated, defined as the number of pixels in the class divided by the total number of pixels. The number of instances in each class is counted, and the class instance imbalance coefficient is calculated. For each class, its instance percentage is calculated, defined as the number of instances in the class divided by the total number of instances. The class imbalance coefficient is normalized to obtain the balance coefficient for each class. Based on the cross-entropy loss, a dynamically adjusted parameter is introduced, defining a generalized class balance loss. The loss value for each pixel is calculated using the generalized class balance loss formula, and the loss values of all pixels are averaged to obtain the final generalized class balance loss value.
[0027] The Generalized Class Balanced Loss (GBal-CE Loss) adds dynamically adjusted parameters γ and β as regulation factors to the cross-entropy loss. By improving the segmentation accuracy in small sample regions (in image segmentation tasks, small sample regions typically refer to pixel categories or instance regions that appear less frequently in the training dataset), it improves the overall image segmentation accuracy. Therefore, while maintaining overall performance, it particularly enhances the model's robustness to long-tailed data distributions. The corresponding formula is as follows:
[0028]
[0029] Where γ1 and γ2 correspond to pixel-level and instance-level adjustable parameters, respectively, β is an adjustable parameter, and p i and q i These correspond to the pixel percentage and instance percentage for each category, respectively. Assume the model's prediction output for all categories is z = [z1, z2, ..., z...]. C ] T , where C is the total number of categories.
[0030] Step 3 specifically involves calculating the generalized class balance loss value and then passing it to the backpropagation algorithm. The backpropagation algorithm is used to calculate the gradient of the network parameters. The gradient calculation is performed using the chain rule, propagating from the output layer to the input layer layer by layer. Based on the calculated gradient, an optimization algorithm is used to update the network parameters. This process is repeated until the loss function converges or the predetermined number of training rounds is reached.
[0031] Network parameters specifically refer to the parameters of the encoder and the decoder. The encoder parameters include the weights and biases of each convolutional layer (including depthwise convolution, strip convolution, and 1×1 convolution) and the parameters of the batch normalization layer (scaling and offset parameters for mean and variance). The decoder parameters include the weights and biases of each upsampling layer, the parameters of each convolutional layer and activation function, and the parameters in the Hamburger module.
[0032] The beneficial effects of this invention are:
[0033] (1) This invention proposes an image segmentation model based on generalized class balance loss. This model quantifies the sample distribution features by calculating the pixel ratio and instance ratio of each semantic region, effectively addressing the challenges of image segmentation under long-tailed data distribution.
[0034] (2) A class-balanced loss function (GBal-CE Loss) is introduced, which balances the loss contribution between different categories by using different adjustment factors. It particularly emphasizes the attention to small sample regions and solves the problem of insufficient accuracy of traditional image segmentation methods based on cross-entropy loss function when dealing with small sample regions.
[0035] (3) This invention not only improves the segmentation accuracy of small sample regions, but also enhances the robustness of the overall image segmentation model to unbalanced data distribution, ensuring that the overall performance is not sacrificed while improving the segmentation accuracy of specific regions.
[0036] This invention not only theoretically fills the gaps in existing image segmentation techniques, but also demonstrates significant improvements and potential in practical applications, providing an effective technical path for solving the problem of imbalanced samples in the field of image segmentation. Attached Figure Description
[0037] Figure 1 This is a schematic diagram of the image segmentation model framework based on generalized class balance loss of the present invention. Detailed Implementation
[0038] The present invention will now be described in further detail with reference to the accompanying drawings.
[0039] like Figure 1 As shown, the image segmentation method based on generalized class balance loss includes the following steps;
[0040] Step 1: Segmenting the network:
[0041] This invention employs SegNext as the segmentation network, a novel encoder-decoder architecture designed to address key challenges in semantic segmentation tasks. Its design is based on several key attributes, including the use of a powerful backbone network, the realization of multi-scale information interaction, and the application of spatial attention, while maintaining low computational complexity. The SegNext network structure mainly consists of an encoder and a decoder. The encoder adopts a pyramid architecture, similar to a VisionTransformer (ViT), but unlike traditional Transformer schemes, it does not use a self-attention mechanism. Instead, it uses a multi-scale convolutional attention (MSCA) module. This module combines depth-wise convolution to aggregate local information, multi-branch depth-wise convolution to capture multi-scale contextual information, and ordinary convolution to model relevance in the channel dimension. The MSCA module uses convolutional outputs as attention weights to reweight the input, thereby enhancing the model's representational capabilities. The decoder employs a lightweight Hammerger module, specifically designed to aggregate features from the last three stages of the encoder for global context modeling. Unlike the SegFormer decoder, the SegNext decoder only aggregates features from stages 2 to 4, because the features from stage 1 contain too much low-level information, which may affect performance, and processing these features would increase the computational burden.
[0042] In terms of parameter settings, SegNext's MSCAN backbone network is an extended version of the VisionAttention Network (VAN), with each module using batch normalization instead of layer normalization. By stacking MSCAN modules to build different MSCAN backbones, SegNext can aggregate information from the bottom up, extract multi-scale contextual information from local to global, and achieve adaptability in spatial and channel dimensions. Furthermore, SegNext's design reduces computational complexity while maintaining the ability to capture high-level semantic information, making it competitive in semantic segmentation tasks, as shown in the performance comparison on the Cityscape and ADE20K datasets.
[0043] In summary, SegNext, through its innovative network structure and parameter settings, can efficiently handle semantic segmentation tasks, provide accurate pixel-level predictions, and maintain low computational costs. Therefore, this invention selects SegNext as the segmentation network.
[0044] Step 2: Generalized class-balanced loss function:
[0045] This invention proposes a generalized class-balanced loss function (GBal-CE Loss) to calculate loss, which is based on cross-entropy loss (CE Loss). In image segmentation tasks, cross-entropy loss is a commonly used loss function to measure the difference between the model's predictions and the true segmentation labels. Cross-entropy loss can help optimize model parameters during training, enabling the model to better learn image segmentation information. In segmentation tasks, each pixel is typically treated as an independent sample and classified into different categories. For each pixel, the model generates a probability distribution representing the probability that the pixel belongs to each category C. The true segmentation labels assign each pixel to a specific category. Cross-entropy loss measures the model's prediction accuracy by comparing the difference between the model's predicted probability distribution and the true labels.
[0046] Specifically, for each pixel, the cross-entropy loss calculates the cross-entropy between the predicted probability distribution p and the true label y, and then sums or averages the cross-entropy of all pixels N.
[0047] The Generalized Class Balanced Loss (GBal-CE Loss) proposed in this invention adds different parameters as adjustment factors to the cross-entropy loss to balance the loss contributions between different classes, especially strengthening the focus on small sample regions. This effectively solves the problem of insufficient accuracy in small sample region segmentation by traditional image segmentation methods based on the cross-entropy loss function. By improving the accuracy in small sample region segmentation, this invention further improves the overall image segmentation accuracy, thereby enhancing the model's robustness to imbalanced data distributions while maintaining overall performance. The corresponding formula is as follows:
[0048]
[0049] Where γ1 and γ2 correspond to pixel-level and instance-level adjustable parameters, respectively, β is an adjustable parameter, and p i and q i These correspond to the pixel percentage and instance percentage for each category, respectively. Assume the model's prediction output for all categories is z = [z1, z2, ..., z...]. C ] T , where C is the total number of categories.
[0050] This invention proposes an image segmentation model based on generalized class-balanced loss, aiming to overcome the sample imbalance problem faced by existing image segmentation techniques, especially the challenges posed by long-tailed data distributions. By introducing dynamically adjustable parameters γ1, γ2, and β into the model, where γ1 and γ2 correspond to pixel-level and instance-level adjustable parameters, respectively, and β is an adjustable parameter, this invention can effectively handle the accuracy problem of small sample region segmentation, thereby improving the overall image segmentation performance.
[0051] Step 3: Experiment:
[0052] 3.1 Experimental Setup
[0053] Datasets: Two datasets were used in all experiments of the image segmentation model based on generalized class balance loss: a face dataset CelebAMask-HQ and a medical image dataset GDPH&SYSUCC. The CelebAMask-HQ dataset has 19 classes, and the GDPH&SYSUCC dataset has 3 classes. During the training and testing processes of each experiment, the corresponding datasets were divided into training, validation, and test sets in an 8:1:1 ratio.
[0054] Implementation details: In the network model settings, the encoder and decoder use MSCAN-B pre-trained with ADE20K as the backbone network and LightHamHead as the decoder head. For training parameters, the batch size is set to 8, each input image is randomly cropped to a size of 512×512, and the image format is a 3-channel RGB image, normalized to the range [0,1] before being input to the encoder. The loss function used is the Generalized Class Balanced Loss (GBal-CE Loss). The optimizer uses a weight decay rate of 3×10⁻⁶. -4 AdamW optimizer.
[0055] 3.2 Experimental Evaluation Indicators
[0056] This experiment uses commonly used evaluation metrics for semantic segmentation to measure the quality of the segmented images generated. These include mIoU and Dice Coefficient (F1-score). These two metrics are also applicable to face segmentation. Since the LaPa dataset uses F1-score and the CelebaMask-HQ dataset uses mIoU, the experiment uses both metrics in combination.
[0057] Mean Intersection over Union (mIoU): mIoU is a commonly used evaluation metric for image semantic segmentation tasks, used to measure the model's ability to accurately segment objects of different categories at the pixel level. mIoU evaluates model performance by calculating the Intersection over Union (IoU) between the predicted results and the ground truth labels. For each category, IoU is defined as the area of the intersection of the predicted results and the ground truth labels divided by the area of their union. mIoU is the average IoU across all categories. The formula for calculating IoU is shown in Equation 3, and mIoU is the average of the IoU across multiple categories, as shown in Equation 4.
[0058]
[0059]
[0060] In the above formula, TP(class) represents the number of pixels correctly predicted as belonging to that class, FP(class) represents the number of pixels incorrectly predicted as belonging to that class, and TN(class) represents the number of pixels of that class that were missed in the prediction. The value of mIoU ranges from 0 to 1, and a higher value indicates a higher degree of agreement between the model's segmentation results and the true labels.
[0061] Dice coefficient (also known as F1-score): The F1-score is a commonly used evaluation metric to measure the similarity or overlap between two sets. In image segmentation tasks, it is often used to evaluate how well the model's segmentation results match the ground truth labels. The formula for calculating Dice is as follows:
[0062]
[0063] The Dice Coefficient ranges from 0 to 1. A value closer to 1 indicates a higher degree of overlap between the model's segmentation result and the ground truth label, meaning the segmentation result is more similar to the ground truth label. In image segmentation tasks, the F1-score can be used to measure the model's ability to accurately segment different categories of objects. It takes into account the overlap between the predicted result and the ground truth label, making it particularly suitable for segmenting objects with blurred boundaries or overlapping areas.
[0064] These evaluation metrics rely on detailed comparisons of segmented images. By comprehensively considering basic evaluation metrics for semantic segmentation and fine-grained comparisons of segmented images, experimental results effectively verify the superiority of this invention in terms of image quality and segmentation rationality.
[0065] 3.3 Experimental Results
[0066] To verify the effectiveness of the generalized class balance loss proposed in this method, this experiment mainly compares the cross-entropy loss used in the basic segmentation module of this application with the method incorporating the generalized class balance loss function on experimental metrics, including the different effects of different parameters. The evaluation metrics used include mIoU and F1-score. Detailed metrics on the CelebaMask-HQ face dataset are shown in Table 1. The table lists the IoU scores for different parameters on various face parts (such as necklace, eyes, and hair, with corresponding pixel percentages of 0.021%, 0.22%, and 31.96%, respectively) in the CelebaMask-HQ dataset, as well as the overall mIoU score. Here, `base` represents the basic segmentation module of this algorithm, which demonstrates advantages in multiple face parts. Note: The mIoU metric is used when calculating the total metric.
[0067] Table 1. Evaluation of IoU and Fscore in the CelebAMask-HQ dataset.
[0068]
[0069] In small sample regions such as necklaces and eyes, the IoU and Fscore scores were significantly improved when γ1 = 0.5, representing increases of 8.3% and 12.4% respectively compared to the cross-entropy loss. This indicates that the generalized class balance loss, when γ1 = 0.5, can more accurately capture boundaries and details when segmenting these facial parts, thus achieving better segmentation results. Furthermore, for large sample regions such as hair, γ1 = 1 also exhibited high IoU and Fscore scores, reaching 92.48% and 96.09% respectively. This demonstrates that the method maintains high accuracy and consistency when handling these complex facial parts.
[0070] Table 2 shows the detailed metrics for the GDPH&SYSUCC medical imaging dataset. The table lists the IoU scores for benign tumor regions, malignant tumor regions, and background regions, as well as the overall mIoU score, in the GDPH&SYSUCC dataset. The pixel percentages for benign tumor regions, malignant tumor regions, and background regions are 0.72%, 9.18%, and 90.1%, respectively.
[0071] Table 2. Evaluation of IoU and Fscore data in the GDPH & SYSUCC dataset.
[0072]
[0073]
[0074] In segmenting medical image datasets, the model incorporating generalized class balance loss achieved overall IoU and Fscore scores of 77.36% and 86.13%, respectively. Particularly when γ1 = 0.5, both IoU and Fscore scores improved for various sample regions, such as benign tumors, with increases of 0.54% and 0.45% compared to cross-entropy loss. High IoU and Fscore scores were also observed when γ1 = 3, particularly with a 1.02% increase in Fscore for benign tumor regions. This indicates that generalized class balance loss can more accurately capture boundaries and details when segmenting these tumor regions, resulting in better segmentation outcomes.
[0075] In summary, the image segmentation model based on generalized class-balanced loss demonstrates significant advantages in addressing the imbalanced sample problem inherent in existing image segmentation techniques. It effectively solves the accuracy deficiency of traditional image segmentation methods based on cross-entropy loss when handling small sample regions. By improving accuracy in small sample region segmentation, this invention further enhances the overall image segmentation accuracy, thereby significantly strengthening the model's robustness to imbalanced data distributions while maintaining overall performance.
[0076] This invention proposes an image segmentation model based on generalized balanced cross-entropy loss (GBal-CE Loss) to address the sample imbalance problem in existing image segmentation techniques, particularly for image segmentation challenges with long-tailed data distributions. The model first quantifies the distribution characteristics of samples by calculating the pixel and instance proportions of each semantic region. Then, it employs a generalized balanced cross-entropy loss function to calculate the loss. This loss function utilizes different adjustment factors to balance the loss contributions between different categories, especially strengthening the focus on small sample regions. This effectively solves the problem of insufficient accuracy in handling small sample region segmentation using traditional image segmentation methods based on cross-entropy loss. By improving the accuracy in small sample region segmentation, this invention further improves the overall image segmentation accuracy, thereby enhancing the model's robustness to imbalanced data distributions while maintaining overall performance.
Claims
1. An image segmentation method based on generalized class-balanced loss, characterized in that, Includes the following steps; Step 1: Input the image into the segmentation network, extract image features through the encoder in the segmentation network, and restore the image features to the original resolution through the decoder to generate the predicted segmented image; Step 2: Using the real segmented image as the ground truth label, calculate the generalized class balance loss between the predicted segmented image and the real segmented image; Step 3: Perform backpropagation using the generalized class balance loss and update the network parameters; In step 1, the segmentation network adopts SegNext, which is an encoder-decoder architecture, including an encoder and a decoder. The encoder adopts a pyramid architecture, and the decoder adopts a lightweight Hammerger module. The decoder receives features from the last three stages of the encoder to preserve high-level semantic information; a lightweight Hamburger module is used to further model the global context; the decoder gradually recovers the spatial resolution of the image through a series of upsampling layers, each followed by a convolutional layer and an activation function to gradually refine the feature map, and finally the decoder output has the same resolution as the input image, generating the predicted segmented image; Step 2 compares the predicted segmented image and the real segmented image at the pixel level to determine the predicted and real categories of each pixel, counts the number of pixels in each category, calculates the class imbalance coefficient, calculates the pixel proportion of each category (defined as the number of pixels in the class divided by the total number of pixels), counts the number of instances in each category, calculates the class instance imbalance coefficient, calculates the instance proportion of each category (defined as the number of instances in the class divided by the total number of instances), normalizes the class imbalance coefficient to obtain the balance coefficient for each category, introduces a dynamic adjustment parameter based on the cross-entropy loss, defines the generalized class balance loss, calculates the loss value of each pixel using the generalized class balance loss formula, and averages the loss values of all pixels to obtain the final generalized class balance loss value. The generalized class balance loss, based on the cross-entropy loss, incorporates dynamically adjusted parameters γ and β as adjustment factors. By improving the accuracy in segmenting small sample regions, it enhances the overall image segmentation accuracy and strengthens the model's robustness to long-tailed data distributions. The corresponding formula is as follows: (1) (2) in, , These correspond to adjustable parameters at the pixel and instance levels, respectively. It is an adjustable parameter. and These correspond to the pixel percentage and instance percentage of each category, respectively. The model's prediction output for all categories is set to... ,in This represents the total number of categories.
2. The image segmentation method based on generalized class balance loss according to claim 1, characterized in that, The pyramid structure in step 1 consists of four stages, with the spatial resolution decreasing sequentially in each stage. , , , It uses a multi-scale convolutional attention module, which combines depthwise segregating convolution to aggregate local information, multi-branch depthwise segregating convolution to capture multi-scale contextual information, and ordinary convolution to model correlation in the channel dimension; The lightweight Hammerger module is used to aggregate features from the last three stages of the encoder for global context modeling, while the SegNext decoder only aggregates features from stages 2 to 4.
3. The image segmentation method based on generalized class balance loss according to claim 2, characterized in that, In each stage of the pyramid structure, a downsampling block and a building block are used. The downsampling block and the building block in each stage work together to progressively extract high-level features of the image. The downsampling block reduces the spatial resolution, allowing the subsequent MSCA module to process higher-level features. In the pyramid structure, the building block in each stage processes features at different resolutions, capturing multi-scale contextual information throughout the network. The downsampling block includes a convolutional layer with a stride of 2 and a kernel size of 3×3, and a batch normalization layer; The building block employs a multi-scale convolutional attention (MSCA) module. The MSCA module uses the convolutional output as attention weights to reweight the input.
4. The image segmentation method based on generalized class balance loss according to claim 3, characterized in that, MSCA consists of three parts: deep convolutions for aggregating local information, multi-branch deep strip convolutions for capturing multi-scale context, and 1×1 convolutions for modeling the relationships between different channels; The output of the 1×1 convolution is directly used as the input of the attention weight reweighted MSCA, and a hierarchical structure is adopted, with a batch normalization layer used in each building block.
5. The image segmentation method based on generalized class balance loss according to claim 2, characterized in that, Regarding the parameter settings of the encoder and decoder, the encoder parameters include the weights and biases of each convolutional layer. The convolutional layers include parameters for depthwise convolution, strip convolution, 1×1 convolution, and batch normalization layers, including scaling and offset parameters for mean and variance. The decoder parameters include the weights and biases of each upsampling layer, the parameters of each convolutional layer and activation function, and the parameters in the Hamburger module. In SegNext's MSCAN backbone network, each module uses batch normalization. Different MSCAN backbones are constructed by stacking MSCAN modules. SegNext is used to aggregate information from the bottom up, extract multi-scale contextual information from local to global, and achieve adaptability in spatial and channel dimensions.
6. The image segmentation method based on generalized class balance loss according to claim 1, characterized in that, Step 3 specifically involves calculating the generalized class balance loss value and then passing it to the backpropagation algorithm. The backpropagation algorithm is used to calculate the gradient of the network parameters. The gradient calculation is performed by the chain rule, propagating from the output layer to the input layer layer by layer. Based on the calculated gradient, the network parameters are updated using an optimization algorithm. The above process is repeated until the loss function converges or the predetermined number of training rounds is reached.