Multitask semantic segmentation and classification system for glomerular crescent lesions

Abstract

The invention provides a glomerular crescent lesion multi-task semantic segmentation and classification system, which relates to the technical field of medical image processing and comprises an input module, an encoder module, a decoder module, a boundary branch module, a main output module and a processing module. The input module is used for receiving images; the encoder module adopts a symmetric topological structure and is used for carrying out multi-stage down-sampling operation on an input image and outputting feature maps of different scales; the decoder module is used for fusing the feature maps from the corresponding stages of the encoder and carrying out up-sampling step by step; the boundary branch module is connected behind the lowest layer output feature at the tail end of the decoder; the main output module is located at the tail end of the decoder; and the processing module is used for calculating a composite loss function and performing back propagation to update parameters of the encoder module, the decoder module and the boundary branch module, and is used for outputting a final segmentation and classification result of the lesion region of the crescent body. The system provided by the invention can realize accurate segmentation and category prediction of the lesion area of the crescent body.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a multi-task semantic segmentation and classification system for glomerular crescent lesions. Background Technology

[0002] Crescent bodies are one of the indicators of activity and chronicity in anti-basement membrane antibody-related glomerulonephritis, immune complex-related glomerulonephritis (such as IgAN and lupus nephritis), and ANCA-associated vasculitis. They are closely related to severe, rapidly progressive glomerulonephritis and acute or subacute renal failure. Therefore, rapid identification and quantification of various crescent bodies in glomerular pathological imaging are of great significance for improving the accuracy of diagnosis, treatment, and prognosis of kidney diseases.

[0003] However, existing deep learning-based image segmentation techniques primarily focus on glomerular segmentation, lacking solutions for segmenting lesion structures within the glomerulus. Furthermore, the morphological and semantic differences between fibrous crescents, cellular crescents, and cellular-fibrous crescents are not significant, exhibiting a degree of continuity and transition. The ambiguity of their morphological boundaries significantly increases the difficulty of extracting discriminative features. The low contrast between the lesion region and adjacent tissues, along with the invasive growth characteristics at the boundary, pose a severe challenge to accurate lesion segmentation. The combined effect of these anatomical characteristics and imaging features creates significant challenges to the accuracy and robustness of crescent segmentation tasks. Summary of the Invention

[0004] To address the technical problems in related technologies, this invention provides a multi-task semantic segmentation and classification system for glomerular crescent lesions.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A multi-task semantic segmentation and classification system for glomerular crescent lesions, comprising: The input module is used to receive images of glomerular regions containing crescent lesions extracted from renal biopsy light microscopic pathology (WSI). The encoder module adopts a symmetrical topology and contains four cascaded feature extraction stages. Each stage includes a hybrid coding module and a downsampling module, which are used to perform multi-level downsampling operations on the input image, construct hierarchical feature representations, and output feature maps at different scales. The decoder module consists of three stages, each of which includes an upsampling operation, a gated attention module, and a convolutional layer, which are used to fuse feature maps from the corresponding stages of the encoder and upsample them step by step to restore spatial resolution. The boundary branch module, connected after the lowest layer output features at the end of the decoder, is used to extract boundary-related features of the crescent lesion region and output a binarized boundary prediction map. The main output module, located at the end of the decoder, is used to receive the final output features of the decoder and generate semantic segmentation probability distribution maps for four categories: cellular crescent, cellular fibrous crescent, fibrous crescent, and background region. The processing module is used to calculate the composite loss function and backpropagate to update the parameters of the encoder module, decoder module and boundary branch module during the training phase, based on the semantic segmentation probability distribution map, the binarized boundary prediction map and the corresponding real semantic segmentation labels and real boundary labels; and to output the final crescent lesion region segmentation and classification results during the prediction phase, based on the semantic segmentation probability distribution map.

[0006] Optionally, the feature map sizes output by the four cascaded feature extraction stages of the encoder module are 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image, respectively, and the corresponding number of output channels are 64, 128, 256 and 512, respectively.

[0007] Optionally, the hybrid encoding module in the encoder module includes: Convolutional blocks are used to perform convolution operations on the input feature map to extract local location information, resulting in a feature map containing local location information. The multi-scale channel attention module is used to perform multi-scale feature extraction and channel attention weighting on the input feature map to obtain a multi-scale feature map with channel attention weighting. The feature fusion unit is used to add the feature map containing local location information to the multi-scale feature map weighted by channel attention element-wise to obtain a fused feature map; The Transformer module is used to convert the fused feature map into a one-dimensional token sequence for processing, and then convert the processed result back into a two-dimensional feature map for output.

[0008] Optionally, the operation of a single stage of the decoder module includes: The upsampling operation is used to upsample the feature information from the corresponding level of the encoder module; The gated attention module is used to fuse the upsampled feature information with the feature information from the previous decoding stage, and enhance the features of key regions through the attention mechanism; Convolutional layers are used to perform convolution operations on the fused features to generate the output features for the current stage.

[0009] Optionally, the boundary branch module consists of a 3×3 convolutional layer, a BatchNorm layer, and a ReLU activation function connected in sequence.

[0010] Optionally, the composite loss function used by the processing module during the training phase is: In the formula, Represents the focus loss function. Represents the Dice loss function. Represents the boundary loss function. , and These are the weighting coefficients for the three loss functions.

[0011] Optionally, the focus loss function The definition of is: In the formula, The total number of crescent body categories. To balance the weights for each category, The model predicts the probability of crescent body category C and , This is a focus factor used to control the degree to which the model pays attention to samples of varying difficulty. To represent the uniquely hot code of the true label on the crescent category C and .

[0012] Optionally, the Dice loss function The definition of is: In the formula, is the total number of crescent body categories, and c is the crescent body category index. For pixel index, Represents pixels The predicted probability for category c. Represents pixels For the true label of category c, This is a smoothing factor.

[0013] Optionally, the boundary loss function The definition of is: In the formula, The total number of pixels in the image. For pixel index, Indicates the first The boundary label of each pixel and , Indicates the first The boundary prediction probability corresponding to each pixel and .

[0014] Optionally, the weighting coefficients in the composite loss function satisfy: .

[0015] Beneficial effects: 1. Through the above technical solution, firstly, the system of the present invention can achieve end-to-end precise segmentation and classification. Specifically, the system of the present invention, through the collaborative design of encoder-decoder symmetrical architecture and boundary branches, enables the system to perform integrated and precise segmentation and pathological type identification of three types of lesion regions and their boundaries in glomerular light microscopy images: cellular crescents, cellular fibrous crescents, and fibrous crescents. This effectively overcomes the shortcomings of traditional methods in segmenting lesion structures within the glomerulus, thereby achieving end-to-end precise segmentation and classification.

[0016] Secondly, the system of the present invention can effectively improve the accuracy of boundary recognition. Specifically, the system of the present invention introduces an independent boundary branch module to specifically extract the boundary features of the lesion area and output a binary boundary map. By enhancing the model's perception ability of low-contrast, invasive crescent-shaped contours through boundary supervision signals, the accuracy of boundary segmentation can be significantly improved, thus effectively improving the accuracy of boundary recognition.

[0017] Third, the system of the present invention can optimize model training and generalization capabilities. Specifically, the processing module of the system of the present invention uses a composite loss function to jointly optimize the segmentation main task and the boundary auxiliary task, and simultaneously improves the robustness of the model to class distribution imbalance (e.g., sparse samples of fibrous crescent bodies) and the modeling ability of complex boundary morphologies during the training process, thereby effectively enhancing the system's generalization performance on diverse pathological images.

[0018] Fourth, the system of the present invention can support digital pathological diagnosis. Specifically, the system of the present invention outputs a four-category probability distribution map (three types of crescent bodies + background) containing semantic information of lesion type, which can provide automated and quantitative technical support for the severity assessment, treatment decision-making and prognostic analysis of kidney disease, reduce the subjectivity and time-consuming nature of manual interpretation, and provide patients with digital pathological diagnosis.

[0019] 2. Other beneficial effects or advantages of the present invention will be described in detail in the specific embodiments. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] in: Figure 1 This is a schematic diagram of a network structure provided in an exemplary embodiment of the present invention; Figure 2 This is a schematic diagram of an encoder module structure provided in an exemplary embodiment of the present invention; Figure 3 This is a schematic diagram of the Basicblock structure provided in an exemplary embodiment of the present invention; Figure 4 This is a schematic diagram of a gated attention module structure provided in an exemplary embodiment of the present invention; Figure 5 This is a diagram showing the segmentation prediction results of three types of crescent lesions provided in an exemplary embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0023] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. It should also be noted that in embodiments of this invention, the words "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in embodiments of this invention should not be construed as preferred or advantageous over other embodiments or designs. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0025] To facilitate a clearer and more accurate understanding of the technical solutions of this invention by those skilled in the art, the existing related technologies and their technical problems will be described in more detail below.

[0026] First, it should be noted that renal pathology images contain rich histological features and lesion information, serving as a crucial foundation for the diagnosis, treatment, and research of kidney diseases. As a data source for pathological images, renal biopsy samples are converted into high-resolution digital pathological images (WSI) using whole-slide digital scanning technology. Renal pathology WSIs contain the complex morphological features and staining characteristics of renal tissue, making them an important basis for the diagnosis of kidney diseases.

[0027] Among the many types of kidney diseases, glomerular diseases are the most numerous and complex. The crescent shape, as a characteristic pathological change, specifically refers to the phenomenon where abnormal components deposit within Bowman's capsule, leading to compression of capillary loops. Based on the proportion of epithelial cells and collagen fibers in the crescent, it is classified into cellular crescents, cellular-fibrous crescents, and fibrous crescents.

[0028] Crescent bodies are one of the indicators of activity and chronicity in anti-basement membrane antibody-related glomerulonephritis, immune complex-related glomerulonephritis (such as IgAN and lupus nephritis), and ANCA-associated vasculitis. They are closely related to severe, rapidly progressive glomerulonephritis and acute or subacute renal failure. Therefore, rapid identification and quantification of various crescent bodies in glomerular pathological imaging are of great significance for improving the accuracy of diagnosis, treatment, and prognosis of kidney diseases.

[0029] In recent years, image segmentation technology based on deep learning (DL) has made significant progress, promoting the intelligentization of medical image interpretation.

[0030] For example, the paper "Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015:18th International Conference,Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer International Publishing, 2015: 234-241" first proposed the U-shaped segmentation architecture U-Net and used skip connections to fuse multi-level features of the encoder and decoder, thus promoting semantic pixel segmentation of medical images. In the analysis of renal pathology images, fine segmentation of morphological lesion regions is of guiding significance for predicting the severity and progression of the disease.

[0031] For example, the literature “Hermsen M, de Bel T, Den Boer M, et al. Deep learning–based histopathologic assessment of kidney tissue[J]. Journal of the American Society of Nephrology, 2019, 30(10): 1968-1979” constructed an integrated U-Net multi-channel semantic segmentation model for segmenting 11 types of tissue structures, including healthy and sclerotic glomeruli, in PAS-stained kidney tissue sections.

[0032] For example, the literature “Ginley B, Jen KY, Han SS, et al. Automated computational detection of interstitial fibrosis, tubular atrophy, and glomerulosclerosis[J]. Journal of the American Society of Nephrology, 2021, 32(4): 837-850” uses DeepLab v2 to segment and quantify atrophic tubules, fibrotic tubulointerstitial tissue, non-sclerotic glomeruli, and sclerotic glomeruli in WSI image patches.

[0033] However, the segmentation targets of the above-mentioned work are limited to the glomerulus, lacking segmentation of the lesion structures within the glomerulus. Currently, the clinical diagnosis of crescentic lesions still mainly relies on the visual assessment of pathologists, and no relevant work has been found to address the semantic segmentation of the three types of crescentic lesions. Given the time-consuming and labor-intensive nature of manual identification and interpretation of crescentic lesions, there is an urgent need to utilize artificial intelligence technology for semantic segmentation and classification of crescentic lesions.

[0034] More importantly, because fibrous crescents, cellular crescents, and cellular-fibrous crescents do not differ significantly in morphological and semantic features, exhibiting a certain degree of continuity and transition, the ambiguity of their morphological boundaries significantly increases the difficulty of extracting discriminative features from the model. The low contrast between the lesion area and adjacent tissues (including texture similarity and staining consistency) and the invasive growth characteristics at the boundaries pose a severe challenge to accurate segmentation of lesion areas. The combined effect of these anatomical characteristics and imaging features collectively creates significant challenges to the accuracy and robustness of crescent segmentation tasks.

[0035] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0036] like Figures 1 to 5 As shown, this embodiment provides a multi-task semantic segmentation and classification system for glomerular crescent lesions, including: The input module is used to receive images of glomerular regions containing crescent lesions extracted from renal biopsy light microscopic pathology (WSI). The encoder module adopts a symmetrical topology and contains four cascaded feature extraction stages. Each stage includes a hybrid coding module and a downsampling module, which are used to perform multi-level downsampling operations on the input image, construct hierarchical feature representations, and output feature maps at different scales. The decoder module consists of three stages, each of which includes an upsampling operation, a gated attention module, and a convolutional layer, which are used to fuse feature maps from the corresponding stages of the encoder and upsample them step by step to restore spatial resolution. The boundary branch module, connected after the lowest layer output features at the end of the decoder, is used to extract boundary-related features of the crescent lesion region and output a binarized boundary prediction map. The main output module, located at the end of the decoder, is used to receive the final output features of the decoder and generate semantic segmentation probability distribution maps for four categories: cellular crescent, cellular fibrous crescent, fibrous crescent, and background region. The processing module is used during the training phase to calculate the composite loss function and backpropagate to update the parameters of the encoder module, decoder module, and boundary branch module based on the semantic segmentation probability distribution map, the binarized boundary prediction map, and the corresponding real semantic segmentation labels and real boundary labels; and during the prediction phase, it is used to output the final crescent lesion region segmentation and classification results based on the semantic segmentation probability distribution map.

[0037] Through the above technical solution, firstly, the system of the present invention can achieve end-to-end precise segmentation and classification. Specifically, the system of the present invention, through the collaborative design of an encoder-decoder symmetrical architecture and boundary branches, enables the system to perform integrated and precise segmentation and pathological type identification of three types of lesion regions and their boundaries in glomerular light microscopy images: cellular crescents, cellular fibrous crescents, and fibrous crescents. This effectively overcomes the shortcomings of traditional methods in segmenting lesion structures within the glomerulus, thereby achieving end-to-end precise segmentation and classification.

[0038] Secondly, the system of the present invention can effectively improve the accuracy of boundary recognition. Specifically, the system of the present invention introduces an independent boundary branch module to specifically extract the boundary features of the lesion area and output a binary boundary map. By enhancing the model's perception ability of low-contrast, invasive crescent-shaped contours through boundary supervision signals, the accuracy of boundary segmentation can be significantly improved, thus effectively improving the accuracy of boundary recognition.

[0039] Third, the system of the present invention can optimize model training and generalization capabilities. Specifically, the processing module of the system of the present invention uses a composite loss function to jointly optimize the segmentation main task and the boundary auxiliary task, and simultaneously improves the robustness of the model to class distribution imbalance (e.g., sparse samples of fibrous crescent bodies) and the modeling ability of complex boundary morphologies during the training process, thereby effectively enhancing the system's generalization performance on diverse pathological images.

[0040] Fourth, the system of the present invention can support digital pathological diagnosis. Specifically, the system of the present invention outputs a four-category probability distribution map (three types of crescent bodies + background) containing semantic information of lesion type, which can provide automated and quantitative technical support for the severity assessment, treatment decision-making and prognostic analysis of kidney disease, reduce the subjectivity and time-consuming nature of manual interpretation, and provide patients with digital pathological diagnosis.

[0041] In this embodiment, it should be noted that, firstly, the system of the present invention employs an encoder-decoder architecture with a symmetrical topology. The encoder constructs a hierarchical representation system of crescent-shaped lesion features through multi-stage, layer-by-layer downsampling; the decoder reconstructs spatial details through progressive upsampling and skip connection mechanisms.

[0042] Second, the encoder architecture employs a hierarchical feature extraction method, consisting of four cascaded feature extraction stages. Each stage comprises a hybrid coding module and a downsampling module, specifically designed for feature learning of crescent-shaped lesions. After the hybrid coding module completes the feature transformation, the downsampling operation progressively reduces the spatial resolution of the feature map, thereby achieving hierarchical abstraction of semantic features.

[0043] The following detailed description of an exemplary embodiment of the present invention is provided in conjunction with the accompanying drawings.

[0044] See also Figure 1 , Figure 1 The overall framework corresponding to the system of the present invention includes the following steps: Step S1: Train the segmentation network to generate a semantic segmentation prediction map of crescent lesions.

[0045] During model training, glomerular images containing crescentic lesions under Masson, PAS, and PASM staining, along with their semantic segmentation labels (the annotation work accurately delineates the crescentic lesion region within the glomerulus, completed during the dataset construction phase), are input into the segmentation network to obtain prediction results for three types of crescentic lesion regions, their lesion types, and non-lesion regions. The segmentation network structure used in this embodiment is as follows: Figure 1 As shown, it consists of an encoder and a decoder.

[0046] For the encoder module, the encoding system adopts a hierarchical feature extraction architecture. Each level integrates a hybrid encoder and a corresponding dimensionality reduction layer. The specific architecture configuration is as follows: Figure 2 As shown, the input image first undergoes primary feature extraction via a convolutional backbone network to form basic low-order feature representations. This backbone network consists of standard convolutional units and spatial extremum pooling units, and the processing method is expressed by the following formula: Output The size is reduced to 1 / 4 of the input image size. Then, a hierarchical encoding process is entered, with the hybrid encoding module including convolutional blocks, multi-scale channel attention, and efficient self-attention (from Vision Transformer). The convolutional blocks are used to capture the local detailed structure of the crescent, enhancing the network's extraction of local features and encoding spatial location information. Let the input feature map from the previous stage be... The processing procedure can be expressed as follows: The convolutional blocks use the convolutional structures from ResNet-34, and each convolutional structure consists of BasicBlocks (such as...). Figure 3 The convolutional blocks between layers differ in the number of BasicBlocks contained in each convolutional block (3, 4, 6, 3) and the number of output channels (64, 128, 256, 512).

[0047] Multi-scale channel attention is used to combine global structural information of crescent lesion regions at different scales, mainly including the following steps: For the input feature map at a certain stage of the encoder First, convolution operations are performed using convolution kernels of different sizes to extract feature representations at different scales: 4×4, 8×8, 16×16, and 32×32, denoted as the set S={4, 8, 16, 32}. The formula is expressed as: in, Indicates the first The stage uses an s×s convolution kernel to perform convolution operations on the input feature map; This represents the feature map of the corresponding token.

[0048] Subsequently, these multi-scale features are concatenated and fused along the channel dimension to form a feature representation containing multi-scale contextual information: The multi-scale feature map obtained by splicing Perform channel attention operations: In the formula, This is a global pooling operation used to aggregate features from different scales; It is a one-dimensional convolutional layer with a kernel size of 3; It is the Sigmoid activation function; This represents element-wise multiplication along the channel dimension.

[0049] Subsequently, the feature map of local location information extracted from the convolutional block is... Channel enhancement feature maps extracted with multi-scale channel attention modules Element-wise addition is performed, and the resulting two-dimensional feature map is converted into a one-dimensional token sequence by the Img2Seq module, which serves as the input to an efficient self-attention module. The processing is expressed by the following formula: No. The processing steps of the phase Transformer are as follows: Seq2Img transforms a one-dimensional sequence into a two-dimensional feature map. MHSA is a multi-head self-attention mechanism, and its calculation formula is as follows: Where Q, K, and V represent query, key, and value, respectively, and are matrices obtained by passing the input through three linear mapping layers; d is the dimension of the input token. FFN (Feed Forward Network) is a feedforward neural network module, formally represented as: in, Let Gaussian error be the activation function of the linear unit. This indicates a fully connected layer.

[0050] After each layer of encoding, downsampling is performed to gradually reduce the size of the feature map and expand the receptive field. The output feature map sizes for the four stages are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. This encoding strategy improves the crescent semantic segmentation network's ability to model multi-scale structures and enhances its adaptability to complex boundary morphology and spatial scale changes in lesion regions, providing feature support for subsequent decoding.

[0051] In the decoder stage, the core design goal of the decoder module is to improve spatial resolution through multi-stage feature reconstruction and achieve the organic integration of cross-level semantic features, ultimately achieving accurate segmentation of complex lesion regions. The overall structure is divided into three stages, each of which includes upsampling, an attention-gated fusion module, and a convolution operation (at the end of the three stages, a 1×1 convolution and softmax activation are used to output the probability distribution of four categories, corresponding to cellular crescents, cellular fibrous crescents, fibrous crescents, and background regions, respectively).

[0052] In each decoding stage, firstly, the low-level detail features from the encoder are upsampled using a transposed convolution to make their size consistent with the size of the high semantic features from the previous decoding layer. These two features are then fused using an attention-gating module. The fused result is then passed through a convolution module to generate the output features for the current stage, as shown in the following formula: In the formula, This represents the output features of each decoder stage. .

[0053] The attention gating module is one of the core components of the decoder, and its structure is as follows: Figure 4 As shown, this module aims to suppress irrelevant background regions and highlight the regional features of the three types of crescents through a guidance mechanism. Essentially, it uses high-level semantic information from the decoder to guide adaptive feature selection and enhancement of low-level features in the encoder. This represents high-level semantic features from the decoder. The low-level semantic features from the encoder are first reduced in dimensionality by a 1×1 convolutional kernel to obtain their respective feature maps. and : In the formula, This represents the 1×1 convolution weights derived from the decoder features. Represents the 1×1 convolution weights from the encoder features; dimensionality reduction and alignment of channels between high-level and low-level semantic features.

[0054] Subsequently, the two are unified in spatial size, then element-wise added and activated by the ReLU function to obtain the fused features. The formula is expressed as follows: in, This is the bias term. At this point, It contains local spatial information provided by the encoder and high-level semantic information provided by the decoder. The fused features are then further processed through a convolution. This is transformed into a single-channel feature map, representing the attention weights at each location: The Sigmoid activation function makes the output... This is an attention weight map within the range [0,1]. Finally, the attention weight map... Compared with the original encoder feature map Element-wise multiplication yields a saliency-enhanced feature map modulated by the attention gating mechanism. : Subsequently, attention enhancement features Features after upsampling by the decoder After fusion, the resulting feature map is input into the convolutional module of the decoder for further decoding. This convolutional module consists of a 3×3 convolutional layer, a BatchNorm layer, and a ReLU function, expressed by the following formula: Additionally, a boundary-assisted prediction branch is introduced at the end of the decoder. It outputs features from the lowest layer. Based on this, a lightweight boundary branch module is added. This module also consists of a 3×3 convolutional layer, a BatchNorm layer, and a ReLU activation function, used to extract boundary-related features (outputting a binarized boundary map for supervised training). The output features are denoted as... The calculation process is expressed as follows: Then, the boundary probability map is obtained through 1×1 convolution. : With boundary labels Supervised training was conducted to improve the model's ability to perceive the boundary structure of the crescent region. Boundary labels. The gradient edges of the crescent region are extracted from the original segmentation labels using the Sobel operator, preserving its boundary with the background, and finally constructing a binary boundary map for supervised training. This boundary map is set to 1 only at the crescent region outline and 0 at other locations, which enhances the model's ability to perceive and locate lesion boundaries.

[0055] In the final output stage, the main branch outputs the probability distributions of four categories through 1×1 convolution and Softmax activation, corresponding to cellular crescents, cellular fibrous crescents, fibrous crescents, and background regions, respectively.

[0056] Step S2: During the training phase, a composite loss function is used to update the segmentation network parameters. Addressing the dual challenges of class imbalance and low tissue boundary discrimination in crescent lesion region segmentation tasks, this invention designs a composite loss function comprising Focal Loss, Dice Loss, and Boundary Loss to comprehensively improve the network's class discrimination and boundary localization capabilities.

[0057] Focal Loss introduces a focusing mechanism on top of traditional cross-entropy loss. By reducing the weight of easily classified samples, it makes the model pay more attention to difficult-to-classify samples. Among the three types of crescent bodies, the number of fibrous crescent body samples is much lower than that of cellular crescent bodies and cellular fibrous crescent bodies. Focal Loss effectively suppresses model training bias caused by data distribution skewness, defined as follows: Where C represents the total number of crescent body categories. , represents the model's predicted probability for category C, where c is the crescent category index. This represents the one-hot encoding of the real label in category C; It is a category balancing weight. It is a focus factor used to control the degree to which the model pays attention to samples of varying difficulty.

[0058] To enhance the model's ability to structurally model the overall morphology of the crescent region and strengthen the learning of the crescent boundary contour, Dice Loss and Boundary Loss are introduced. The definition of Dice Loss is as follows: in, For crescent body category index, It is a pixel index; Represents pixels The predicted probability for category c; Represents pixels The true label for category c; It is a smoothing factor. The definition of Boundary Loss is as follows: in, Indicates the first The boundary label of each pixel; Indicates the first The boundary prediction probability corresponding to each pixel; N is the total number of pixels in the image.

[0059] The final optimization objective function is expressed as: in, , and These are the weighting coefficients for the three loss functions, used to control the influence of the three loss functions. For the crescent-shaped lesion semantic segmentation task, the following settings are configured: The combined use of the three loss functions during training can synergistically enhance the ability of pixel-level category segmentation and boundary modeling, effectively improving the model's structural sensitivity and boundary recognition accuracy for the three types of crescent lesion regions.

[0060] The technical solution proposed in this invention demonstrates significant advantages in the field of glomerular pathological image analysis. This invention primarily addresses the challenge of segmenting crescent-shaped lesions in glomerular images using optical microscopy, and designs a segmentation framework that integrates multi-dimensional features.

[0061] The core innovations of this invention are reflected in three aspects: First, a multi-level encoding and decoding architecture is constructed, and a progressive downsampling strategy combined with cross-scale feature modeling is adopted to realize multi-level feature representation of lesion areas; Second, an attention screening mechanism is innovatively integrated into the cross-layer feature fusion path, and the semantic information of key areas is effectively enhanced through adaptive weighting of feature channels; Finally, a dual-path collaborative learning mechanism is introduced in the decoding stage, and an independent edge feature extraction branch is set up to work in parallel with the main segmentation network, and the contour recognition accuracy is improved by using a joint optimization strategy of boundary prediction loss and region segmentation loss.

[0062] Experimental data (see Table 1 below) show that, compared with traditional segmentation networks U-Net and SegFormer, the average intersection-union ratio of the present invention is improved by 4.95% and 2.53%, respectively; the average recall rate is improved by 5.36% and 2.84%, respectively; and the F1 score is improved by 5.06% and 2.14%, respectively. All indicators are better than the baseline network, and it shows stronger stability and generalization ability.

[0063] Table 1. Comparison of the present invention with traditional segmentation networks. The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-task semantic segmentation and classification system for glomerular crescent lesions, characterized in that, include: The input module is used to receive images of glomerular regions containing crescent lesions extracted from renal biopsy light microscopic pathology (WSI). The encoder module adopts a symmetrical topology and contains four cascaded feature extraction stages. Each stage includes a hybrid coding module and a downsampling module, which are used to perform multi-level downsampling operations on the input image, construct hierarchical feature representations, and output feature maps at different scales. The decoder module consists of three stages, each of which includes an upsampling operation, a gated attention module, and a convolutional layer, which are used to fuse feature maps from the corresponding stages of the encoder and upsample them step by step to restore spatial resolution. The boundary branch module, connected after the lowest layer output features at the end of the decoder, is used to extract boundary-related features of the crescent lesion region and output a binarized boundary prediction map. The main output module, located at the end of the decoder, is used to receive the final output features of the decoder and generate semantic segmentation probability distribution maps for four categories: cellular crescent, cellular fibrous crescent, fibrous crescent, and background region. The processing module is used to calculate the composite loss function and backpropagate to update the parameters of the encoder module, decoder module and boundary branch module during the training phase, based on the semantic segmentation probability distribution map, the binarized boundary prediction map and the corresponding real semantic segmentation labels and real boundary labels; and to output the final crescent lesion region segmentation and classification results during the prediction phase, based on the semantic segmentation probability distribution map.

2. The multi-task semantic segmentation and classification system for glomerular crescent lesions according to claim 1, characterized in that, The feature map sizes output by the four cascaded feature extraction stages of the encoder module are 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image, respectively, and the corresponding number of output channels are 64, 128, 256 and 512.

3. The multi-task semantic segmentation and classification system for glomerular crescent lesions according to claim 1, characterized in that, The hybrid encoding module in the encoder module includes: Convolutional blocks are used to perform convolution operations on the input feature map to extract local location information, resulting in a feature map containing local location information. The multi-scale channel attention module is used to perform multi-scale feature extraction and channel attention weighting on the input feature map to obtain a multi-scale feature map with channel attention weighting. The feature fusion unit is used to add the feature map containing local location information to the multi-scale feature map weighted by channel attention element-wise to obtain a fused feature map; The Transformer module is used to convert the fused feature map into a one-dimensional token sequence for processing, and then convert the processed result back into a two-dimensional feature map for output.

4. The multi-task semantic segmentation and classification system for glomerular crescent lesions according to claim 1, characterized in that, The operation of a single stage of the decoder module includes: The upsampling operation is used to upsample the feature information from the corresponding level of the encoder module; The gated attention module is used to fuse the upsampled feature information with the feature information from the previous decoding stage, and enhance the features of key regions through the attention mechanism; Convolutional layers are used to perform convolution operations on the fused features to generate the output features for the current stage.

5. The multi-task semantic segmentation and classification system for glomerular crescent lesions according to claim 1, characterized in that, The boundary branch module consists of a 3×3 convolutional layer, a BatchNorm layer, and a ReLU activation function connected in sequence.

6. The multi-task semantic segmentation and classification system for glomerular crescent lesions according to claim 1, characterized in that, The composite loss function used by the processing module during the training phase is: In the formula, Represents the focus loss function. Represents the Dice loss function. Represents the boundary loss function. , and These are the weighting coefficients for the three loss functions.

7. The multi-task semantic segmentation and classification system for glomerular crescent lesions according to claim 6, characterized in that, The focus loss function The definition of is: In the formula, The total number of crescent body categories. To balance the weights for each category, Let c be the predicted probability of the model for the crescent body category c and , This is a focus factor used to control the degree to which the model pays attention to samples of varying difficulty. To represent the unique hot encoding of the true label on the crescent category c and .

8. The multi-task semantic segmentation and classification system for glomerular crescent lesions according to claim 6, characterized in that, The Dice loss function The definition of is: In the formula, is the total number of crescent body categories, and c is the crescent body category index. For pixel index, Represents pixels The predicted probability for category c. Represents pixels For the true label of category c, This is a smoothing factor.

9. The multi-task semantic segmentation and classification system for glomerular crescent lesions according to claim 6, characterized in that, The boundary loss function The definition of is: In the formula, The total number of pixels in the image. For pixel index, Indicates the first The boundary label of each pixel and , Indicates the first The boundary prediction probability corresponding to each pixel and .

10. The multi-task semantic segmentation and classification system for glomerular crescent lesions according to claim 6, characterized in that, The weighting coefficients in the composite loss function satisfy: .

Images