Kidney pathological image segmentation model and construction method, segmentation method and system thereof

Abstract

The invention belongs to the field of image analysis, and particularly discloses a kidney pathological image segmentation model, a construction method thereof, a segmentation method and a system, and the method comprises the steps: training a network model through a training set, and enabling the trained network model to be a segmentation model; the training set comprises a kidney pathology image and a corresponding mask label, and the mask label is used for marking the contour of a target area in the kidney pathology image; the network model is constructed based on U-net and comprises an encoder, a KAN module and a decoder, and the encoder adopts a Hira backbone of SAM2; during training, the encoder extracts a feature map based on a kidney pathology image, the KAN module performs marking processing on the feature map, and the decoder generates a segmentation map based on the marked feature map, thereby obtaining a target area contour in the kidney pathology image. According to the invention, accurate segmentation of the kidney tubule contour in the pathological image can be realized, and then representation and analysis of the degree of atrophy of the kidney tubule are provided.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image analysis, and more particularly relates to a kidney pathological image segmentation model and a construction method, a segmentation method and a system thereof. BACKGROUND

[0002] Almost all primary or secondary chronic kidney diseases can cause structural changes in the renal tubules (and interstitium), and the atrophy characteristics of the renal tubules have outstanding prognostic value for kidney transplant evaluation.

[0003] With the development of medical imaging technology, a large amount of medical data has been digitized, and high-resolution whole slide digital images have enabled pathologists to break free from the dependence on microscopes, simplify the tedious operation of repeatedly adjusting magnification to locate the region of interest in traditional microscopy, and break through the time and space limitations of traditional pathological diagnosis in analysis, management and accessibility. However, current clinical diagnosis and treatment still mainly relies on manual reading of pathological slides by pathologists to evaluate the area ratio of the atrophic region of the renal tubule. Due to the shortage of manpower, the large amount of images and the requirement for pathologists to have rich experience, the process of manual interpretation is full of subjectivity, difference, poor repeatability and labor-intensive, time-consuming and laborious, and is prone to misjudgment and missed detection. The rapid development of artificial intelligence technology has brought new opportunities for pathological diagnosis. Image visual processing technology based on deep learning has shown broad application prospects in computer-aided diagnosis, and more and more automated detection systems have been integrated into various clinical tasks to assist doctors in making decisions and improve the efficiency and accuracy of diagnosis.

[0004] Chinese invention patent CN113313685A discloses a method and system for identifying atrophic regions of renal tubules based on deep learning, which realizes the detection of atrophic regions through an improved instance segmentation network Mask-RCNN and a cascade training strategy; patent CN112508854A discloses a UNET-based renal tubule detection and segmentation method, which can identify target regions and separate adjacent renal tubule tissues; patent CN116563261A discloses a multi-stained kidney pathological image segmentation method based on self-supervised meta-learning, which realizes segmentation of the region of interest of different stained kidney pathological sections; and patent CN112330687A discloses a kidney pathological image segmentation model, method and system based on AI technology, which constructs a U-net full convolutional neural network segmentation model set, and takes the optimal binary graph to restore the pathological result graph.

[0005] However, the renal tubule structure on the pathological image is complex, with large differences in shape, size and direction, and a large number of densely distributed tubules. The appearance of normal tubules and atrophic tubules is similar and adjacent, which leads to poor generalization of the deep learning model, affects further analysis of the pathological image, and limits the application of artificial intelligence in clinical practice. SUMMARY

[0006] In view of the above defects or improvement needs of the prior art, the present application provides a kidney pathological image segmentation model and a construction method, a segmentation method and a system thereof, which aims to realize accurate segmentation of the target region contour in the kidney pathological image.

[0007] To achieve the above-mentioned purpose, according to the first aspect of the present application, a kidney pathological image segmentation model construction method is provided, comprising the following steps:

[0008] The network model is trained by the training set, and the trained network model is the segmentation model;

[0009] The training set includes kidney pathological images and corresponding mask labels, and the mask label is the annotation of the target region contour in the kidney pathological image.

[0010] The network model is constructed based on U-net and includes an encoder, a KAN module and a decoder, the encoder uses the Hiera backbone of SAM2; during training, the encoder extracts a feature map based on the kidney pathological image, the KAN module performs tokenization processing on the feature map, and the decoder generates a segmentation map based on the tokenized feature map, thereby obtaining the target region contour in the kidney pathological image.

[0011] As a further preferred, in the network model, multi-scale semantic information is fused through a skip connection, specifically as follows:

[0012] Z k * =Cat(Z k+1 * ,Z k )

[0013] Wherein, Cat(·) represents a feature concatenation operation, Z k represents the feature map Z k+1 * output by the k+1 layer decoder, and Z k * represents the feature map output by the k layer decoder.

[0014] As a further preferred, the feature map extracted by each layer of the encoder is merged with the partial feature of the corresponding layer of the decoder after passing through the EMA attention module and the attention gate unit.

[0015] As a further preferred, the construction method of the training set is as follows:

[0016] A plurality of kidney pathological images are obtained, and the target region contour in the kidney pathological image is annotated to obtain a mask label;

[0017] Abnormal renal pathology images were removed, and the remaining renal pathology images were subjected to color normalization and histogram equalization, and the renal pathology images were expanded by random rotation and flipping;

[0018] Finally, the kidney pathology images and their corresponding mask labels are cropped into image patches of fixed size.

[0019] As further preferred, the target area is one of renal tissue, glomerulus, renal artery and renal tubule.

[0020] According to a second aspect of the present invention, a kidney pathology image segmentation model is provided, which is constructed using the above-mentioned kidney pathology image segmentation model construction method.

[0021] According to a third aspect of the present invention, a method for segmenting a kidney pathology image is provided, comprising the following steps:

[0022] The kidney pathology image is segmented into image blocks and then input into the above segmentation model to obtain the segmentation map of the image blocks, which are then spliced ​​and restored to obtain the segmentation map of the kidney pathology image, thereby obtaining the outline of the target area.

[0023] As a further preferred method, after obtaining the segmentation map of the kidney pathology image, the target area contour in the segmentation map is identified, and a connected domain analysis is performed to filter out noise impurities with an area smaller than a preset threshold; then the retained contour is subjected to a morphological opening operation of first corrosion and then expansion and a watershed algorithm to obtain the final target area contour.

[0024] According to a fourth aspect of the present invention, a method for measuring renal tubular atrophy is provided, comprising the following steps:

[0025] The renal pathology images were input into the segmentation models with the target areas being renal tissue, glomerulus, renal artery, and renal tubule, respectively, to obtain the contours of renal tissue, glomerulus, renal artery, and renal tubule on the renal pathology images; each segmentation model was constructed using the above-mentioned renal pathology image segmentation model construction method;

[0026] The degree of renal tubular atrophy α is:

[0027] α=(S all -S glomerulus -S artery -S tubule ) / S all

[0028] Among them, S all 、S glomerulus 、S artery 、S tubule They are the contour areas of renal tissue, glomerulus, renal artery, and renal tubule, respectively.

[0029] According to a fifth aspect of the present invention, a renal tubular atrophy measurement system is provided, comprising a processor, wherein the processor is configured to execute the above-mentioned renal tubular atrophy measurement method.

[0030] In general, the above technical solutions conceived by the present invention have the following technical advantages compared with the existing technology:

[0031] 1. This invention builds a segmentation model based on the SAM2 large model and the KAN module. The encoder uses SAM2's Hiera backbone and extracts multi-scale features through multi-stage downsampling, providing rich global semantic information for the subsequent decoder. Combined with KAN, it captures global dependencies at a lower computational cost, reducing the number of parameters while maintaining sensitivity to local details in pathological images, thereby enhancing the model's feature extraction capabilities and improving segmentation accuracy. The model based on deep neural networks can improve physician efficiency and provide key reference information for subsequent renal pathology analysis.

[0032] 2. The segmentation model of the present invention embeds the EMA attention module and the attention gating unit in the skip connection, so that the decoding process can fuse multi-scale semantic information, filter out the noise in the low-level semantic information, suppress irrelevant background features, and focus more on the target area of ​​interest.

[0033] 3. The present invention adopts image standardization operations such as histogram equalization to highlight the characteristics of pathological structures such as renal tubules, improve visibility, provide clear and accurate image data for model segmentation, and improve segmentation accuracy.

[0034] 4. The present invention adopts connected domain analysis to filter out noise impurities on the segmentation map, and performs morphological operations of first corrosion and then expansion and watershed algorithm processing to effectively fill the holes in the initial segmentation and improve the adhesion phenomenon.

[0035] 5. The method for measuring renal tubular atrophy provided by the present invention takes a different approach by removing the glomeruli, renal arteries, and renal tubules from the renal tissue to represent the renal interstitial region, and indirectly reflects the degree of renal tubular atrophy through mapping. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of a method for segmenting kidney pathology images according to an embodiment of the present invention;

[0037] Figure 2 An example of a pathological image of a renal tubule and its mask label according to an embodiment of the present invention;

[0038] Figure 3 Schematic diagram of cropping and splicing operations of an entire pathological image according to an embodiment of the present invention;

[0039] Figure 4This is a schematic diagram of the segmentation model structure according to an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of the encoder network architecture according to an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of the attention gating mechanism according to an embodiment of the present invention;

[0042] Figure 7 This is a schematic structural diagram of a renal tubular atrophy measurement system according to an embodiment of the present invention;

[0043] Figure 8 Schematic diagram of the final kidney tissue segmentation result and evaluation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0045] The embodiment of the present invention provides a method for constructing a kidney pathology image segmentation model. Figure 1 As shown, the following steps are included:

[0046] S1. Obtain a real kidney pathology image, perform pixel-level annotation on the contour of the target area, obtain the mask label corresponding to the original image, and form an image set.

[0047] The pathological sections of renal biopsy collected clinically are stained with H&E to highlight the tissue structure and morphology, and then converted into digital pathological images by the scanner. Doctors can use the Labelme tool to mark the outline of the target area on the pathological image to generate a JSON file, which is further converted into a binary mask label PNG, such as Figure 2 shown.

[0048] Specifically, the target area may be renal tissue, glomerulus, renal artery or renal tubule.

[0049] S2. Remove abnormal samples from the image set, preprocess and expand the remaining image set, and then crop the image and its corresponding mask into small image blocks of fixed size to construct a training set.

[0050] Specifically, blurred, bubble-containing, and abnormally stained samples were removed, and the remaining renal pathology images were preprocessed. The renal pathology images were expanded by random rotation and flipping to increase the diversity of the dataset and thus improve the generalization ability of the model.

[0051] Further preprocessing includes staining normalization and histogram equalization. Simple and effective staining normalization methods such as Macenko, Reinhard, and Vahadane can be used to make subsequent image analysis more accurate. Histogram equalization can also be used to enhance contrast and highlight the contours of pathological features.

[0052] Furthermore, taking the target area as the renal tubule as an example, the specific operation of cropping the image and its corresponding mask into small image blocks is explained: Figure 3 As shown, first set the size of the sliding window to be consistent with the image input size required by the segmentation model. In this embodiment, the value is 512×512 (i.e., the size of each small image block), which not only ensures that the features of the renal tubules are clear and conducive to segmentation and recognition, but also retains some boundary information around the renal tubules to improve the robustness of the model. Then, the right and lower mirror images of the pathological image are filled to an integer multiple of 512 for easy cropping. Taking into account that segmentation will cause incomplete renal tubular structure, an overlapping cropping method is adopted, and a step size of 256 is taken in this embodiment. In the obtained small image blocks, image blocks with a tissue ratio greater than a preset 80% threshold are screened as data sets, and divided into training sets and test sets in a ratio of 8:2.

[0053] S3. Iteratively train the network model using the training set to minimize the loss function until convergence. The trained network model is the segmentation model.

[0054] The network model is built based on the encoder-decoder structure of U-net, such as Figure 4 As shown, it includes an encoder, a KAN module and a decoder. The encoder extracts a feature map based on the kidney pathology image, the KAN module tokenizes the feature map, and the decoder obtains a segmentation map based on the tokenized feature map.

[0055] Furthermore, the encoder adopts the Hiera backbone of SAM2, and with the help of a layered pyramid structure, it gradually compresses the image resolution and expands the channel dimension through multi-stage downsampling to extract multi-scale features. When a 512×512 image is input, the Hiera layer sequentially outputs feature maps with resolutions of 128×128, 64×64, and 32×32, providing rich global semantic information for the subsequent decoder. Figure 5 As shown, each Hiera layer consists of two linear layers, two GeLU activation functions, a multi-scale module, and an MLP layer. A lightweight adapter is inserted before each multi-scale module, freezing all Hiera parameters. Only the small number of parameters introduced by the adapter needs to be trained for efficient fine-tuning, lowering the deployment threshold and adapting to medical scenarios with limited computing power.

[0056] After the encoder completes feature extraction, the feature map is tokenized through the KAN block at the neck. This is then combined with deep convolution (DwConv) and residuals to achieve efficient nonlinear mapping. Furthermore, the EMA attention module and attention gate are embedded in the skip connections, enabling the decoding process to integrate multi-scale semantic information, filter out noise in low-level semantic information, suppress irrelevant background features, and focus on the target area of ​​interest.

[0057] Specifically, KAN can be expressed as a stack of learnable nonlinear activation functions, replacing the traditional MLP. The formula is:

[0058]

[0059] Among them, Φ represents the activation function matrix and Z represents the input feature vector.

[0060] Output after KAN layer processing:

[0061] Z k =LN(Z k-1 +DwConv(KAN(Z K-1 )))

[0062] Among them, LN represents layer normalization, DwConv represents depth convolution, and Z K-1 , Z k Represent the input and output features of the KAN layer respectively.

[0063] Decode the concatenated output of upsampling and skip connections:

[0064] Z k * =Cat(Z k+1 * , Z k )

[0065] Among them, Cat(·) represents the feature concatenation operation, Z k Represents the feature vector of the k-layer downsampling output to the k-layer decoder through the skip connection, Z k+1 * Represents the feature vector of the k+1 layer upsampled output, Z k * Represents the feature vector of the upsampled output of layer k.

[0066] like Figure 6 As shown, the attention gating module can be expressed as:

[0067] g1,g2,g3=Split(g)

[0068] g *=σ(ω(g1+x)+b+x)·x+σ(g2)·tanh(g3)

[0069] Among them, the high-level semantic feature g is evenly divided into three parts g1, g2, g3, g * represents the gated output, x represents the low-level semantic feature, ω and b represent the weights, and σ and tanh represent the activation function.

[0070] The main parameters set during model training in this example are: batch size = 8, initial learning rate = 0.0001, weight decay = 0.0001, and backpropagation training epochs = 300 using the Adam optimizer with a cosine annealing learning rate scheduling strategy. A loss function combines BCE and Dice, with metrics such as IoU and F1 score used for quantitative evaluation of segmentation results. The model is iterated until convergence.

[0071] Based on the above kidney pathology image segmentation model, the embodiment of the present invention also provides a kidney pathology image segmentation method, such as Figure 1 As shown, the following steps are included:

[0072] S4: The entire kidney pathology image to be segmented is subjected to the same cropping operation as in S2. The resulting small image blocks are then fed into the segmentation model. The mask output by the segmentation model is then reassembled to the original image size based on the cropping positions. After post-processing operations such as denoising, the complete outline of the target area on the segmented image is captured and outlined on the original image, achieving accurate segmentation and visualization of the target area on the pathology image.

[0073] Specifically, first, the pathological image to be segmented is subjected to the overlapping cropping operation in S2 to obtain a list of small images, which are named "row-column", i.e., position coordinates. The corresponding masks output by the model are named the same. According to the preset batch size, the images are input into the trained segmentation model in batches, and the corresponding mask results are output. Then, the images are spliced ​​and restored to the original image size according to the position coordinates at the time of cropping. The overlapping areas take the prediction results with the higher IoU (intersection over union) value. Taking the renal tubules as an example, Figure 3 As shown in the figure, due to overlapping cropping, a single renal tubule may appear in four small images. This means that a single renal tubule will generate at most four different predictions, which may be inconsistent with each other. When restoring the original image size by position coordinates, the predictions must be merged, and the one with the highest IoU value must be retained as the final segmentation result.

[0074] Furthermore, the OpenCV findContours function is used to identify the target area contours within the mask. A threshold is set for connected domain analysis to filter out small noise impurities, thereby reducing interference in subsequent analysis. In this example, a threshold of 8000 is used. The retained contours are then subjected to a morphological opening operation (erosion followed by dilation) and a watershed algorithm to fill holes in the mask and improve adhesion. The post-processed target area contours are then drawn on the original image using the OpenCV drawContours function for visualization.

[0075] Specifically, the erosion operation uses a 10×5 elliptical kernel to perform an erosion operation, eliminating edge pixels and separating adhered contours. The dilation operation uses the same kernel to perform a dilation operation, expanding the tubule boundary, filling holes in the contour, and restoring the contour that was shrunk by the erosion operation.

[0076] For deep adhesion areas, a watershed algorithm based on distance transformation is used. OpenCV's distanceTransform function is used to obtain the Euclidean distance from the foreground pixel to the nearest background pixel. Peak detection is used to create markers, and the watershed function is used to perform flood fill segmentation. The algorithm treats the image as a terrain surface, with areas of high distance values ​​acting as watersheds. This simulates a water filling process to achieve precise segmentation. Compared to simple thresholding, watershed can improve segmentation accuracy of adhesion structures by 35%.

[0077] An embodiment of the present invention provides a method for measuring renal tubular atrophy, comprising the following steps:

[0078] The target regions are set as kidney tissue, glomerulus, renal artery, and renal tubule, and the segmentation models for kidney tissue, glomerulus, renal artery, and renal tubule are constructed respectively using the aforementioned method;

[0079] The renal pathology image is input into the segmentation model with the target areas being renal tissue, glomerulus, renal artery, and renal tubule, and the contours of renal tissue, glomerulus, renal artery, and renal tubule on the renal pathology image are obtained accordingly;

[0080] The degree of tubular atrophy α is:

[0081] α=(S all -S glomerulus -S artery -S tubule ) / S all

[0082] Among them, S all 、S glomerulus 、S artery 、S tubule They are the contour areas of renal tissue, glomerulus, renal artery, and renal tubule, respectively.

[0083] The embodiment of the present invention provides a renal tubular atrophy measurement system, such as Figure 7 As shown, it includes an image preprocessing module, a feature segmentation module, and atrophy measurement module, wherein:

[0084] The image preprocessing module divides the renal pathology images obtained in the clinic into small image lists according to the position coordinates, names them as "row-column", and performs standardization processing such as histogram equalization and staining normalization.

[0085] The feature segmentation module includes a renal tissue segmentation model, a glomerulus segmentation model, a renal artery segmentation model, and a renal tubule segmentation model; the preprocessed small images are input into each segmentation module in batches for segmentation to obtain the contours of the renal tissue, glomerulus, renal artery, and renal tubule.

[0086] The atrophy measurement module first restores the segmentation mask to the original image size according to the coordinates at the time of segmentation, and then performs post-processing operations such as denoising, connected domain analysis, morphological opening operation, and watershed algorithm. The contours of the processed renal tissue, glomerulus, renal artery, and renal tubules are drawn on the original image in different colors using OpenCV's findContours and drawContours functions to achieve visualization. The contour area of ​​the renal tissue, glomerulus, renal artery, and renal tubule is then measured using the contourArea function, and the area of ​​the renal interstitium is replaced by (the contour area of ​​the renal tissue - the contour area of ​​the glomerulus, renal artery, and renal tubule). The ratio of the area of ​​the renal interstitium to the entire renal tissue is mapped to the degree of atrophy of the renal tubules, and the atrophy percentage is marked on the original renal pathology image, such as Figure 8 shown.

[0087] Furthermore, the system also includes a display module, which provides an interactive interface for users to input pathological images for analysis and displays the segmentation results and evaluation scores. Users can pan and zoom to view detailed information about each area and can correct incorrect predictions by redrawing the contours directly on the result image.

[0088] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a kidney pathology image segmentation model, characterized in that: The steps include: The network model is trained through the training set, and the trained network model is the segmentation model; The training set includes kidney pathology images and corresponding mask labels, where the mask labels are annotations of the contours of target areas in the kidney pathology images; The network model is built based on U-net and includes an encoder, a KAN module, and a decoder. The encoder uses the Hiera backbone of SAM2. During training, the encoder extracts feature maps based on kidney pathology images, the KAN module tokenizes the feature maps, and the decoder generates a segmentation map based on the tokenized feature maps, thereby obtaining the contours of the target area in the kidney pathology images.

2. The method for constructing a renal pathology image segmentation model according to claim 1, wherein: In the network model, multi-scale semantic information is fused through skip connections, as follows: Z k * =Cat(Z k+1 * ,Z k ) Among them, Cat(·) represents the feature concatenation operation, Z k Represents the feature map Z output by the k-layer encoder to the k-layer decoder via skip connection k+1 * represents the feature map output by the k+1 layer decoder, Z k * Represents the feature map output by the k-layer decoder.

3. The method for constructing a renal pathology image segmentation model according to claim 2, wherein: The feature map extracted by each layer of encoder is merged with the partial features of the decoder of the corresponding layer after passing through the EMA attention module and the attention gating unit.

4. The method for constructing a renal pathology image segmentation model according to claim 1, wherein: The training set is constructed as follows: Acquire multiple renal pathology images, mark the contours of the target areas in the renal pathology images, and obtain mask labels; Abnormal renal pathology images were removed, and the remaining renal pathology images were subjected to color normalization and histogram equalization, and the renal pathology images were expanded by random rotation and flipping; Finally, the kidney pathology images and their corresponding mask labels are cropped into image patches of fixed size.

5. The method for constructing a renal pathology image segmentation model according to any one of claims 1 to 4, wherein: The target area is one of renal tissue, glomerulus, renal artery, and renal tubule.

6. A kidney pathology image segmentation model, characterized in that: The renal pathology image segmentation model is constructed using the method for constructing the renal pathology image segmentation model according to any one of claims 1 to 5.

7. A kidney pathology image segmentation method, characterized in that: The steps include: The kidney pathology image is segmented into image blocks and then input into the segmentation model as claimed in claim 6 to obtain a segmentation map of the image blocks, which is then spliced ​​and restored to obtain a segmentation map of the kidney pathology image, thereby obtaining the outline of the target area.

8. The kidney pathology image segmentation method according to claim 7, wherein: After obtaining the segmentation map of the kidney pathology image, the target area contour in the segmentation map is identified, and connected domain analysis is performed to filter out noise impurities with an area smaller than a preset threshold; then the retained contour is subjected to a morphological opening operation of first erosion and then dilation, and the watershed algorithm is used to obtain the final target area contour.

9. A method for measuring renal tubular atrophy, characterized in that: The steps include: The renal pathology image is input into the segmentation models whose target regions are renal tissue, glomerulus, renal artery, and renal tubule, respectively, to obtain the contours of the renal tissue, glomerulus, renal artery, and renal tubule on the renal pathology image; each segmentation model is constructed using the renal pathology image segmentation model construction method according to any one of claims 1 to 5; The degree of renal tubular atrophy α is: α=(S all -S glomerulus -S artery -S tubule ) / S all Among them, S all 、S glomerulus 、S artery 、S tubule They are the contour areas of renal tissue, glomerulus, renal artery, and renal tubule, respectively.

10. A renal tubular atrophy measurement system, characterized in that: The device comprises a processor configured to execute the method for measuring renal tubular atrophy according to claim 9.

Images