Light microscope kidney pathological image analysis method based on artificial intelligence

Abstract

The invention provides a light microscope kidney pathological image analysis method based on artificial intelligence, and relates to the technical field of image recognition, and the method comprises the steps: obtaining a multi-mode light microscope kidney pathological image; performing positioning analysis on the glomerular structure of the multi-mode light microscope kidney pathological image by using the target detection network to obtain a position prediction result; classifying the position prediction result by using a classification network to obtain a classification result; utilizing a semantic segmentation network to perform pathological feature quantitative extraction on the complete glomerulus in the classification result to obtain a quantitative pathological index; and performing multi-level feature fusion on the classification result and the quantitative pathological indexes, and analyzing the obtained multi-level fusion features by using a multi-instance learning network of hierarchical attention to obtain a multi-modal light microscope kidney pathological image analysis result, thereby completing analysis of the light microscope kidney pathological image. The problem that an existing artificial intelligence method is difficult to accurately quantify the light microscope kidney pathological image is solved.

Description

Technical Field

[0001] This specification relates to the field of image recognition technology, and in particular to an artificial intelligence-based method for analyzing kidney pathology images under light microscopy. Background Technology

[0002] Diabetic nephropathy (DN) is one of the most significant microvascular complications of diabetes and has become the leading cause of chronic kidney disease (CKD) and end-stage renal disease (ESRD) worldwide. In clinical practice, accurately distinguishing between diabetic nephropathy (DN) and non-diabetic nephropathy (NDRD) is crucial because their treatment plans and prognoses are drastically different.

[0003] First, the identification process is complex and highly resource-dependent. Traditional identification procedures heavily rely on a "three-in-one" approach combining light microscopy (LM), immunofluorescence (IF), and electron microscopy (EM). Among these, electron microscopy is a key tool for identifying early-stage DN (such as RPSI stage, characterized by glomerular basement membrane (GBM) thickening) and differentiating certain non-denatured disease disorders (NDRDs) (such as minimal change disease, MCD). However, electron microscopy equipment is expensive, complex to operate, and has high maintenance costs, resulting in extremely low adoption rates in primary healthcare institutions.

[0004] Secondly, manual interpretation is subjective and limited. Although the Renal Society of Nephrology (RPS) grading criteria for DN are clear, in practice, the quantitative assessment of lesions (such as the degree of mesangial proliferation, tuberous sclerosis, etc.) relies on the pathologist's personal experience, resulting in significant inter-observer variability and affecting the consistency and accuracy of identification.

[0005] Secondly, existing artificial intelligence (AI)-assisted identification technologies have shortcomings. Although some studies have attempted to apply AI to renal pathology image analysis, most suffer from the following problems: 1) "Black box" models: Some models classify by extracting deep learning features such as color and texture, but their decision-making process is opaque, lacks pathological interpretability, and is difficult to gain the trust of clinicians. 2) Disconnect from clinical guidelines: Most existing AI methods fail to closely align with the RPS grading standards and cannot output quantifiable pathological indicators (such as mesangial proliferation ratio, KW nodules, etc.) that conform to clinical practice, thus limiting their practical value. Summary of the Invention

[0006] To address the aforementioned shortcomings in existing technologies, this invention provides an artificial intelligence-based light microscopic renal pathology image analysis method that solves the problem that existing artificial intelligence methods struggle to accurately quantify light microscopic renal pathology images.

[0007] To achieve the aforementioned objectives, the present invention employs the following technical solution: a light microscopic kidney pathology image analysis method based on artificial intelligence, comprising: S1: Acquire multimodal light microscopic images of kidney pathology; S2: Using an object detection network, the glomerular structure of multimodal light microscopy kidney pathology images is analyzed for localization to obtain location prediction results; a classification network is used to classify the location prediction results to obtain classification results; wherein, the object detection network and the classification network are obtained through training; S3: Using a semantic segmentation network, the pathological features of intact glomeruli in the classification results are quantitatively extracted to obtain quantitative pathological indicators; the semantic segmentation network is obtained through training. S4: Multi-level feature fusion is performed on the classification results and quantitative pathological indicators. The obtained multi-level fused features are analyzed using a hierarchical attention multi-instance learning network to obtain the multimodal light microscopy kidney pathology image analysis results, thus completing the analysis of the light microscopy kidney pathology images. The hierarchical attention multi-instance learning network is obtained through training.

[0008] The beneficial effects of this invention are as follows: This invention provides an artificial intelligence-based light microscopy kidney pathology image analysis method, which extracts ultrastructural features from conventional light microscopy (such as PAS, PASM staining) images, replacing the function of electron microscopy in identifying and differentiating early DN (stage I), thereby reducing identification costs and improving technical accessibility; it establishes an automated, quantitative analysis process that follows the RPS clinical guidelines, accurately segmenting and quantifying key pathological structures (such as mesangial area, KW nodules, sclerotic glomeruli), eliminating subjective differences in human interpretation; it can achieve high-precision identification of DN / NDRD and accurate grading of RPSI-IV under light microscopy; and it constructs a "guideline-heuristic" framework, making its decision-making basis quantify pathological indicators, consistent with pathological logic, and making the AI ​​identification process transparent and reliable.

[0009] Further, S1 includes: Downsampling was performed on the full-field section images of renal biopsy tissue stained with periodic acid-hexamine silver to obtain a low-resolution overview image; By searching for the optimal rigid transformation, the low-resolution overview image is processed to obtain a rigidly aligned image. The normalized cross-correlation score is calculated for the overlapping region of the rigidly aligned image; The rigidly aligned image with the highest normalized cross-correlation score was pixel-level aligned to obtain multimodal light microscopic kidney pathology images.

[0010] By first performing rigid alignment on a low-resolution overview image, then combining the normalized cross-correlation score to select the optimal result, and finally completing pixel-level registration, this method can significantly improve the alignment accuracy between PAS and PASM whole-slice images (WSI), reduce errors and workload caused by manual intervention, achieve high-precision fusion of multimodal light microscopy kidney pathology images, and provide a reliable data foundation for subsequent pathological analysis.

[0011] Furthermore, the expression for the normalized cross-correlation score is: ; in, Represents the normalized cross-correlation score. This represents the first rigidly aligned image. This represents the second rigidly aligned image. Represents the x-coordinate of pixels. Represents the pixel ordinate. This represents the image mean of the first rigidly aligned image. This represents the image mean of the second rigidly aligned image. It represents the square root.

[0012] Furthermore, the target detection network includes: The backbone network is used to extract features from multimodal light microscopy kidney pathology images to obtain location features; The neck network is used to perform multi-scale feature fusion of location features to obtain a fused feature map; The detection head is used to process the fused feature map, predict the bounding box and confidence score of the target, and obtain the location prediction result.

[0013] The above methods can achieve precise localization and automatic classification of glomeruli in multimodal light microscopy kidney pathology images, effectively improving the efficiency and accuracy of pathological analysis, reducing the workload of manual annotation, and providing reliable data support for the diagnosis and classification of kidney diseases, thereby enhancing the scientific nature and reproducibility of clinical decisions.

[0014] Furthermore, the classification network includes: Patch segmentation and linear embedding layers are used to perform image patching and linear embedding on the location prediction results to obtain visual features; Multi-head self-attention layer is used to construct hierarchical features based on visual features through window multi-head self-attention mechanism and shift window multi-head self-attention mechanism; The classification head layer is used to calculate the classification probability of hierarchical features and obtain the classification result.

[0015] Furthermore, the expression for the loss function used in training the object detection network is: ; in, This represents the loss function used for training the object detection network. The weights represent the bounding box regression loss. This represents the bounding box regression loss. The weights representing the targeted loss. Indicates targeted loss. The weights represent the classification loss. Indicates classification loss; The expression for the loss function used in training the classification network is: ; in, This represents the loss function used for training the classification network. Indicates the actual label category. This represents the probability distribution predicted by the classification network.

[0016] The above methods can achieve precise localization and automatic classification of glomeruli in multimodal light microscopy kidney pathology images, effectively improving the efficiency and accuracy of pathological analysis, reducing the workload of manual annotation, and providing reliable data support for the diagnosis and classification of kidney diseases, thereby enhancing the scientific nature and reproducibility of clinical decisions.

[0017] Furthermore, the semantic segmentation network includes: The encoder is used to capture multi-scale contextual information of the complete glomerular image in the classification result using a convolutional neural network and a cascaded ASPP module, and obtain the encoder feature map. The decoder upsamples the encoder feature map and fuses it with the feature map from the shallow layers of the encoder to obtain quantified pathological indicators; the expression for the loss function of the semantic segmentation network training is as follows: ; in, This represents the loss function used in training the semantic segmentation network. Indicates the total number of pixels. This represents the true class one-hot encoding of pixel (i,j). This represents the class probability predicted by the semantic segmentation network.

[0018] This method enables multi-level feature fusion of classification results and quantitative pathological indicators. By utilizing a hierarchical attention-based multi-instance learning network to comprehensively analyze the fused features, it achieves accurate interpretation of multimodal light microscopy kidney pathological images, effectively supports automated diagnosis and grading, and improves the efficiency, accuracy, and reliability of pathological analysis and clinical decision-making.

[0019] Furthermore, the multi-instance learning network for hierarchical attention includes: The instance feature embedding layer is used to map the classification results and quantitative pathological indicators to a high-dimensional feature space through a fully connected network to obtain the feature vector of each instance; among them, the instances include two modalities: PAS and PASM. The attention aggregation layer is used to calculate the importance weight of each instance's feature vector using the self-attention mechanism; the cross-attention mechanism is used to let the slice-level features of one modality be used as the query and the instance features of another modality be used as the key and value, so as to perform inter-modal information interaction and fusion to obtain the slice-level feature vector. The classification head is used to pass slice-level feature vectors through a fully connected layer and an activation function to obtain multimodal light microscopy kidney pathology image analysis results. Attached Figure Description

[0020] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 This is an exemplary flowchart of an artificial intelligence-based light microscopic kidney pathology image analysis method according to some embodiments of this specification; Figure 2 This is an exemplary schematic diagram of an artificial intelligence-based light microscopic kidney pathology image analysis method according to some embodiments of this specification. Detailed Implementation

[0021] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0022] Example Figure 1 This is an exemplary flowchart illustrating an artificial intelligence-based light microscopy method for analyzing kidney pathology images, according to some embodiments of this specification. Figure 1 As shown, the process includes the following steps. In some embodiments, the process may be executed by a processor.

[0023] S1: Acquire multimodal light microscopic images of kidney pathology.

[0024] Multimodal light microscopy renal pathology images are pixel-aligned PAS and PASM images. For example, multimodal light microscopy renal pathology images can include multimodal light microscopy renal pathology images of diabetic nephropathy.

[0025] In some embodiments, the processor can acquire conventional optical microscopic whole-slice scan (WSI) images of renal biopsy tissue from a patient, including at least two different stained sections, such as PAS-stained and PASM-stained images. Using the VALIS image registration algorithm framework, PAS- and PASM-stained WSI images from the same tissue block are aligned at the pixel level to generate registered multimodal image pairs for subsequent collaborative feature analysis.

[0026] In some embodiments, the processor can downsample the full-field slice image (PAS-PASMWSI) of a renal biopsy tissue stained with periodate-hexamine silver to obtain a low-resolution overview image; By searching for the optimal rigid transformation, the low-resolution overview image is processed to obtain a rigidly aligned image. The normalized cross-correlation score is calculated for the overlapping region of the rigidly aligned image; The rigidly aligned image with the highest normalized cross-correlation score was pixel-level aligned to obtain multimodal light microscopic kidney pathology images of diabetic nephropathy.

[0027] In some embodiments, the processor can downsample all whole-field-of-view (WSI) slice images (WSI) by a factor of 100 to generate a low-resolution overview image. Then, for multiple periodic acid-hexamine silver staining (PAS-PASM) image pairs from the same patient, rigid registration (translation and rotation) is performed on the downsampled images, and a normalized cross-correlation (NCC) score is calculated between each pair. Based on the NCC score, the system automatically selects the PAS-PASM image pair with the highest matching degree as the object for subsequent processing. This step aims to quickly find the pair with the most similar tissue content and morphology from multiple consecutive slices, laying the foundation for fine-grained registration.

[0028] In some embodiments, the processor can perform a 100x downsampling of each WSI based on multiple pairs of input PAS-PASMWSI to generate a low-resolution overview map, thereby significantly reducing computational complexity. A rigidly aligned image is an image that maximizes the overlap area between two overlapping low-resolution overview images.

[0029] In some embodiments, the processor may, for each pair of low-resolution PAS-PASM overview maps, systematically search for the optimal rigid transformation (containing only translation and rotation) that maximizes the overlap between the two maps.

[0030] Normalized cross-correlation score is a robust similarity measure that is insensitive to changes in illumination. The score ranges from -1 to 1, with scores closer to 1 indicating a stronger correlation and better matching.

[0031] In some embodiments, the expression for the normalized cross-correlation score can be: ; in, Represents the normalized cross-correlation score. This represents the first rigidly aligned image. This represents the second rigidly aligned image. Represents the x-coordinate of pixels. Represents the pixel ordinate. This represents the image mean of the first rigidly aligned image. This represents the image mean of the second rigidly aligned image. It represents the square root.

[0032] In some embodiments, the processor may output a pair of PAS-PASM images with the highest NCC score as the sole input for subsequent processing.

[0033] In some embodiments, the processor can perform non-rigid fine registration based on full-resolution images. For the best image pairs selected in the first stage, non-rigid registration (also known as soft registration) is performed at their full resolution. This registration process corrects for local, nonlinear deformations caused by physical processes such as tissue slicing, dehydration, and fixation with pixel-level precision by calculating a dense deformation field.

[0034] In some embodiments, the processor can apply the rigid transformation parameters calculated in the first stage to perform initial alignment of the full-resolution image. Subsequently, using a B-spline-based or other differentiable image deformation model, a smooth, dense non-rigid deformation field is calculated by optimizing an objective function that includes an image similarity term (such as mutual information or local NCC) and a regularization term (penalizing the severity of deformation). This deformation field is applied to the image to be registered, and resampling is performed using an interpolation algorithm to generate the final pixel-level aligned PAS and PASM images.

[0035] S2: Using an object detection network, the glomerular structure of multimodal light microscopy kidney pathology images is localized and analyzed to obtain the location prediction results; the location prediction results are then classified using a classification network to obtain the classification results; the object detection network and the classification network are obtained through training.

[0036] Object detection networks are neural networks used to localize and analyze glomerular structures in multimodal light microscopic images of kidney pathology. For example, an object detection network could be a YOLOv11 network.

[0037] In some embodiments, the object detection network may include: The backbone network is used to extract features from multimodal light microscopy kidney pathology images to obtain location features; The neck network is used to perform multi-scale feature fusion of location features to obtain a fused feature map; The detection head is used to process the fused feature map, predict the bounding box and confidence score of the target, and obtain the location prediction result.

[0038] Location features are features related to location information in multimodal light microscopy kidney pathology images.

[0039] A fused feature map is a feature map that combines features from multiple locations.

[0040] The location prediction results are the result of combining the bounding boxes and confidence scores of the multimodal light microscopy renal pathology images with the multimodal light microscopy renal pathology images.

[0041] Classification networks are neural networks used to classify multimodal light microscopic images of kidney pathology. For example, classification networks can include architectures such as the SwingTransformer.

[0042] In some embodiments, the classification network includes: The PatchPartition & LinearEmbedding layer is used to perform image patching and linear embedding on the location prediction results to obtain visual features. The multi-head self-attention layer (SwinTransformerBlocks layer) is used to construct hierarchical features based on visual features through W-MSA and SW-MSA mechanisms; The classification head layer is used to calculate the classification probability of hierarchical features and obtain the classification result.

[0043] Visual features are characteristics of multimodal light microscopy kidney pathology images that are related to visual type.

[0044] The classification results can include intact glomeruli (IG), glomerulosclerotic glomeruli (GS), glomeruli containing Kimmelstiel-Wilson (KW) nodules (KW-G), incomplete glomeruli (ICG), and non-evaluable glomeruli (NGG).

[0045] The network used for glomerular structure classification adopts a SwinTransformer-based architecture. Its initial weights are loaded from the CHIEF pre-trained pathological baseline model to leverage general features learned from large-scale pathological image data and improve classification accuracy.

[0046] In some embodiments, the object detection network can be trained using multiple labeled multimodal light microscopic kidney pathology images. For example, multiple labeled multimodal light microscopic kidney pathology images can be input into an initial object detection network. A loss function is constructed using the labels and the results of the initial object detection network. Based on the loss function, the parameters of the initial object detection network are iteratively updated using gradient descent or other methods. The model training is complete when preset conditions are met, resulting in a trained object detection network. These preset conditions may include loss function convergence, the number of iterations reaching a threshold, etc., and the labels may be the ground truth locations.

[0047] In some embodiments, the classification network can be trained using multiple labeled location predictions. For example, multiple labeled location predictions can be input into an initial classification network. A loss function is constructed using the labels and the results of the initial classification network. Based on the loss function, the parameters of the initial classification network are iteratively updated using gradient descent or other methods. The model training is complete when preset conditions are met, resulting in a trained classification network. These preset conditions may include loss function convergence, the number of iterations reaching a threshold, etc., and the labels may be the true categories.

[0048] In some embodiments, the expression for the loss function used to train the object detection network can be: ; in, This represents the loss function used for training the object detection network. The weights represent the bounding box regression loss. This represents the bounding box regression loss. The weights representing the targeted loss. Indicates targeted loss. The weights represent the classification loss. This represents classification loss.

[0049] In some embodiments, the expression for the loss function during classification network training can be: ; in, This represents the loss function used for training the classification network. Indicates the actual label category. This represents the probability distribution predicted by the classification network.

[0050] S3: Using a semantic segmentation network, the pathological features of intact glomeruli in the classification results are quantitatively extracted to obtain quantitative pathological indicators; the semantic segmentation network is obtained through training.

[0051] Semantic segmentation networks are neural network models used for the quantitative extraction of pathological features.

[0052] In some embodiments, the processor can build and train a semantic segmentation network (such as DeepLabv3+) to accurately segment the mesangial area and KW nodules within the glomerulus. Based on the segmentation results, key pathological indicators are automatically calculated, including: mesangial proliferation score (the proportion of the mesangial area to the total glomerular area) and KW nodule score.

[0053] In some embodiments, the semantic segmentation network includes: The encoder is used to capture multi-scale contextual information of the complete glomerular image in the classification result using a convolutional neural network and a cascaded ASPP module, and obtain the encoder feature map. The decoder is used to upsample the encoder feature map and fuse it with the feature map from the shallow layer of the encoder to obtain quantified pathological indicators.

[0054] The quantitative pathological index is a pixel-level mask of the same size as the input image, where the value of each pixel represents its category, including mesangial proliferation score (the proportion of mesangial area to the total glomerular area) and KW nodule score.

[0055] In some embodiments, the processor can assign a category label to each pixel of the input image, including labels such as background, mesangial region, and KW nodule, and employ a convolutional neural network (ResNet) cascaded with an ASPP (Atrous Spatial Pyramid Pooling) module to capture multi-scale contextual information. The encoder feature map is upsampled and fused with feature maps from shallower encoder layers to recover fine spatial resolution.

[0056] In some embodiments, the semantic segmentation network can be trained using multiple labeled complete glomerular images. For example, multiple labeled complete glomerular images can be input into an initial semantic segmentation network. A loss function is constructed using the labels and the results of the initial semantic segmentation network. Based on the loss function, the parameters of the initial semantic segmentation network are iteratively updated using gradient descent or other methods. The model training is complete when preset conditions are met, resulting in a trained semantic segmentation network. These preset conditions may include loss function convergence, the number of iterations reaching a threshold, etc., and the labels may be real-world quantitative pathological indicators.

[0057] In some embodiments, the expression for the loss function used to train the semantic segmentation network can be: ; in, This represents the loss function used in training the semantic segmentation network. Indicates the total number of pixels. This represents the true class one-hot encoding of pixel (i,j). This represents the class probability predicted by the semantic segmentation network.

[0058] S4: Multi-level feature fusion is performed on the classification results and quantitative pathological indicators. The obtained multi-level fused features are analyzed using a hierarchical attention multi-instance learning network to obtain the analysis results of multimodal light microscopy kidney pathology images, thus completing the analysis of diabetic nephropathy in light microscopy kidney pathology images. The hierarchical attention multi-instance learning network is obtained through training.

[0059] The hierarchical attention multi-instance learning network is a neural network used for analyzing diabetic nephropathy from light microscopic kidney pathology images.

[0060] In some embodiments, the processor can be designed within the MIL framework to incorporate an attention mechanism (cross-attention module) that fuses complementary features from registered PAS and PASM images. PAS features highlight mesangial matrix and cell proliferation, while PASM features highlight basement membrane structure.

[0061] In some embodiments, the input data for the hierarchical attention multi-instance learning network can be a "bag" representing a WSI. The bag contains multiple "instances," each corresponding to a glomerulus, whose features are composed of the outputs of S2 and S3.

[0062] In some embodiments, a multi-instance learning network for hierarchical attention includes: The InstanceFeatureEmbedding layer is used to map the classification results and quantified pathological indicators to a high-dimensional feature space through a fully connected network, obtaining the feature vector of each instance; among them, instances include two modalities: PAS and PASM. The attention aggregation layer is used to calculate the importance weight of each instance's feature vector using the self-attention mechanism; the cross-attention mechanism is used to let the slice-level features of one modality be used as the query and the instance features of another modality be used as the key and value, so as to perform inter-modal information interaction and fusion to obtain the slice-level feature vector. The classification head is used to pass slice-level feature vectors through a fully connected layer and an activation function to obtain multimodal light microscopy kidney pathology image analysis results.

[0063] In some embodiments, the processor can map the features (classification results, quantification metrics) extracted for each glomerular instance in S2 and S3 to a high-dimensional feature space through a fully connected network (Embedding Layer) to obtain the feature vector for each instance. Within the PAS and PASM modalities, a self-attention mechanism is used to calculate the importance weight of each instance, and then the instance features are weighted and summed to obtain a slide-level representation for each modality. A cross-attention mechanism is used, where the slide-level features of one modality are used as the query, and the instance features of the other modality are used as the key and value, thereby realizing the interaction and fusion of information between modalities. The fused final slide-level feature vector is input into one or more fully connected layers, and finally, the WSI-level recognition and classification results are output through a Softmax or Sigmoid activation function.

[0064] The multimodal light microscopy renal pathology image analysis results are two parallel probability vectors: one for binary classification of DN / NDRD, and the other for five-category classification of DN (Class I-IV), used to determine whether the case belongs to diabetic nephropathy (DN) or non-diabetic nephropathy (NDRD). If classified as DN, a specific pathological grade is given according to the RPS grading criteria (Class I, IIa, IIb, III, IV). This grading decision-making process is strongly correlated with the quantified guideline indicators in S3 (mesangial proliferation, KW nodules, and glomerular sclerosis rate).

[0065] In some embodiments, the loss function of the hierarchical attention multi-instance learning network may include Lbinary_cls: loss for the DN / NDRD binary classification task, using binary cross-entropy loss (BCELoss); Lmulti_cls: loss for the DN pathology grading task, using multi-class cross-entropy loss (Cross-EntropyLoss).

[0066] In some embodiments, such as Figure 2 As shown, pathological whole-field sections (WSIs) stained with PAS and PASM are first obtained. For PAS-stained WSIs, this invention first generates a foreground region mask and retains the slide window based on the foreground ratio (ensuring the foreground ratio of the slide window is greater than 10%). Subsequently, glomerular region detection is performed based on YOLOv11, and the patch-level prediction results are fused by combining edge distance weighting and non-maximum suppression (NMS) to ensure that there are no missed or duplicate detections of glomeruli at the WSI level. Finally, the WSI-level glomerular detection results are output, and the glomerular regions are obtained by cropping from them.

[0067] For PASM-stained WSIs, the sections were first reduced by a factor of 100 for rigid registration, and the optimal PAS-PASM registration pair was selected based on the NCC index. Soft registration was then performed at the original resolution to obtain registered PAS-PASM WSIs. Subsequently, identical glomerular regions were cropped from the registered sections. For the cropped PAS-PASM glomerular pairs obtained from the WSIs, this invention trained an RPS-grade-guided glomerular classification model, including: intact glomeruli (IG), glomerulosclerotic glomeruli (GS), glomeruli containing Kimmelstiel-Wilson (KW) nodules (KW-G), incomplete glomerular capsule (ICG), and non-evaluable glomeruli (NGG). This yielded multimodal RPS-grade-guided AI features for each glomerulus, and the multimodal pathological large-scale model AI features were extracted using the CHIEF pathological pedestal model. Further downstream analysis was performed using a multi-instance learning (MIL) model based on hierarchical attention, including DN grading, DN and NDRD classification, and RPS grading for DN cases. Finally, the model outputs were compared with pathologist-annotated results.

[0068] Meanwhile, to perform correlation analysis with the RPS grading guidelines, the obtained PAS glomerular regions were further segmented using Deeplabv3+ to obtain segmentation masks for the glomerular regions, Kimmelstiel–Wilson (KW) nodules, and mesangium. Based on the RPS-guided glomerular classification model results, NGG and ICG glomeruli were filtered out, and RPS features were extracted to obtain indicators including mesangial area ratio, KW nodules, and glomerular sclerosis rate. Finally, a correlation analysis was performed with the RPS guidelines.

[0069] Overall, the system architecture consists of multiple modular neural networks, forming a cascaded processing flow: The front-end processing module includes an image registration network (implemented based on the VALIS framework) and a glomerular detector (based on YOLOv11). This module is responsible for extracting standardized, aligned glomerular image instances containing location information from the raw WSI.

[0070] Core feature quantification module: includes a substructure segmentation network based on DeepLabv3+ for extracting mesangial proliferation and KW nodule features, and a glomerular RPS grading-guided classifier based on CHIEF-SwinTransformer.

[0071] Backend recognition and decision module: It adopts a hierarchical multi-instance learning (MIL) network based on attention mechanism. This network can effectively integrate the local features (classification results and quantitative indicators) and global information of all instances (glomeruli) on the slice, and fuse the features of PAS and PASM modalities to finally generate slice-level recognition and classification.

[0072] This method constructs an end-to-end automated, high-precision, and interpretable identification system by simulating the identification logic of pathology experts (location → classification → quantification → comprehensive judgment).

[0073] In some embodiments of this specification, an artificial intelligence-based light microscopy kidney pathology image analysis method is provided. This method extracts ultrastructural features from conventional light microscopy (e.g., PAS, PASM staining) images, replacing the function of electron microscopy in identifying early-stage DN (stage I) and differential diagnosis, thereby reducing identification costs and improving technological accessibility. It establishes an automated, quantitative analysis process following the RPS clinical guidelines, accurately segmenting and quantifying key pathological structures (e.g., mesangial area, KW nodules, sclerotic glomeruli), eliminating subjective differences in human interpretation. It enables high-precision identification of DN / NDRD and accurate RPSI-IV grading under a light microscope. Furthermore, it constructs a "guideline-heuristic" framework, quantifying pathological indicators for decision-making, aligning with pathological logic, and making the AI ​​identification process transparent and reliable.

Claims

1. A light microscopic kidney pathology image analysis method based on artificial intelligence, characterized in that, include: S1: Acquire multimodal light microscopic images of kidney pathology; S2: Using an object detection network, the glomerular structure of multimodal light microscopy kidney pathology images is analyzed for localization to obtain location prediction results; a classification network is used to classify the location prediction results to obtain classification results; wherein, the object detection network and the classification network are obtained through training; S3: Using a semantic segmentation network, the pathological features of intact glomeruli in the classification results are quantitatively extracted to obtain quantitative pathological indicators; the semantic segmentation network is obtained through training. S4: Multi-level feature fusion is performed on the classification results and quantitative pathological indicators. The obtained multi-level fused features are analyzed using a hierarchical attention multi-instance learning network to obtain the multimodal light microscopy kidney pathology image analysis results, thus completing the analysis of the light microscopy kidney pathology images. The hierarchical attention multi-instance learning network is obtained through training.

2. The method for analyzing kidney pathological images based on artificial intelligence using light microscopy according to claim 1, characterized in that, S1 includes: Downsampling was performed on the full-field section images of renal biopsy tissue stained with periodic acid-hexamine silver to obtain a low-resolution overview image; By searching for the optimal rigid transformation, the low-resolution overview image is processed to obtain a rigidly aligned image. The normalized cross-correlation score is calculated for the overlapping region of the rigidly aligned image; The rigidly aligned image with the highest normalized cross-correlation score was pixel-level aligned to obtain multimodal light microscopic kidney pathology images.

3. The method for analyzing kidney pathological images using light microscopy based on artificial intelligence according to claim 2, characterized in that, The expression for the normalized cross-correlation score is: ； in, This represents the normalized cross-correlation score. This represents the first rigidly aligned image. This represents the second rigidly aligned image. Represents the x-coordinate of pixels. Represents the pixel ordinate. This represents the image mean of the first rigidly aligned image. This represents the image mean of the second rigidly aligned image. It represents the square root.

4. The method for analyzing kidney pathology images based on artificial intelligence using light microscopy according to claim 1, characterized in that, The target detection network includes: The backbone network is used to extract features from multimodal light microscopy kidney pathology images to obtain location features; The neck network is used to perform multi-scale feature fusion of location features to obtain a fused feature map; The detection head is used to process the fused feature map, predict the bounding box and confidence score of the target, and obtain the location prediction result.

5. The method for analyzing kidney pathology images based on artificial intelligence using light microscopy according to claim 1, characterized in that, The classification network includes: Patch segmentation and linear embedding layers are used to perform image patching and linear embedding on the location prediction results to obtain visual features; Multi-head self-attention layer is used to construct hierarchical features based on visual features through window multi-head self-attention mechanism and shift window multi-head self-attention mechanism; The classification head layer is used to calculate the classification probability of hierarchical features and obtain the classification result.

6. The method for analyzing kidney pathological images based on artificial intelligence using light microscopy according to claim 1, characterized in that, The expression for the loss function used in training the object detection network is: ； in, This represents the loss function used for training the object detection network. The weights represent the bounding box regression loss. This represents the bounding box regression loss. The weights representing the targeted loss. Indicates targeted loss. The weights represent the classification loss. Indicates classification loss; The expression for the loss function used in training the classification network is: ； in, This represents the loss function used for training the classification network. Indicates the actual label category. This represents the probability distribution predicted by the classification network.

7. The method for analyzing kidney pathological images based on artificial intelligence using light microscopy according to claim 1, characterized in that, The semantic segmentation network includes: The encoder is used to capture multi-scale contextual information of the complete glomerular image in the classification result using a convolutional neural network and a cascaded ASPP module, and obtain the encoder feature map. The decoder upsamples the encoder feature map and fuses it with the feature map from the shallow layers of the encoder to obtain quantified pathological indicators; the expression for the loss function of the semantic segmentation network training is as follows: ； in, This represents the loss function used in training the semantic segmentation network. Indicates the total number of pixels. This represents the true class one-hot encoding of pixel (i,j). This represents the class probability predicted by the semantic segmentation network.

8. The method for analyzing kidney pathological images based on artificial intelligence using light microscopy according to claim 1, characterized in that, The multi-instance learning network for hierarchical attention include: The instance feature embedding layer is used to map the classification results and quantitative pathological indicators to a high-dimensional feature space through a fully connected network to obtain the feature vector of each instance; among them, the instances include two modalities: PAS and PASM. The attention aggregation layer is used to calculate the importance weight of each instance's feature vector using the self-attention mechanism; the cross-attention mechanism is used to let the slice-level features of one modality be used as the query and the instance features of another modality be used as the key and value, so as to perform inter-modal information interaction and fusion to obtain the slice-level feature vector. The classification head is used to pass slice-level feature vectors through a fully connected layer and an activation function to obtain multimodal light microscopy kidney pathology image analysis results.

Images