IgA nephropathy prognosis method, system and device based on multi-modal data

Abstract

The invention discloses an IgA nephropathy prognosis method, system and device based on multi-modal data, and belongs to the technical field of image data processing, the method comprises the following steps: collecting historical data including clinical data and an IgA immunofluorescence map; based on a visual identification method, pathological features are extracted from the IgA immunofluorescence image; screening clinical characteristics; based on a machine learning method, the training set is trained according to pathological features and clinical features, a prognosis model is obtained, and the prognosis model is used for IgA nephropathy prognosis. On the basis of a computer vision method, pathological features are extracted from an IgA immunofluorescence map, and the prognosis of the IgA nephropathy is predicted in combination with clinical features, so that automatic prediction is facilitated, mistakes and omissions caused by artificial naked eye recognition are avoided, stable prediction ability is expressed, and multi-modal data fusion reflects a gain effect on long-term prognosis prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method, system and device for IgA nephropathy prognosis based on multimodal data. Background Art

[0002] IgA nephropathy (IgAN) is characterized by dominant or codominant IgA deposition within the glomerular mesangium. It is considered a primary glomerular disease and the most common form of primary glomerulonephritis. Studies have shown that IgA deposition in the mesangium and glomerular capillary loops is associated with more severe clinical and histopathological manifestations.

[0003] The core pathological hallmark of IgA nephropathy is the presence of immune complex deposits, primarily IgA, in the glomerular mesangium. These deposits are typically diffusely distributed in granular or clumpy forms and may extend along capillary loops in some cases. Immunofluorescence examinations, through specific antibody labeling, can visually demonstrate the localization of IgA within the glomeruli and are essential for diagnosing the disease. IgA immunofluorescence examinations, through specific antibody labeling, can visually demonstrate the localization of IgA within the glomeruli and are essential for diagnosing the disease. IgA immunofluorescence images can display features such as the location, brightness, and shape of the deposits. Currently, these features are extracted from images using the naked eye. However, there are issues of subjectivity in image reading, heterogeneity between doctors, and the naked eye may miss key information. Summary of the Invention

[0004] In response to the above-mentioned technical problems existing in the prior art, the present invention provides an IgA nephropathy prognosis method, system and device based on multimodal data, extracts pathological features based on IgA immunofluorescence images, and establishes a prognosis model based on the pathological features and clinical data, and performs IgA nephropathy prognosis through the prognosis model.

[0005] The present invention discloses a prognosis method for IgA nephropathy based on multimodal data, comprising the following steps: collecting historical data and randomly dividing the data into a training set and a test set, wherein the historical data includes clinical data and IgA immunofluorescence images; extracting pathological features from the IgA immunofluorescence images based on a visual recognition method; screening clinical features based on the clinical data; and training the training set based on the pathological features and clinical features based on a machine learning method to obtain a prognosis model, wherein the prognosis model is used for the prognosis of IgA nephropathy.

[0006] Preferably, the method for extracting pathological features comprises the following steps: dividing the immunofluorescence image into a plurality of small blocks; Establishing labels for the small blocks to obtain a second training set; Based on the machine learning method, the second training set is trained to obtain a recognition model; Based on the recognition model, predict the probability of small blocks and their labels; Based on the probabilities and their labels, a pathological feature is constructed.

[0007] Preferably, the method for extracting pathological features comprises the following steps: The IgA immunofluorescence image was divided into several small blocks; Create a label for the block Patch prob , obtain the second training set; Based on the machine learning method, the second training set is trained to obtain a recognition model; Based on the recognition model, the probability of small blocks and their labels are predicted to construct the third feature; Combined with histogram statistics, the third small block is binned to obtain buckets; Construct the fourth feature based on the probability of the small blocks in the bucket and the frequency of the label; The third feature is vectorized based on the bag-of-words model, and the fifth feature is obtained based on the TF-IDF transformation; The fourth feature and the fifth feature are fused to obtain a pathological feature.

[0008] Preferably, the specific method for extracting pathological features includes: Preprocessing of IgA immunofluorescence images; The pretreated IgA immunofluorescence image was divided into several small blocks; screening valid small blocks from the plurality of small blocks; Normalize the valid small blocks to obtain the third small block; Create labels for the third small piece and obtain the second training set; Based on the machine learning method, the second training set is trained to obtain a recognition model; Based on the recognition model, predict the probability of the third small block; According to the prediction threshold and probability, the label of the third small block is obtained, and the third feature is constructed according to the probability and label; Combined with histogram statistics, the probability distribution of the third small block is obtained; Bin the third small piece to obtain buckets; After calculating the probability of the third small block and the frequency of the label in the bucket, normalization is performed to construct the fourth feature of the bucket; After vectorizing the third feature based on the bag-of-words model, the fifth feature is obtained based on the TF-IDF transformation. Fusing the fourth feature and the fifth feature to obtain a sixth feature vector and a feature set; The sixth eigenvector of the feature set is screened to obtain pathological features.

[0009] Preferably, the pathological feature or the sixth eigenvector is expressed as: ; in, is the feature connector, features fusion Denoted as the sixth eigenvector or pathological feature, Histo Prob and Histo Pred is the parameter of the fourth characteristic, Bow prob and Bow pred is the parameter of the fifth feature.

[0010] Preferably, the method of screening effective small blocks from a plurality of small blocks comprises: Determining whether the proportion of bright pixels in the small block exceeds a first threshold; If so, exclude the small piece; If not, keep the small piece.

[0011] Preferably, the machine learning method for training the recognition model includes any of the following models: ResNet18, ResNet50, Inception_v3 and DenseNet121; Prognostic models included: Cox proportional hazards model or nomogram model; Based on the Min-Max normalization method, the probability and frequency of the label in the bucket are normalized.

[0012] Preferably, the method for screening the sixth eigenvector includes: A correlation-based filtering method retains the sixth eigenvector whose Pearson correlation coefficient exceeds the fourth threshold; Using univariate Cox regression, the sixth eigenvector with a p-value below the fifth threshold was retained; Select non-zero features to obtain pathological features.

[0013] Preferably, clinical characteristics are screened by combining univariate analysis, multivariate analysis, and Pearson's method; Clinical characteristics were selected from the following categories: age, proteinuria, mean arterial pressure, baseline estimated glomerular filtration rate, RAS inhibitor use, immunosuppressant use, mesangial proliferation, endothelial proliferation, segmental sclerosis, tubular atrophy, and cellular crescents.

[0014] The present invention also provides a system for implementing the above-mentioned IgA nephropathy prognosis method, comprising an acquisition module, a pathological feature recognition module, a clinical feature extraction module and a prognosis module. The acquisition module is used to collect historical data; The pathological feature recognition module is used to extract pathological features from IgA immunofluorescence images based on visual recognition methods; The clinical feature extraction module is used to extract clinical features based on the clinical data; The prognostic module is used to predict clinical data and their IgA immunofluorescence images based on the prognostic model to obtain the prognosis.

[0015] The present invention also provides a device comprising a processor and a memory, The memory stores codes, and when the codes are processed by the processor, the above-mentioned IgA nephropathy prognosis method is implemented.

[0016] Compared with the existing technology, the beneficial effects of the present invention are: based on the computer vision method, pathological features are extracted from IgA immunofluorescence images, and the prognosis of IgA nephropathy is predicted in combination with clinical characteristics, which is conducive to automated prediction and avoids errors and omissions caused by manual visual recognition, demonstrating robust prediction capabilities. Multimodal data fusion has a gain effect on long-term prognosis prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of the IgA nephropathy prognosis method based on multimodal data of the present invention; Figure 2 It is a flow chart of the method for extracting pathological features; Figure 3 It is the ROC curve of the training set of the recognition model; Figure 4 is the test set ROC curve of the recognition model; Figure 5 It is the KM curve diagram of the training set of the pathological model; Figure 6 It is the KM curve diagram of the test set of the pathological model; Figure 7 This is a diagram showing the activation of Grad-CAM; Figure 8 This is the structure diagram of the COX univariate regression analysis of clinical characteristics; Figure 9 It is the KM curve diagram of the training set of the first model; Figure 10 This is the KM curve of the test set of the first model; Figure 11 It is the KM curve diagram of the training set of the prognostic model; Figure 12 It is the KM curve diagram of the test set of the prognostic model; Figure 13 is the nomogram model for survival risk; Figure 14 It is a system logic block diagram of the present invention. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0019] The present invention is described in further detail below with reference to the accompanying drawings: Overview: IgA nephropathy (IgAN) is characterized by immunofluorescence detection of IgA deposits with or without C3 in the mesangial region. Microscopic pathological changes vary widely, ranging from mild lesions to crescentic nephritis (including minimal change disease, endocapillary proliferation, FSGS, membranoproliferative, crescentic, proliferative sclerosis, combined membranous nephropathy, and combined MCD). The most common pathological type is mesangial cell proliferation and matrix increase in the mesangial region, often accompanied by segmental sclerosis, crescents of varying sizes, and endothelial cell proliferation. The most popular classification is the Oxford EST (MEST) classification, proposed in 2009. It assesses pathological severity based on M (mesangial cell proliferation), E (endothelial cell proliferation), S (segmental sclerosis), and T (extent of tubulointerstitial damage in the renal cortex). This classification method relies primarily on subjective visual identification.

[0020] The pathogenesis of immunoglobulin A (IgA) nephropathy is the production of galactose-deficient IgA1 (Gd-IgA1) antibodies. Gd-IgA1 forms immune complexes that accumulate in the renal mesangium and cause renal damage. Gd-IgA1 not only has a tendency to self-aggregate but also amplifies the pathogenic effects of renal deposition through abnormal binding to serum proteins such as fibronectin. Deposition of IgA in the mesangium and glomerular capillary loops is associated with more severe clinical and histopathological manifestations.

[0021] The first aspect of the present invention provides a method for prognosticating IgA nephropathy based on multimodal data, such as Figure 1 As shown, the following steps are included: Step 101: Collect historical data and randomly divide it into a training set and a test set.

[0022] The historical data includes clinical data and IgA immunofluorescence images. The detection method of IgA immunofluorescence images is a prior art and will not be described in detail in the present invention.

[0023] Step 102: Extract pathological features from the IgA immunofluorescence image based on a visual recognition method.

[0024] Step 103: Screen clinical characteristics based on the clinical data.

[0025] Step 104: Based on a machine learning method, the training set is trained according to the pathological characteristics and clinical characteristics to obtain a prognostic model.

[0026] The machine learning method may adopt a Cox proportional hazards model to analyze the effects of multiple covariates on survival time.

[0027] Step 105: Validate the prognostic model using the validation set.

[0028] Step 106: Based on the prognostic model, the clinical data and the IgA immunofluorescence image are predicted to obtain the prognosis.

[0029] Based on computer vision methods, pathological features are extracted from IgA immunofluorescence images and combined with clinical characteristics to predict the prognosis of IgA nephropathy. This facilitates automated prediction, avoids errors and omissions caused by manual visual recognition, demonstrates robust prediction capabilities, and realizes multimodal data fusion. Multimodal data fusion has a gain effect on long-term prognosis prediction.

[0030] In step 101, historical data from patients diagnosed with IgA nephropathy by renal biopsy and followed for at least six months were received from the First Medical Center of a certain hospital. A total of 488 subjects were included and randomly divided into a training set (n=341) and a test set (n=147). The prognostic endpoint was set as end-stage renal disease or a 50% decrease in eGFR. The collection of historical data was approved by the medical committee and the patients.

[0031] In step 102, if Figure 2 ,The methods for extracting pathological features include: Step 201: Preprocessing the IgA immunofluorescence image.

[0032] Preprocessing can include data augmentation and normalization. During image preparation, we applied Z-score normalization to the RGB channels to standardize the intensity distribution. During training, we used online data augmentation techniques, including random cropping and horizontal and vertical flipping, to increase data variability. During model testing, only normalization was applied to small patches to ensure consistency in data processing.

[0033] Step 202: Divide the pre-processed IgA immunofluorescence image into a plurality of patches.

[0034] In one specific test, the size of the small block was 512x512 pixels, and its large size could be processed at 20x magnification.

[0035] Step 203: Filter valid small blocks from the multiple small blocks.

[0036] For example, small blocks filled with mainly bright pixels are excluded, where the proportion of bright pixels in these blocks exceeds a first threshold and contains the least useful data. This can be achieved by eliminating all black background.

[0037] Step 204: normalize the valid small blocks to obtain the third small block.

[0038] Step 205: Create a label for the third piece Patch prob , and obtain the second training set.

[0039] Specifically, based on end-stage renal disease or a second threshold for eGFR decline (e.g., 50%), IgA immunofluorescence images and patients are divided into high-risk and low-risk groups, and the corresponding small blocks / third small blocks are labeled with the high-risk and low-risk group categories. However, the grouping method is not limited to this. X-tile software can also be used to determine the optimal survival cutoff point, divide patients into high-risk and low-risk groups, and label the grouping labels on the corresponding small blocks.

[0040] Step 206: Based on a machine learning method, the second training set is trained to obtain a recognition model.

[0041] Specifically, the recognition models used are ResNet18, ResNet50, Inception_v3, and DenseNet121, with the ResNet18 model being preferred. The performance of each model in the training set is shown in Table 1, and the performance in the test set is shown in Table 2. Among them, DenseNet121 is the most complex, while ResNet50 is a widely used convolutional neural network model.

[0042] To ensure robust performance across different groups, transfer learning was used to initialize model parameters using weights pre-trained on the ImageNet dataset, enabling the model to utilize common visual features. To promote better generalization, a cosine decay learning rate scheduling strategy was used, which is defined as: η t = η min + 0.5 × (η max - η min ) × (1+cos(π × T cur / T i )) in, η t is the current learning rate, η min represents the minimum learning rate, η max is the maximum learning rate, T cur The current iteration number, T i Corresponding to the total number of iteration cycles, such as 16.

[0043] The training process is optimized using stochastic gradient descent (SGD) and softmax cross entropy as the loss function to improve the efficiency and performance of the model.

[0044] Table 1

[0045] Table 2

[0046] Among them, Acc (Accuracy) refers to the proportion of correct predictions among all prediction results. AUC (Area Under the Curve) is usually used to evaluate the performance of a binary classification model. It is the area under the ROC curve (Receiver Operating Characteristic Curve). Sensitivity is also called the true positive rate, which refers to the proportion of people who are tested positive among the population that actually has the disease. Specificity is also called the true negative rate, which refers to the proportion of people who are tested negative among the population that actually does not have the disease. PPV (Positive Predictive Value) refers to the proportion of people who are actually sick among the population with a positive test result. NPV (Negative Predictive Value) refers to the proportion of people who are actually not sick among the population with a negative test result.

[0047] The ROC curve of the training set of the recognition model can be found in Figure 3 , the ROC curve of the test set can be found in Figure 4. In the ResNet18 model, the AUC values ​​in the training and test cohorts were 0.610 and 0.617, respectively. These AUC metrics represent the model's ability to effectively distinguish between categories, indicating that its performance is at a moderate level. The AUC for the training set fell within the 95% confidence interval (CI) of 0.5977-0.6213, while the AUC for the test set fell within the 95% CI of 0.5968-0.6372, indicating that the model's performance on different datasets has a certain degree of consistency, although its discriminative ability is limited.

[0048] The analysis results indicate that while the ResNet18 model performs acceptably, it is not the highest-performing model available. However, this choice may have been influenced by other factors, such as model simplicity, computational efficiency, or generalization performance during training and testing. In applications, efforts should focus on enhancing the model's robustness and discriminative capabilities, perhaps by exploring model tuning, augmentation techniques, or incorporating more discriminative features during training.

[0049] After the recognition model is trained, it can be extended to predict the label and probability of each patch. These probabilities are aggregated through histogram features and bag-of-words (BoW) features to facilitate prediction at the level of the entire immunofluorescence image.

[0050] Step 207: Based on the recognition model, predict the probability of the third small block Patch pred , according to the prediction threshold and probability, obtain the label of the third small block and the third feature.

[0051] If the probability of the third small block being a high-risk group is predicted, when the probability exceeds the third threshold, the third small block is determined to be a high-risk group. The third feature of the third small block includes: probability Patch pred and tags Patch prob .

[0052] Step 208: Combined with histogram statistics, the third small block is binned to obtain buckets; and the fourth feature is obtained based on the probability and label frequency of the third small block in the bucket.

[0053] More specifically, the probability distribution of the third small block is obtained by combining histogram statistics (e.g., number of bins = 50, step size = 0.02, but not limited to this). The third small block is binned to obtain buckets. The probability of the third small block and the frequency of the label in the bucket are calculated and normalized to construct the fourth feature of the bucket: Histo Prob and Histo Pred .

[0054] The Min-Max normalization method can be used to normalize the probability and frequency of the label in the bucket.

[0055] Step 209: Based on the Bag-of-Words model (BOW), the third feature is vectorized and the fifth feature is obtained based on the TF-IDF transformation: Bow prob and Bow pred .

[0056] Step 210: Fuse the fourth feature and the fifth feature to obtain the sixth feature vector features fusion and feature sets, i.e., pathological features.

[0057] The feature set includes the sixth feature vectors of multiple small blocks, the sixth feature vector features fusion Expressed as: ; in, Feature connector.

[0058] Step 211: Screen the sixth feature vector of the feature set to obtain pathological features / pathological features pathomics .

[0059] First, a correlation-based filtering method is used to retain features with a Pearson correlation coefficient exceeding a fourth threshold (e.g., 0.9). Then, a further selection is performed using univariate Cox regression to retain only features with a p-value below a fifth threshold (e.g., 0.05). Non-zero features can also be selected.

[0060] In one specific implementation, after normalizing and partitioning the IgA immunofluorescence image (512×512 pixels / block), a pre-trained ResNet18 model was used to extract the third eigenvector (containing morphological parameters and spatial distribution characteristics) of each image block. This was accomplished using a weakly supervised learning strategy to generate predicted labels and corresponding probabilities for each individual block / third block. This led to the generation of two core feature sets: 101 probabilistic features: based on the high-risk / low-risk probability distribution of the image block, the proportion of blocks at each risk level was calculated using histogram binning; and two predicted label features.

[0061] Step 212: The training set is trained based on machine learning methods and pathological features to obtain a pathological model. For example, the pathological model is trained based on a Cox proportional hazards model (also known as a multivariate Cox model).

[0062] like Figure 5 and Figure 6The Kaplan-Meier (KM) curve for the survival analysis of the pathological model showed that the C-index (Concordance Index) for the training and test sets was 0.757 and 0.755, respectively. The C-index, or concordance index, is used to evaluate the predictive ability of the model.

[0063] Visualization can be achieved based on Grad-CAM, using Gradient Weighted Class Activation Mapping (Grad-CAM) to visualize the recognition ability of deep learning models on various samples. Figure 7 The Grad-CAM activations of the final convolutional layer are shown, key regions influencing prognostic prediction are identified, and insights into model interpretability are provided.

[0064] In step 103, clinical characteristics can be divided into continuous variables, categorical variables, and Oxford pathology classification variables. Univariate analysis and multivariate analysis can be used to analyze clinical characteristics, such as using Cox regression to screen clinical characteristics (P < 0.05).

[0065] As shown in Table 3, baseline comparisons of the training and test sets showed that there were no significant differences in the continuous variables: age, proteinuria, mean arterial pressure (MAP), and baseline estimated glomerular filtration rate (eGFRbaseline) between the training and test sets (p>0.05). Specifically, the mean age was 35.16±11.01 years, 35.16±11.19 in the training and test sets, and 35.18±10.63 in the test sets (p=0.645). The mean proteinuria level was 1.61±1.53 g / d, and the distribution was consistent between the two groups (p=0.176).

[0066] Categorical variables: RAS inhibitor use (86.27% vs 87.76%, p=0.63) and immunosuppression use (56.30% vs 51.02%, p=0.329) were balanced between the two groups. Continuous variables were normalized using the Shapiro-Wilk test and analyzed using the t-test and Mann-Whitney U test, respectively. Categorical variables were analyzed using the chi-squared test. In one specific embodiment, the analysis was performed using Python 3.7.12 and scikit-learn 1.0.2, and the model was trained using an NVIDIA 4090 GPU and PyTorch 1.8.1.

[0067] There was no statistical difference in the distribution of each component (M / E / S / T) of the Oxford pathological classification (MEST-C score): segmental sclerosis (S1) accounted for 72.13% of the whole group, 73.31% of the training set, and 69.39% of the test set (p=0.437), and tubular atrophy (T1-2) accounted for 73.18% of the whole group, with no significant shift between groups (p=0.143).

[0068] Table 3

[0069] In Table 3, continuous variables are presented as mean ± standard deviation; categorical variables are presented as frequency (percentage). P values ​​reflect the difference between the training and test sets; *p < 0.05 was considered statistically significant. RAS: renin-angiotensin system inhibitor; MEST-C: Oxford classification (mesangial proliferation (M), endothelial proliferation (E), segmental sclerosis (S), tubular atrophy (T), and cellular crescents (C).

[0070] The method for screening features in step 103 can be a method based on COX univariate regression analysis or a method based on Pearson correlation coefficient. Figure 8 Oxford classification of M, S, and T lesions, immunosuppressant use, urine protein, and mean arterial pressure were associated with the prognosis of IgA nephropathy. X-tile was used to determine the optimal risk stratification threshold, dividing the patients into low-risk and high-risk groups.

[0071] Based on the machine learning method, the first model is trained according to the clinical characteristics. Figure 9 and Figure 10 The survival analysis KM (Kaplan-Meier) curve showed that the C-index (Concordance Index, C-Index) of the training set and test set were 0.855 and 0.860, respectively.

[0072] The prognostic model in step 105 is constructed based on the integration of immunofluorescence omics features and clinical features, such as Figure 11 and Figure 12 The survival analysis KM curves showed that the model's C-index for the training and test sets was 0.882 and 0.888, respectively. This performance outperformed the first model based solely on clinical features. X-tile was used to determine the optimal risk stratification threshold, dividing patients into low-risk and high-risk groups.

[0073] Time-dependent survival analysis: As shown in Table 4, in the 5-year survival prediction analysis, the AUC values ​​of different models showed significant differences in predictive efficacy. The first model, also known as the clinical model, achieved AUC values ​​of 0.884 and 0.927 in the training and test sets, respectively, demonstrating robust predictive ability. The prognostic model, also known as the combined model, significantly improved its AUC value to 0.931 (training set) and 0.940 (test set) by integrating clinical and pathological features, validating the synergistic advantage of multimodal data (p < 0.001).

[0074] In the 7-year survival prediction, the AUC values ​​of each model were generally lower than those in the 5-year analysis, but still maintained a consistent pattern: the AUC of the first model training set was 0.908, and the test set slightly dropped to 0.871; the AUC of the prognostic model training set and test set were 0.917 and 0.901, respectively, continuing to lead, further confirming the gain effect of cross-modal integration on long-term prognosis prediction (ΔAUC = +4.7% vs clinical model).

[0075] Table 4

[0076] The present invention also constructs a nomogram model of 3-year and 5-year survival risk based on pathological characteristics and clinical characteristics, such as Figure 13 , the corresponding survival risk can be found based on the corresponding characteristics. A nomogram is a graphical tool used to predict or evaluate the probability or value of an event. It integrates multiple related variables for comprehensive analysis.

[0077] The second aspect of the present invention provides a system for implementing the above method, such as Figure 14 , including acquisition module 1, pathological feature recognition module 2, clinical feature extraction module 3 and prognosis module 5, The acquisition module 1 is used to collect historical data; Pathological feature recognition module 2 is used to extract pathological features from IgA immunofluorescence images based on a visual recognition method; The clinical feature extraction module 3 is used to extract clinical features based on the clinical data; The prognostic module 5 is used to predict the clinical data and the IgA immunofluorescence image based on the prognostic model to obtain the prognosis.

[0078] The system also provides a model training module 4 for obtaining a prognosis model after training a training set based on pathological characteristics and clinical characteristics based on a machine learning method.

[0079] A third aspect of the present invention provides a device comprising a processor and a memory, wherein the memory stores codes, and when the codes are processed by the processor, the above-mentioned prognosis method is implemented.

[0080] The present invention uses a computer vision-based method to extract pathological features from IgA immunofluorescence images, which can automatically extract pathological features and avoid errors and omissions caused by manual / visual identification. It also predicts the prognosis of IgA nephropathy in combination with clinical characteristics, which is conducive to automated prediction, has robust predictive capabilities, and realizes multimodal data fusion. Multimodal data fusion has a gain effect on long-term prognosis prediction.

[0081] A multimodal survival prediction model was constructed through feature fusion, and ultimately a Cox proportional hazards model was established to complete survival analysis. Experiments demonstrated that this method effectively captured underlying pathological patterns. The multimodal model exhibited a C-index of 0.888 in the test set, providing reliable prognostic estimates for medical professionals or patients and serving as a reliable tool for personalized medical assessment. It should be noted that the prognostic method of this invention is intended to provide advice to medical professionals or patients and does not constitute a method for treating or diagnosing a disease.

[0082] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A prognostic method for IgA nephropathy based on multimodal data, characterized in that: The following steps are involved: Collecting historical data and randomly dividing them into a training set and a test set, wherein the historical data includes clinical data and IgA immunofluorescence images; Pathological features were extracted from IgA immunofluorescence images based on visual recognition methods; Based on the clinical data, clinical characteristics are screened; Based on a machine learning method, a training set is trained according to pathological characteristics and clinical characteristics to obtain a prognostic model, which is used for the prognosis of IgA nephropathy.

2. The IgA nephropathy prognosis method according to claim 1, characterized in that: The method for extracting pathological features includes the following steps: The IgA immunofluorescence image was divided into several small blocks; Establishing labels for the small blocks to obtain a second training set; Based on the machine learning method, the second training set is trained to obtain a recognition model; Based on the recognition model, the probability of small blocks and their labels are predicted to construct the third feature; Combined with histogram statistics, the third small block is binned to obtain buckets; Construct the fourth feature based on the probability of the small blocks in the bucket and the frequency of the label; The third feature is vectorized based on the bag-of-words model, and the fifth feature is obtained based on the TF-IDF transformation; The fourth feature and the fifth feature are fused to obtain a pathological feature.

3. The IgA nephropathy prognosis method according to claim 2, characterized in that: Specific methods for extracting pathological features include: Preprocessing of IgA immunofluorescence images; The pretreated IgA immunofluorescence image was divided into several small blocks; screening valid small blocks from the plurality of small blocks; Normalize the valid small blocks to obtain the third small block; Create labels for the third small piece and obtain the second training set; Based on the machine learning method, the second training set is trained to obtain a recognition model; Based on the recognition model, predict the probability of the third small block; According to the prediction threshold and probability, the label of the third small block is obtained, and the third feature is constructed according to the probability and label; Combined with histogram statistics, the probability distribution of the third small block is obtained; Bin the third small piece to obtain buckets; After calculating the probability of the third small block and the frequency of the label in the bucket, normalization is performed to construct the fourth feature of the bucket; After vectorizing the third feature based on the bag-of-words model, the fifth feature is obtained based on the TF-IDF transformation. Fusing the fourth feature and the fifth feature to obtain a sixth feature vector and a feature set; The sixth eigenvector of the feature set is screened to obtain pathological features.

4. The IgA nephropathy prognosis method according to claim 2 or 3, characterized in that: The pathological feature or the sixth eigenvector is expressed as: ； in, is the feature connector, features fusion Denoted as the sixth eigenvector or pathological feature, Histo Prob and Histo Pred is the parameter of the fourth characteristic, Bow prob and Bow pred is the parameter of the fifth feature.

5. The IgA nephropathy prognosis method according to claim 3, characterized in that: Methods for screening valid small blocks from multiple small blocks include: Determining whether the proportion of bright pixels in the small block exceeds a first threshold; If so, exclude the small piece; If not, retain the small piece.

6. The IgA nephropathy prognosis method according to claim 3, characterized in that: Machine learning methods for training recognition models include any of the following models: ResNet18, ResNet50, Inception_v3 and DenseNet121; Prognostic models included: Cox proportional hazards model or nomogram model; Based on the Min-Max normalization method, the probability and frequency of the label in the bucket are normalized.

7. The IgA nephropathy prognosis method according to claim 3, characterized in that: Methods for selecting the sixth eigenvector include: A correlation-based filtering method retains the sixth eigenvector whose Pearson correlation coefficient exceeds the fourth threshold; Using univariate Cox regression, the sixth eigenvector with a p-value below the fifth threshold was retained; Select non-zero features to obtain pathological features.

8. The IgA nephropathy prognosis method according to claim 1, characterized in that: Clinical characteristics were screened using a combination of univariate and multivariate analyses and Pearson's method; Clinical characteristics were selected from the following categories: age, proteinuria, mean arterial pressure, baseline estimated glomerular filtration rate, RAS inhibitor use, immunosuppressant use, mesangial proliferation, endothelial proliferation, segmental sclerosis, tubular atrophy, and cellular crescents.

9. A system, characterized in that: Used to implement the IgA nephropathy prognosis method according to any one of claims 1 to 8, comprising an acquisition module, a pathological feature recognition module, a clinical feature extraction module and a prognosis module, The acquisition module is used to collect historical data; The pathological feature recognition module is used to extract pathological features from IgA immunofluorescence images based on visual recognition methods; The clinical feature extraction module is used to extract clinical features based on the clinical data; The prognostic module is used to predict clinical data and their IgA immunofluorescence images based on the prognostic model to obtain the prognosis.

10. A device, characterized in that: including processor and memory, The memory stores codes, and when the codes are processed by the processor, the IgA nephropathy prognosis method according to any one of claims 1 to 8 is implemented.

Images