Method for constructing interpretable machine learning model for risk prediction of immunotherapy-related liver injury
By constructing an interpretable model based on machine learning, the problems of insufficient accuracy and interpretability in liver injury prediction in existing technologies are solved, achieving efficient and robust liver injury risk assessment and improving the safety and individualized management of immunotherapy.
Patent Information
- Application Number
- CN202510939550.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-08
- Publication Date
- 2025-10-31
AI Technical Summary
Existing predictive models lack accuracy and interpretability in predicting liver injury associated with immune checkpoint inhibitor therapy, and traditional statistical methods are significantly inadequate when dealing with complex, nonlinear, and high-dimensional data, making it difficult to meet clinical needs.
An interpretable model based on machine learning was constructed. Through data collection, preprocessing, variable selection, and model optimization, the random forest algorithm was used in conjunction with the SHAP method for feature importance analysis to build a multi-dimensional data prediction model for assessing the risk of liver injury in patients.
It improves the accuracy of liver injury risk prediction and the interpretability of the model, provides personalized risk assessment tools, enhances the safety and efficacy of immunotherapy, and reduces the incidence of liver injury.
Smart Images
Figure CN120878142A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an interpretable machine learning model construction method for predicting the risk of liver injury related to immunotherapy. The constructed model is used to predict the risk of liver injury in cancer patients who are receiving immune checkpoint inhibitor therapy for the first time, and belongs to the field of biomedical technology. Background Technology
[0002] The advent of immune checkpoint inhibitors (ICIs) marked a milestone in cancer immunotherapy, significantly altering the treatment landscape for various malignancies. With the widespread application of ICIs in multiple cancer types, the incidence of immune-related adverse events (irAEs) has gradually attracted clinical attention. Among these, ICI-related liver injury is a significant clinical challenge, with its incidence varying from 1% to 25% depending on the treatment regimen and patient population. This type of hepatotoxicity manifests in diverse ways, progressing from mild elevations in liver enzymes to severe hepatitis, and in rare cases, even life-threatening liver failure. The high heterogeneity of liver injury underscores the importance of early identification and timely intervention to ensure treatment safety and minimize the risk of immunotherapy interruption. Therefore, conducting research to achieve early identification of ICI-related liver injury and to halt its progression to severe toxicity is crucial for improving patient prognosis and ensuring the successful implementation of immunotherapy.
[0003] Current predictive models primarily focus on assessing overall immune-related adverse events, immunotherapy response, or survival prognosis, with relatively limited research on ICI-related liver injury. Zheng et al. constructed a logistic regression-based model to predict sintilimab-induced immune-mediated liver injury, achieving a C-index of 0.71 but an AUC of only 0.56, indicating limited predictive performance. Similarly, Jiang et al. developed a Cox regression model for MELD (Medium-terminal Liver Disease) but did not validate the proportional hazards assumption. In contrast, Yamamoto et al. attempted to cluster and stratify the risk of immune-related liver injury using unsupervised machine learning methods, but did not further develop a predictive model. Notably, none of the aforementioned studies used ICI-related liver injury as a specific predictive endpoint, which could have significant clinical warning value.
[0004] Furthermore, previous studies have largely employed traditional statistical methods, resulting in predictive models with performance limitations, and some models have failed to validate the proportional hazards assumption. While traditional methods offer simplicity and interpretability, they are significantly inadequate for handling complex, nonlinear, and high-dimensional data. In contrast, machine learning (ML) algorithms demonstrate powerful modeling capabilities when processing large and complex datasets, and are therefore considered an effective tool for solving such predictive challenges. Summary of the Invention
[0005] The objective of this invention is to develop and validate a machine learning-based predictive model for assessing the risk of liver injury in patients receiving ICI treatment for the first time.
[0006] To address the aforementioned technical problems, the present invention discloses a method for constructing an interpretable machine learning model for predicting the risk of liver injury related to immunotherapy, characterized by comprising the following steps:
[0007] Step 1, Data Collection: Collect clinical data from cancer patients receiving immune checkpoint inhibitor therapy;
[0008] Step 2, Data Preprocessing: Perform preprocessing operations on the raw data collected in Step 1 to ensure data quality and the feasibility of subsequent modeling;
[0009] Step 3: Dataset partitioning: Divide the complete dataset obtained in Step 2 into a training set and a test set for model training and performance validation.
[0010] Step 4: Pre-model construction: On the training set, all candidate variables are included, and N machine learning algorithms are used to build preliminary prediction models. Hyperparameters are optimized based on the grid search method of five-fold cross-validation. The optimal prediction model with AUC value as the evaluation index is selected, where N≥1.
[0011] Step 5, Variable Selection: Based on the N machine learning pre-models obtained in Step 4, the importance of each variable is evaluated using the SHAP method, and the intersection of the top 20 important variables in all models is selected to finally determine the core variables.
[0012] Step 6: Final Model Construction: Based on the selected core variables, the final prediction model is reconstructed on the basis of the best-performing prediction model architecture, and the model parameters are further optimized by combining five-fold cross-validation with grid search.
[0013] Step 7, Model Validation: Evaluate the generalization ability of the final model on the test set and test its stability and accuracy on untrained data;
[0014] Step 8: Evaluate model performance;
[0015] Step 9, Model Interpretation: The SHAP method is used to perform interpretability analysis on the final model to enhance its clinical application value.
[0016] Preferably, in step 1, the clinical data covers multiple time points before and during treatment.
[0017] Preferably, in step 1, the clinical data includes clinical data, laboratory tests, medication information, and past medical history.
[0018] Preferably, in step 2, the preprocessing operation includes missing value imputation and data standardization.
[0019] Preferably, in step 3, the complete dataset is divided into the training set and the test set according to a ratio of 80% to 20%.
[0020] Preferably, in step 4, the machine learning algorithm includes random forest, extreme gradient boosting, gradient boosting decision tree, and neural network.
[0021] Preferably, in step 8, the AUC value, accuracy, sensitivity, specificity, positive predictive value, and negative predictive value are calculated by plotting ROC curves, and the model performance is comprehensively evaluated by combining indicators including calibration curves and decision curves.
[0022] Preferably, in step 9, the interpretability analysis includes a visual interpretation of feature importance ranking and individual risk contribution.
[0023] This invention integrates multi-dimensional information such as patient baseline characteristics, liver function test indicators, and treatment plans, and applies various machine learning algorithms to construct a robust predictive model. This predictive model provides a personalized risk assessment tool for clinical practice, enabling the early identification of high-risk patients in clinical settings, guiding treatment decisions and monitoring protocols, thereby reducing the incidence of ICI-related liver injury, minimizing adverse events, improving the safety and efficacy of immunotherapy, and enhancing treatment safety and individualized management.
[0024] Compared with existing technical solutions, the present invention has the following beneficial effects:
[0025] 1) High prediction accuracy: The constructed machine learning model outperforms traditional statistical methods in all performance metrics and has stronger predictive capabilities;
[0026] 2) Good generalization ability: It performs stably on independent test sets and has good potential for widespread application;
[0027] 3) High model interpretability: The contribution of each variable to the prediction results is quantified by the SHAP method, which improves the transparency of the model and makes it easier for clinical understanding and trust.
[0028] 4) High clinical application value: The variables used in the model are derived from routine clinical examinations, which are easy to obtain, and the output results are intuitive and clear, making it convenient for doctors to make auxiliary decisions;
[0029] 5) High scalability: The model architecture is flexible, making it easy to introduce more feature variables, and it can also be extended to the risk prediction of other immune-related adverse events. Attached Figure Description
[0030] Figure 1 A flowchart was designed to illustrate the entire process of data collection, preprocessing, model building, parameter optimization, model validation, and performance evaluation.
[0031] Figure 2 The pre-model performance and variable ranking are illustrated, where:
[0032] A illustrates the ROC curves of four machine learning pre-models;
[0033] B illustrates the trend of AUC value changes under different numbers of features;
[0034] C illustrates the AUC, accuracy, sensitivity, and specificity performance of the random forest model under different numbers of features;
[0035] D is a honeycomb diagram of the SHAP feature importance of the top 20 features in the four pre-models;
[0036] Figure 3 The final random forest model's predictive performance evaluation is illustrated, where:
[0037] A illustrates the ROC curve of the model on the test set;
[0038] B illustrates the performance of the model based on five-fold cross-validation;
[0039] C illustrates the performance of the model based on 10-fold cross-validation;
[0040] Figure 4 The analysis of the decision curve and calibration curve of the final random forest model is illustrated, where:
[0041] A illustrates the decision curve analysis on the training set;
[0042] B illustrates the decision curve analysis on the test set;
[0043] C illustrates the calibration curve on the training set;
[0044] D represents the calibration curve on the test set;
[0045] Figure 5The diagram illustrates the global interpretability analysis of the final random forest model based on the SHAP method, where:
[0046] A is a SHAP bar chart representing feature importance;
[0047] B is a SHAP honeycomb diagram representing the importance of features;
[0048] C is the SHAP dependency graph, which shows the relationship between key variables and model output. Detailed Implementation
[0049] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.
[0050] This invention discloses a method for constructing an interpretable machine learning model for predicting the risk of liver injury related to immunotherapy. The aim is to build a predictive model based on an interpretable machine learning algorithm, integrating multi-dimensional data such as patient clinical data and biochemical test results to accurately predict the risk of liver injury after ICI treatment. It provides explanations of model prediction results and feature importance, facilitating understanding and application by clinicians. The method specifically includes the following steps:
[0051] Step 1, Data Collection: This invention is based on clinical data from 863 cancer patients treated with immune checkpoint inhibitors (ICIs) at a tertiary hospital, such as... Figure 1 As shown. The collected data includes variables such as liver function indicators, blood lipid levels, underlying diseases, and medication regimens before and during treatment. The raw data is cleaned and preliminarily analyzed to remove non-standard or erroneous records.
[0052] Step 2, Data Preprocessing: Missing rates were assessed for all variables in the dataset, and variables with missing rates exceeding 25% were removed. Finally, 31 candidate variables were included for model construction. Missing data were imputed using LightGBM-based Multiple Imputation by Chained Equations. The baseline characteristics of the 31 candidate variables and the research subjects are detailed in Table 1 below.
[0053] Table 1 Baseline characteristics of the study population
[0054]
[0055] Step 3: Dataset Splitting: Use stratified random sampling to divide the dataset into a training set and a test set in an 8:2 ratio, ensuring that the proportion of events occurring in both groups is consistent. The training set is used for model training and variable selection, while the test set is used for final performance validation.
[0056] Step 4: Predictive Model Construction: In the training set, all 31 candidate variables were included, and four mainstream machine learning algorithms were used to construct predictive models: Neural Network (NN), Gradient Boosting Classifier (GBC), Extreme Gradient Boosting (XGBoost), and Random Forest (RF). All models underwent hyperparameter tuning using five-fold cross-validation combined with grid search. The performance evaluation metric was AUC. Results showed that the Random Forest model performed best, with an AUC of 0.83 (95% CI: 0.76–0.91). Subsequently, the SHAP method was used to evaluate the marginal contribution of each variable to the model output and ranked them by feature importance. Features were then incorporated into the model in order of importance. The AUC value of the RF model showed a stable trend, indicating robust predictive performance. This model was ultimately selected as the basis for constructing the final model. Figure 2 As shown.
[0057] Step 5: Variable Screening: Based on the SHAP feature importance analysis of the four machine learning pre-models mentioned above, the top 20 key variables in each model were extracted. The intersection of these variables was then used to identify 11 core variables: gamma-glutamyl transferase (GGT), aspartate aminotransferase (AST), lymphocyte count (Lym), alkaline phosphatase (AKP), body mass index (BMI), whether it is primary liver cancer, whether chemotherapy drugs are being used, whether targeted therapy drugs are being used, hypertension, diabetes, and number of lymph node metastases.
[0058] Step 6: Final Model Construction and Validation: Based on the 11 selected core variables, the final prediction model was constructed on the aforementioned optimal RF model architecture. Model parameters were determined using a five-fold cross-validation grid search. The final model's AUC on the training set was 0.91 (95% CI: 0.89–0.94), and its AUC on the test set was 0.81 (95% CI: 0.73–0.89), as shown in Table 2 below. The model's accuracy, sensitivity, and specificity on the test set were also evaluated. To further validate model stability, five-fold and ten-fold cross-validation was performed on the full sample, with AUCs of 0.79 and 0.80, respectively. Figure 3As shown in the figure. Decision Curve Analysis (DCA) results show that the final model outperforms both the "all treatment" and "no treatment" strategies in terms of net benefit on both the training and test sets, indicating its good clinical applicability. Figure 4 As shown in AB, calibration curve analysis further confirms that the model's predicted probabilities match the actual occurrence rates well in both the training and test sets, demonstrating high calibration performance. Figure 4 As shown in CD.
[0059] Table 2 Predictive performance of machine learning models
[0060] AUC Accuracy Sensitivity Specificity PPV NPV Pre-models Training set NN 0.75(0.70,0.79) 0.71(0.68，0.75) 0.67(0.59，0.74) 0.73(0.69，0.76) 0.42(0.36,0.48) 0.88(0.85,0.91) GBC 0.88(0.86，0.91) 0.79(0.75，0.82) 0.86(0.80，0.91) 0.77(0.73，0.80) 0.52(0.46,0.58) 0.95(0.92，0.97) XGBoost 1.00(1.00，1.00) 1.00(0.99，1.00) 1.00(0.98,1.00) 1.00(0.99,1.00) 1.00(0.98,1.00) 1.00(0.99,1.00) RF 0.85(0.82，0.89) 0.69(0.65，0.72) 0.90(0.84,0.94) 0.62(0.58，0.66) 0.41(0.36,0.47) 0.95(0.93，0.97) Test set NN 0.78(0.69,0.86) 0.79(0.72，0.84) 0.67(0.50，0.81) 0.82(0.75，0.88) 0.52(0.37,0.66) 0.89(0.83，0.94) GBC 0.79(0.71,0.87) 0.83(0.76，0.88) 0.62(0.45，0.77) 0.89(0.82,0.94) 0.62(0.45,0.77) 0.89(0.82，0.94) XGBoost 0.79(0.72，0.87) 0.75(0.67，0.81) 0.82(0.66，0.92) 0.72(0.64，0.80) 0.46(0.34,0.59) 0.93(0.87,0.97) RF 0.83(0.76,0.91) 0.81(0.74，0.86) 0.79(0.64，0.91) 0.81(0.74，0.88) 0.55(0.41，0.69) 0.93(0.87，0.97) Final model Training set 0.91(0.89,0.94) 0.82(0.79，0.85) 0.90(0.84，0.94) 0.80(0.76,0.83) 0.56(0.50，0.63) 0.96(0.94，0.98) Test set 0.81(0.73，0.89) 0.80(0.73，0.85) 0.74(0.58,0.87) 0.81(0.74，0.88) 0.54(0.40,0.67) 0.92(0.85,0.96)
[0061] Step 7, Model Interpretation: To improve model transparency and interpretability, the SHAP method is used for global interpretation of the final model. The feature importance bar chart shows that GGT is the most important variable, followed by AST and primary liver cancer, as shown below. Figure 5 As shown in A in the diagram. Honeycomb diagram (as shown in the diagram). Figure 5 Figure B shows the specific contribution distribution of each variable to the prediction results. Further analysis using the SHAP dependency plot revealed that when GGT ≥ 45 U / L or AST ≥ 40 U / L, the SHAP value is mostly greater than 0, and the prediction result tends towards the "liver injury" category; when Lym (0.7-2.0×10) 9 When BMI (45-135 U / L) and AKP (45-135 U / L) are within the normal range, the prediction is more inclined towards the "non-hepatic injury" category; BMI ≥ 29 kg / m² 2 At that time, the model predicted a significantly increased risk of liver injury (e.g. Figure 5 (as shown in C).
[0062] This invention models large-scale, multi-dimensional clinical data using machine learning algorithms, exhibiting strong nonlinear fitting capabilities and the ability to identify complex relationships between variables. Simultaneously, it introduces the SHAP (SHapley Additive explanations) method to enhance model interpretability, effectively addressing the "opaqueness" and "lack of interpretability" issues inherent in traditional "black box" models in clinical applications, thereby improving the model's acceptability and practical value in a clinical setting. The methodological design of this invention aligns with actual clinical needs, demonstrating scientific rigor, rationality, and broad application prospects.
Claims
1. A method for constructing an interpretable machine learning model for predicting the risk of immunotherapy-related liver injury, characterized in that, Includes the following steps: Step 1, Data Collection: Collect clinical data from cancer patients receiving immune checkpoint inhibitor therapy; Step 2, Data Preprocessing: Perform preprocessing operations on the raw data collected in Step 1 to ensure data quality and the feasibility of subsequent modeling; Step 3: Dataset partitioning: Divide the complete dataset obtained in Step 2 into a training set and a test set for model training and performance validation. Step 4: Pre-model construction: On the training set, all candidate variables are included, and N machine learning algorithms are used to build preliminary prediction models. Hyperparameters are optimized based on the grid search method of five-fold cross-validation. The optimal prediction model with AUC value as the evaluation index is selected, where N≥1. Step 5, Variable Selection: Based on the N machine learning pre-models obtained in Step 4, the importance of each variable is evaluated using the SHAP method, and the intersection of the top 20 important variables in all models is selected to finally determine the core variables. Step 6: Final Model Construction: Based on the selected core variables, the final prediction model is reconstructed on the basis of the best-performing prediction model architecture, and the model parameters are further optimized by combining five-fold cross-validation with grid search. Step 7, Model Validation: Evaluate the generalization ability of the final model on the test set and test its stability and accuracy on untrained data; Step 8: Evaluate model performance; Step 9, Model Interpretation: The SHAP method is used to perform interpretability analysis on the final model to enhance its clinical application value.
2. The method for constructing an interpretable machine learning model for predicting the risk of immunotherapy-related liver injury as described in claim 1, characterized in that, In step 1, the clinical data covers multiple time points before and during treatment.
3. The method for constructing an interpretable machine learning model for predicting the risk of immunotherapy-related liver injury as described in claim 1, characterized in that, In step 1, the clinical data includes clinical data, laboratory tests, medication information, and past medical history.
4. The method for constructing an interpretable machine learning model for predicting the risk of immunotherapy-related liver injury as described in claim 1, characterized in that, In step 2, the preprocessing operations include missing value imputation and data standardization.
5. The method for constructing an interpretable machine learning model for predicting the risk of immunotherapy-related liver injury as described in claim 1, characterized in that, In step 3, the complete dataset is divided into the training set and the test set according to a ratio of 80% and 20%.
6. The method for constructing an interpretable machine learning model for predicting the risk of immunotherapy-related liver injury as described in claim 1, characterized in that, In step 4, the machine learning algorithms include random forest, extreme gradient boosting, gradient boosting decision tree, and neural network.
7. The method for constructing an interpretable machine learning model for predicting the risk of immunotherapy-related liver injury as described in claim 1, characterized in that, In step 8, the AUC value, accuracy, sensitivity, specificity, positive predictive value, and negative predictive value are calculated by plotting ROC curves, and the model performance is comprehensively evaluated by combining indicators including calibration curves and decision curves.
8. The method for constructing an interpretable machine learning model for predicting the risk of immunotherapy-related liver injury as described in claim 1, characterized in that, In step 9, the interpretability analysis includes a visual interpretation of feature importance ranking and individual risk contribution.