Thyroid nodule auxiliary diagnosis model and system based on interpretable machine learning

CN118942673APending Publication Date: 2024-11-12CHONGQING MEDICAL UNIVERSITY
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410975034.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Traditional thyroid nodules diagnostic methods are at risk of over-piercing or missed diagnosis, and the lack of explanatory nature of traditional machine learning models leads to lack of confidence and transparency in clinical physicians' diagnosis results.

Method used

Using an assisted diagnostic model based on interpretability machine learning, an automated feature engineering framework is built by acquiring and preprocessing patient data, selecting the optimal feature combination, building and adjusting machine learning models, and using interpretability methods such as SHAP and LIME Model explanation.

Benefits of technology

It improves the accuracy and transparency of the diagnosis, enhances doctors and patients' understanding and trust in the diagnosis results, provides the credibility and interpretability of model decisions, and helps clinicians make more reliable judgments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004954288810000041
    Figure BDA0004954288810000041
  • Figure HDA0004954288830000011
    Figure HDA0004954288830000011
  • Figure HDA0004954288830000012
    Figure HDA0004954288830000012
Patent Text Reader

Abstract

The invention relates to a thyroid nodule auxiliary diagnosis model and system based on interpretable machine learning, and a construction method comprises the steps: collecting clinical data and related inspection data of a thyroid nodule patient, and constructing a thyroid nodule patient data set; the method comprises the steps of preprocessing data through a preset learning framework, then carrying out feature decreasing elimination through feature engineering so as to establish a prediction model, carrying out model performance evaluation through a cross validation method, selecting an optimal prediction performance model, and then further optimizing the model by adjusting hyper-parameters so as to obtain the optimal prediction performance. And finding out an optimal model through cross validation and iteration methods, and finally explaining the model through an interpretability method. The interpretable machine learning model constructed by the invention can integrate a large amount of clinical data so as to reliably predict benign and malignant risks of the thyroid nodule patient, thereby performing individualized intelligent diagnosis and treatment on the thyroid nodule patient.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a thyroid nodule auxiliary diagnosis model and system based on explainable machine learning, belonging to the technical field of intelligent medical diagnosis. Background Art

[0002] Thyroid nodules are a common condition in the endocrine system, manifested as abnormal proliferation of thyroid tissue. These nodules may be benign lesions or malignant tumors, which have a significant impact on the patient's health and quality of life. Traditional diagnostic methods include physical examination, ultrasound imaging, fine needle aspiration biopsy (FNA) and blood tests. Among them, ultrasound examination and fine needle aspiration biopsy are important means to distinguish between benign and malignant nodules. However, with the sharp increase in the number of patients with thyroid nodules and the limitations of evaluation methods, a large number of patients are at risk of over-puncture or missed diagnosis.

[0003] With the continuous advancement of computer science and medical technology, machine learning technology is increasingly used in the diagnosis of thyroid nodules. By analyzing large amounts of patient data, machine learning can extract complex patterns and features, thereby improving the accuracy and efficiency of diagnosis. However, traditional machine learning models are often viewed as "black boxes" whose internal mechanisms are complex and difficult to explain, which makes clinicians lack confidence and transparency when using these models.

[0004] In recent years, Explainable AI (XAI) has gradually attracted widespread attention as an emerging field. The goal of Explainable AI is to develop models that can provide high-performance predictions while allowing humans to understand and trust their decision-making process. This is particularly important in the medical field, because doctors need to understand the decision-making basis of the model in order to make more reliable and responsible judgments in clinical practice.

[0005] In the auxiliary diagnosis of thyroid nodules, the use of interpretable machine learning models can not only improve the accuracy of diagnosis, but also provide the reasons and basis behind each decision. For example, through interpretable methods, doctors can understand which features (such as nodule size or nodule grade in ultrasound images) contribute most to the model's prediction, thereby enhancing their understanding and trust in the model's decisions. Although some studies have begun to apply machine learning to the diagnosis of thyroid nodules, there are still few studies that are both interpretable and highly accurate. Therefore, the development of a thyroid nodule auxiliary diagnosis system based on interpretable machine learning can not only improve diagnostic efficiency, but also enhance doctors and patients' understanding and trust in the diagnostic results, which has important clinical significance and application prospects. Summary of the invention

[0006] The purpose of the present invention is to provide a thyroid nodule auxiliary diagnosis model and system based on explainable machine learning to solve the problems in the above-mentioned background technology.

[0007] In order to solve the above technical problems, the present invention provides a thyroid nodule auxiliary diagnosis model based on explainable machine learning, and the auxiliary diagnosis model comprises the following steps:

[0008] S1) obtaining original data related to patients with thyroid nodules, including patient gender, age, thyroid nodule ultrasound results, and related medical examination data and pathological data, and preprocessing the obtained data;

[0009] S2) Build an automated feature engineering framework to select the optimal feature combination through feature reduction and elimination;

[0010] S3) constructing a machine learning model and inputting the processed features to evaluate the performance indicators of the constructed models;

[0011] S4) selecting the machine learning model with the best performance according to the performance indicators, and then further adjusting the hyperparameters of the model, and performing cross-validation and iterative training to obtain the best machine learning model;

[0012] S5) Apply interpretability methods to explain the best machine learning model obtained, and use the best machine learning model for auxiliary diagnosis.

[0013] The data of the interpretable machine learning model of thyroid nodules constructed by feature engineering in the present invention are all from clinical sources, which are easy to obtain, authentic and reliable. The machine learning model obtained through cross-validation and continuous training is more accurate and reliable than the traditional model. Finally, the working mechanism of the model is fully explained through interpretability technology, which deepens the understanding and trust of clinical workers in the model. At the same time, the model can also continuously analyze and integrate clinical data to improve model performance, thereby providing reliable auxiliary diagnosis for the treatment of thyroid nodules. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the auxiliary diagnosis model of thyroid nodules of the present invention;

[0015] Figure 2 is a receiver operating characteristic curve (ROC) diagram of the best prediction model for thyroid nodules of the present invention;

[0016] Figure 3 Schematic diagram of the DCA curve of the best prediction model obtained in the embodiment of the present invention;

[0017] Figure 4 SHAP feature importance diagram of the best prediction model obtained in the embodiment of the present invention;

[0018] Figure 5 SHAP scatter diagram of the best prediction model obtained in the embodiment of the present invention

[0019] Figure 6 It is a LIME schematic diagram of the best prediction model obtained in the embodiment of the present invention. DETAILED DESCRIPTION

[0020] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings.

[0021] The present invention first obtains the original data related to patients with thyroid nodules, including basic patient information, test and related medical examination data, etc., and preprocesses the obtained data; then, the preprocessed data is subjected to feature selection, and a training data set is constructed based on the selected features; different prediction models are constructed using a preset machine learning algorithm, and each prediction model is cross-validated to determine the performance indicators of each prediction model; the machine learning model (prediction model) with the best performance is selected according to the performance indicators, and then the hyperparameters of the model are further adjusted, and cross-validation and iterative training are performed to obtain the best machine learning model (best prediction model); the obtained best prediction model is used for auxiliary diagnosis; and the best prediction model is interpreted and analyzed based on interpretability technology. The implementation process of this method is as follows: Figure 1 The following is a detailed description of the process.

[0022] 1. Obtain raw data related to patients with thyroid nodules and preprocess them.

[0023] The present invention obtains data related to patients with thyroid nodules, including information such as gender, age, medical history, clinical symptoms, laboratory tests, imaging and pathological examinations. Laboratory tests include thyroid function tests, blood indicators, etc.; imaging tests include ultrasound, CT or MRI examinations; pathological examinations include postoperative pathological results of patients. Since the acquired original data may contain missing values, duplicate values, and large differences in values, the present invention also preprocesses the acquired data to ensure the integrity and accuracy of the data, wherein the preprocessing methods include duplicate value deletion, outlier processing, and missing value filling, etc. The missing value filling is mainly completed by multiple interpolation technology based on the KNN algorithm.

[0024] 2. Build an automated feature engineering framework and input the preprocessed data into the automated feature engineering framework for feature engineering processing.

[0025] Since the acquired data contains many types, there may be highly correlated and collinear features, zero variance or near-zero variance features in the data set. Therefore, the present invention needs to perform feature engineering on the acquired data to delete redundant features through feature engineering. The feature engineering process of the present invention mainly includes feature preprocessing and feature selection.

[0026] Feature preprocessing mainly involves data set division, data conversion, one-hot encoding, feature normalization, and unbalanced data processing on the acquired data.

[0027] Feature selection mainly further selects feature data after feature preprocessing, wherein the feature selection method of the present invention includes: the nearZeroVar function is used to identify near-zero variance variables. The parameters freqCut and uniqueCut set the frequency ratio and the ratio of unique values ​​respectively. The index of the near-zero variance variable is extracted, and these variables are deleted, and the numerical variables are selected using the sapply function. The correlation matrix descrCorr of the numerical variables is calculated. The findCorrelation function is used to identify highly correlated variables (correlation coefficient greater than 0.90) and delete variables with high collinearity.

[0028] The feature data obtained through this step and its corresponding label data are made into a data set, and the data set is divided into a training set and a test set in a ratio of 8:2.

[0029] 3. Build an automatic machine learning framework and input the processed features, use the machine learning framework to automatically traverse the preset machine learning models, and cross-validate each machine learning model (also called prediction model) to determine the performance indicators of each prediction model.

[0030] The preset machine learning algorithms used in the present invention include XGBoost, Neural Network, RandomForest Classifier, Logistic Regression, Multilayer Perceptron, C5.0DecisionTree, Gaussian Process, Gradient Boosting Machine, KNN, SVM-Linear Kernel and NaiveBayes Classifier, etc. Different prediction models are constructed using the above machine learning algorithms, and they are trained using the training set. The prediction models obtained are cross-validated in the validation set using the default parameters of the algorithm to obtain the performance indicators of each model, and the models are sorted based on the obtained performance indicators.

[0031] The performance indicators used in the present invention include: AUC-ROC, Accuracy, F1-score, Sensitivity and Specificity, where AUC-ROC represents the area under the receiver operating characteristic curve, which is an indicator for evaluating the model's ability to distinguish positive and negative samples at different thresholds. The closer the value is to 1, the better the model performance. Accuracy represents the ratio of samples correctly predicted and classified by the model to the total samples, which is the success rate of the model;

[0032] F1-score is the harmonic mean of Recall and Precision; Sensitivity is also called Recall or True Positive Rate (TPR). Sensitivity indicates the proportion of samples that are correctly predicted as positive among all samples that are actually positive. Specificity is also called True Negative Rate (TNR). Specificity indicates the proportion of samples that are correctly predicted as negative among all samples that are actually negative.

[0033] The present invention uses the validation set to validate each prediction model, and the performance indicators of each prediction model are shown in Table 1. It can be seen from Table 1 that the XGBoost model performs best among all models, and the performance indicators of the prediction model such as Accuracy, AUC-ROC, F1-score, Sensitivity, and Specificity are 0.76, 0.81, 0.70, 0.77, and 0.76, respectively.

[0034] Table 1 Performance indicators of each model on the test set

[0035]

[0036] 4. Select the prediction model with the best performance based on the performance indicators, then further adjust the model's hyperparameters, and perform cross-validation and iterative training to obtain the best prediction model.

[0037] The present invention takes Accuracy as a benchmark, selects the model with the highest Accuracy as the optimal model, further adjusts the hyperparameters of the model to obtain the optimal model, and uses a cross-validation method for iterative training to obtain the optimal prediction model.

[0038] The hyperparameter adjustment of the model can be performed by grid search, random search, Bayesian optimization, etc. For this embodiment, the model with the highest accuracy is the XGBoost model, and the hyperparameters of the model after grid search tuning are:

[0039] eXtreme Gradient Boosting Pre-processing:centered(9),scaled(9)

[0040] Resampling:Cross-Validated(5fold)

[0041] Summary of sample sizes:713,713,712,714,712

[0042] Resampling results across tuning parameters:

[0043] eta max_depth gamma colsample_bytree min_child_weightsubsamplenrounds ROC Sens Spec ROCSD SensSD SpecSD 0.1 6 0.5 0.7 1 0.8 50 0.8705579 0.7989289 0.7686935 0.0225699 0.06570371 0.04806882

[0045] 5. Performance indicators of the best prediction model after hyperparameter tuning.

[0046] The present invention uses a test set to verify the model with adjusted hyperparameters, and obtains performance indicators such as Accuracy, AUC-ROC, F1-score, Sensitivity, and Specificity of the prediction model, which are 0.79, 0.83, 0.80, 0.78, and 0.75, respectively; the Accuracy of the prediction model after hyperparameter adjustment is 3% higher than that of the original model.

[0047] To better illustrate the performance of the model, the following is a visual analysis of the performance of the best prediction model. The automatic machine learning framework can automatically plot the performance indicators of the model, including the ROC curve ( Figure 2 ) and DCA curve ( Figure 3 ) etc. The DCA curve is drawn based on the model prediction results, and the clinical practicality of the model is evaluated by the net benefits under different thresholds. The calculation of net benefits takes into account the gains and losses predicted by the model, and the DCA curve shows the net benefits of the model under different decision thresholds. When drawing the DCA curve, the threshold is used as the horizontal axis and the net benefit is used as the vertical axis. By comparing the net benefits of different thresholds, the optimal decision threshold can be determined to improve the clinical practicality of the model.

[0048] 6. Performing interpretability analysis on the optimal model after hyperparameter adjustment can further understand the model and provide a reference for medical decision-making.

[0049] Explainability is very important in the medical field. A medical diagnosis system must be transparent and explainable to gain the trust of doctors and patients. Explainability technologies include: SHAP and LIME.

[0050] The SHAP distribution importance of the features of the best prediction model obtained in this example in the data set is as follows Figure 4 , 5 As shown, Figure 4 The importance of each feature to the model prediction is shown. The horizontal axis represents the mean absolute value of the SHAP value (mean(|SHAPvalue|)). The higher the value, the greater the impact of the feature on the model prediction. Each bar in the figure represents a feature, which is arranged in descending order of importance from top to bottom. It can be seen that Age.year, Tumor.size and C.TIRADS have the greatest impact on the performance of the constructed model. Figure 5 This is a scatter plot of SHAP, which shows the SHAP value distribution of all samples for each feature. The horizontal axis represents the SHAP value and the vertical axis represents the feature. The color represents the size of the feature value (from low to high), which is used to show how the feature value affects the predicted value and show the overall contribution of the feature to the model prediction.

[0051] LIME can be used to observe the impact of each feature on the prediction in each specific sample, such as Figure 6 The results for the 150th patient are shown. The vertical axis in the figure represents the importance ranking of the patient's features, and the horizontal axis represents the feature weight, with blue representing support and red representing opposition. In addition, the model's predicted probability and fitness are also given in the figure. Through this visual analysis, we can intuitively see to what extent which features support or oppose the model's prediction results for this sample. This explanation helps to understand the model's decision-making process and provides in-depth analysis of a single sample.

[0052] 7. Interpretable Machine Learning Thyroid Nodule Diagnosis System

[0053] The auxiliary diagnosis system for thyroid nodules of the present invention includes a processor, a memory and a display, and the processor executes a computer program stored in the memory to implement the auxiliary diagnosis method of the present invention. The specific description is as follows: The processor can be a microprocessor (such as MCU) or a programmable logic device (such as FPGA), which is used to execute the computer program stored in the memory. The processor is equipped with various operating systems, such as Windows operating system, Linux system, Android system, iOS system, etc. The memory includes a physical device for storing information, and common ones are RAM, ROM, hard disk, USB flash drive, etc. The display is used to display the diagnosis results and explanation information, which is convenient for clinicians to refer to and make decisions. The working principle of the system is as follows: The system realizes auxiliary diagnosis of thyroid nodules based on an interpretable machine learning model. The processor executes the computer program in the memory, and the specific steps are as follows: The data input system receives the patient's clinical data and ultrasound data, etc. Data preprocessing preprocesses the input data, including data cleaning, feature extraction and other steps. Model prediction uses the trained interpretable machine learning model to predict the preprocessed data and generate a diagnosis result. The model uses SHAP (Shapley Additive Explanations) and LIME methods to explain the contribution of each feature to the prediction result. The result interpretation system generates visual interpretations such as SHAP value graphs and feature importance graphs to show the impact of each feature on the diagnosis result. These explanatory information helps clinicians understand the basis for model decisions and improves the transparency and credibility of the diagnosis. The result display shows the diagnosis results and explanatory information for doctors' reference. Doctors can make more accurate diagnostic decisions based on the information provided by the system.

[0054] From the above process, it can be seen that the present invention can be effectively used to analyze and integrate large-scale medical data to help doctors make decisions and guide clinical practice, bringing precision medicine to patients with thyroid nodules, which has important clinical significance.

[0055] The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Many changes and modifications may be made without departing from the scope of the present invention. Therefore, it is intended that the above detailed description is considered to be illustrative rather than restrictive, and it should be understood that the following claims (including all equivalents) are intended to limit the spirit and scope of the present invention. The above embodiments should be understood to be only used to illustrate the present invention and not to limit the scope of protection of the present invention. After reading the content of the record of the present invention, the technician can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall into the scope defined by the claims of the present invention.

Claims

1. A thyroid nodule auxiliary diagnosis model based on explainable machine learning, characterized in that: The method includes the following steps: S1, retrospectively collecting clinical data, examination and test data of patients with thyroid nodules, constructing a dataset of patients with thyroid nodules, and preprocessing the data; S2. Build an automated feature engineering framework and input the preprocessed data into the automated feature engineering framework for feature engineering. S3. Build a machine learning model with the features processed by feature engineering and evaluate the model. S4. Select the optimal model through performance parameters and then determine the optimal model through hyperparameter adjustment, cross-validation, and iterative training. S5. Perform interpretable processing of the overall features and individualized analysis of the best-performing model.

2. The thyroid nodule auxiliary diagnosis model based on explainable machine learning according to claim 1, characterized in that: The patient information obtained in step S1) includes the patient's gender, thyroid nodule ultrasound results, and related medical examination data and pathological data.

3. The thyroid nodule auxiliary diagnosis model based on explainable machine learning according to claim 1, characterized in that: The data preprocessing includes deleting duplicate values, processing outliers, filling missing values, etc. The filling of missing values ​​is mainly completed by multiple interpolation technology based on KNN algorithm.

4. The thyroid nodule auxiliary diagnosis model based on explainable machine learning according to claim 1, characterized in that: The feature engineering of step S2) includes feature preprocessing and feature selection, wherein feature preprocessing includes data set partitioning, data conversion, unique hot encoding, feature normalization and unbalanced data processing of the acquired data. Feature selection includes the nearZeroVar function for identifying near-zero variance variables. The parameters freqCut and uniqueCut set the frequency ratio and the ratio of unique values ​​respectively. Extract the index of the near-zero variance variables, delete these variables, and use the sapply function to select numerical variables. Calculate the correlation matrix descrCorr of the numerical variables. Use the findCorrelation function to identify highly correlated variables (correlation coefficient greater than 0.90) and delete variables with high collinearity.

5. The thyroid nodule auxiliary diagnosis model based on explainable machine learning according to claim 1, characterized in that: The main evaluation performance indicators in step S3) include: AUC-ROC, Accuracy, F1-score, Sensitivity and Specificity, where AUC-ROC represents the area under the receiver operating characteristic curve, which is an indicator for evaluating the model's ability to distinguish between positive and negative samples at different thresholds. The closer the value is to 1, the better the model performance. Accuracy represents the ratio of samples correctly predicted by the model to the total samples, which is the success rate of the model; F1-score is the harmonic mean that comprehensively considers Recall and Precision; Sensitivity is also called Recall or True Positive Rate (TPR). Sensitivity represents the proportion of samples correctly predicted by the model as positive samples among all samples that are actually positive samples. Specificity is also called True Negative Rate (TNR). Specificity represents the proportion of samples correctly predicted by the model as negative samples among all samples that are actually negative samples.

6. The thyroid nodule auxiliary diagnosis model based on explainable machine learning according to claim 1, characterized in that: The machine learning models preset in step S3) include XGBoost, Neural Network, RandomForest Classifier, Logistic Regression, Multilayer Perceptron, C5.0DecisionTree, Gaussian Process, Gradient Boosting Machine, KNN, SVM-Linear Kernel and NaiveBayes Classifier, etc.

7. The thyroid nodule auxiliary diagnosis model based on explainable machine learning according to claim 5, characterized in that: In the step S4), the optimal prediction model is selected based on Accuracy.

8. The thyroid nodule auxiliary diagnosis model based on explainable machine learning according to claim 1, characterized in that: The interpretability methods of step S5) include: SHAP and LIME.

9. A thyroid nodule auxiliary diagnosis system based on explainable machine learning, characterized in that: The auxiliary diagnosis system includes a processor, a memory and a display; the memory is used to store executable instructions of the processor; wherein the processor is configured to implement the thyroid nodule auxiliary diagnosis model based on explainable machine learning as described in any one of claims 1-8 by executing the executable instructions, and the display displays the results.

Citation Information

Cited By

  • Artificial intelligence multi-group pulmonary nodule auxiliary diagnosis system

    CN121075617A