Medical AI decision interpretability enhancement method and system

Abstract

The invention relates to the field of artificial intelligence medical treatment, in particular to a medical AI decision interpretability enhancement method and system, and the method comprises the steps: obtaining the case data of a patient; predicting an explanatory diagnosis result which comprises a diagnosis result and an explanatory diagnosis feature; generating an interpretive diagnosis result confrontation sample and a background sample; screening data related to the explanatory diagnosis features; respectively training a medical AI decision interpretability enhancement model and an AI decision model related to patient diseases; a diagnosis result is predicted through a medical AI decision interpretability enhancement model, and the method further comprises the steps of constructing an uncertainty weight set, carrying out weight removal operation on interpretation features and model behavior data, obtaining interpretability medical image data, generating anti-fact interpretation and carrying out visualization processing. The transparency of AI auxiliary diagnosis is improved by innovatively introducing a double-sample cooperation mechanism and combining the technologies of interpretive diagnosis result generation, hierarchical screening multi-stage training sample set construction, uncertainty weight reconstruction model and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence in medicine, in particular to a medical AI decision explainability enhancement method and system, aiming to solve the medical AI black box problem and improve the transparency and credibility of AI-assisted diagnosis. BACKGROUND

[0002] With the in-depth application of artificial intelligence technology in the medical field, AI-assisted diagnosis systems have shown great potential in disease detection, diagnosis and prognosis prediction. However, the current medical AI systems generally have the problem of black box, that is, although the system can give the diagnosis result, it cannot explain the basis and process of making a specific diagnosis decision. This opacity seriously restricts the wide application of AI technology in the high-risk field of medicine.

[0003] Traditional AI explainability methods mainly include feature importance-based explanations (such as SHAP, LIME, etc.) and example-based explanations (such as prototype networks). Although these methods can provide explanations to some extent, they have the shortcomings of one-sided, incomplete, and difficult to understand. In particular, in the medical scenario, a single explanation method is difficult to meet the understanding needs of doctors for complex diagnosis decisions, and it is also difficult to establish the trust of doctors in AI systems.

[0004] For the explainability problem of medical AI, there are three types of solutions in the prior art: first, use models with certain explainability, such as decision trees, rule systems, etc., but such models often perform worse than deep learning models when dealing with complex medical data; second, explain the trained black box model after the fact, such as using feature attribution methods, but such methods can usually only provide local explanations and are difficult to fully reveal the model decision mechanism; third, design a special explainability architecture, but existing architectures often ignore the special nature of medical decisions and are difficult to provide medically meaningful explanations.

[0005] Therefore, there is an urgent need for a medical AI decision explainability enhancement method that can provide comprehensive explanations from multiple angles and levels to solve the medical AI black box problem, improve the transparency and credibility of AI-assisted diagnosis, and promote the safe and effective application of AI technology in the medical field. SUMMARY

[0006] The purpose of the present application is to provide a medical AI decision explainability enhancement method and system, which fundamentally solves the medical AI black box problem, improves the transparency and credibility of AI-assisted diagnosis, by innovatively introducing a double-sample collaborative mechanism, combining the generation of explainable diagnosis results, the collaborative application of adversarial samples and background samples, the construction of a multi-level training sample set with hierarchical screening, the modeling of uncertainty weights, and the visualization of counterfactual explanations.

[0007] The application provides a medical AI decision explainability enhancement method, comprising:

[0008] Obtaining patient case data;

[0009] Based on the patient case data, an explanatory diagnosis result of the patient's disease is predicted, wherein the explanatory diagnosis result of the patient's disease includes a patient disease diagnosis result and an explanatory diagnosis feature corresponding to the patient disease diagnosis result;

[0010] According to a first preset mode, an explanatory diagnosis result adversarial sample is generated based on the patient case data and the explanatory diagnosis result;

[0011] According to a second preset mode, an explanatory diagnosis result background sample is generated based on the patient case data and the explanatory diagnosis result;

[0012] According to the explanatory diagnosis feature corresponding to the patient disease diagnosis result, relevant data of the explanatory diagnosis feature is screened from the explanatory diagnosis result adversarial sample data and the explanatory diagnosis result background sample data respectively;

[0013] For the explanatory diagnosis result background sample, an AI decision model related to the patient disease is trained based on the screened relevant data;

[0014] For the explanatory diagnosis result adversarial sample, a medical AI decision explainability enhancement model is trained based on the screened relevant data; and

[0015] Through the medical AI decision explainability enhancement model, it is predicted whether the diagnosis result of the patient is the patient disease diagnosis result.

[0016] As a preferred, according to the first preset mode, the explanatory diagnosis result adversarial sample is generated based on the patient case data and the explanatory diagnosis result, specifically comprising:

[0017] According to the first preset mode, a random adversarial sample is generated based on the patient case data and the explanatory diagnosis result, and an input adversarial feature of the random adversarial sample is confirmed based on an explainability technology;

[0018] Based on the explanatory diagnosis result, a preset threshold is determined;

[0019] When the difference between the explanatory diagnosis result and the preset threshold is less than the preset threshold, the input adversarial feature of the random adversarial sample is updated by using a stochastic gradient descent method;

[0020] Based on the updated input adversarial feature, the explanatory diagnosis result adversarial sample is generated.

[0021] Preferably, according to the second preset mode, the explanatory diagnosis result background sample is generated based on the patient case data and the explanatory diagnosis result, and specifically includes:

[0022] According to the second preset mode, a random background sample is generated based on the patient case data and the explanatory diagnosis result, and based on an explainable technology, input background features of the random background sample are confirmed;

[0023] According to the explanatory diagnosis features corresponding to the patient condition diagnosis result, the input background features are screened to obtain input background features related to the explanatory diagnosis features;

[0024] According to a first preset screening rule, part of the input background features related to the explanatory diagnosis features are selected to generate the explanatory diagnosis result background sample.

[0025] Preferably, based on the relevant data screened, the medical AI decision explainability enhancement model is trained for the explanatory diagnosis result adversarial sample, and specifically includes:

[0026] According to the explanatory diagnosis features corresponding to the patient condition diagnosis result, a first training sample set is established for the relevant data screened in the explanatory diagnosis result adversarial sample data;

[0027] According to the explanatory diagnosis features corresponding to the patient condition diagnosis result, a second training sample set is established for the data other than the relevant data screened in the explanatory diagnosis result adversarial sample data;

[0028] The first training sample set and the second training sample set are input into a machine learning model;

[0029] The machine learning model includes the medical AI decision explainability enhancement model.

[0030] Preferably, after obtaining the input background features related to the explanatory diagnosis features, it further includes:

[0031] According to a second preset screening rule, part of the input background features related to the explanatory diagnosis features are selected to generate the explanatory diagnosis result background sample;

[0032] According to the explanatory diagnosis features corresponding to the patient condition diagnosis result, a third training set sample is established for the relevant data screened in the explanatory diagnosis result background sample data;

[0033] According to the explanatory diagnostic features corresponding to the diagnostic results of the patient's condition, relevant data outside the data in the explanatory diagnostic result background sample data is screened to establish a fourth training set sample;

[0034] The third training set sample and the fourth training set sample are input into the medical AI decision explainability enhancement model.

[0035] As a preferred, the method further comprises:

[0036] Obtaining the explanation features of the data set and the associated model behavior data;

[0037] Constructing an uncertainty weight set between the explanation features and the target variable data;

[0038] According to the uncertainty weight set, the explanation features and the associated model behavior data are respectively de-weighted to obtain an explainability weight set of the explanation features of the model;

[0039] According to the explainability weight set of the explanation features of the model, obtain the explainable medical image data.

[0040] As a preferred, the method further comprises:

[0041] Based on the uncertainty weight set and the explainability weight set, selecting a key variable as a benchmark for counterfactual explanation;

[0042] Based on the key variable, generating a plurality of counterfactual explanations;

[0043] The plurality of counterfactual explanations are weighted as test data to reversely generate diagnostic results;

[0044] The plurality of counterfactual explanations are visualized to form visualized counterfactual explanations.

[0045] As a preferred, the visualizing processing comprises:

[0046] Selecting a visualization type suitable for the counterfactual explanation, the visualization type comprising at least one of feature importance heat map, feature comparison chart and diagnostic result trend chart;

[0047] Designing visualization elements, including color scheme, graphic layout and interactive function;

[0048] Constructing an interactive interface to allow medical personnel to adjust feature values and observe diagnostic result changes in real time;

[0049] Integrating the counterfactual explanation, feature importance and visualization elements to form a hierarchical display interface.

[0050] As preferred, the patient case data comprises:

[0051] demographic features of the patient, including age, gender and weight;

[0052] medical features of the patient, including blood pressure, heart rate and body temperature;

[0053] medical imaging data of the patient, including CT imaging data and X-ray imaging data;

[0054] laboratory examination data of the patient, including blood examination data and urine examination data.

[0055] a medical AI decision explainability enhancement system, comprising:

[0056] a data input module configured to obtain patient case data;

[0057] an explainable diagnosis result prediction module configured to predict an explainable diagnosis result of a patient condition based on the patient case data; wherein the explainable diagnosis result of the patient condition comprises a patient condition diagnosis result and an explainable diagnosis feature corresponding to the patient condition diagnosis result;

[0058] an adversarial sample generation module configured to generate an explainable diagnosis result adversarial sample based on the patient case data and the explainable diagnosis result in a first preset manner;

[0059] a background sample generation module configured to generate an explainable diagnosis result background sample based on the patient case data and the explainable diagnosis result in a second preset manner;

[0060] a data screening module configured to screen data related to the explainable diagnosis feature from the explainable diagnosis result adversarial sample data and the explainable diagnosis result background sample data respectively according to the explainable diagnosis feature corresponding to the patient condition diagnosis result;

[0061] an enhancement model training module configured to train an AI decision model related to the patient condition based on the screened related data for the explainable diagnosis result background sample, and train a medical AI decision explainability enhancement model based on the screened related data for the explainable diagnosis result adversarial sample;

[0062] a diagnosis prediction module configured to predict whether the diagnosis result of the patient is the patient condition diagnosis result through the medical AI decision explainability enhancement model.

[0063] The present application has the following advantages:

[0064] 1. Comprehensive Interpretation: By the synergy of adversarial samples and background samples, both local interpretation (exploring decision boundaries) and global interpretation (exploring feature space) are provided, achieving comprehensive and deep interpretation. Compared with single interpretation method, it can reveal the internal mechanism of AI decision from multiple dimensions, greatly improving the understanding depth of medical staff on AI decision.

[0065] 2. Accurate Feature Importance Evaluation: Uncertainty weight modeling is introduced to consider the uncertain influence of features, providing more accurate feature importance evaluation. Through the counterfactual verification mechanism, it ensures that the generated explanation is consistent with the actual model behavior, significantly improving the accuracy and reliability of the explanation, and reducing misleading explanations.

[0066] 3. Intuitive Decision Explanation: Through the visualization and dynamic generation of counterfactual explanations, intuitive and easy-to-understand decision explanations are provided. Interactive interface design allows doctors to adjust feature values and observe the changes in diagnosis results in real time, greatly improving the usability of explanations and reducing the cognitive burden of doctors understanding AI decisions.

[0067] 4. Improve diagnostic accuracy: Doctors can understand AI diagnosis results through the explanations provided by the system, combine their own experience to make judgments, and form a doctor-AI collaborative diagnosis mode, which significantly improves the diagnostic accuracy and reduces the misdiagnosis rate.

[0068] 5. Promote medical knowledge discovery: Counterfactual explanations generated by the system can help discover potential disease-related factors, providing new ideas and directions for medical research, and also serving as a medical education tool to help medical students understand complex diagnostic decision-making processes.

[0069] 6. Promote clinical application of medical AI: Solving the key obstacle to the clinical application of medical AI-the black box problem, making AI-assisted diagnosis systems evolve from giving answers to giving answers and reasons, greatly improving the adoption rate of AI systems in clinical practice, and promoting medical AI from research to widespread clinical application. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 Flowchart of the medical AI decision explainability enhancement method of the present application;

[0071] Figure 2 Flowchart of the explanation diagnosis result generation of the present application;

[0072] Figure 3 Flowchart of the adversarial sample generation of the present application;

[0073] Figure 4 Flowchart of the background sample generation of the present application;

[0074] Figure 5A flowchart of a multi-stage training sample set construction process for the layered screening of the present application;

[0075] Figure 6 A flowchart of an uncertainty weight modeling process for the present application;

[0076] Figure 7 A flowchart of a counterfactual explanation generation and visualization process for the present application;

[0077] Figure 8 A structural diagram of a medical AI decision explainability enhancement system for the present application. DETAILED DESCRIPTION

[0078] Reference should be made to the accompanying drawings and specific embodiments described below. Figures 1-8 The following detailed description is provided to better understand the present application, and is not intended to limit the scope of the present application. Those skilled in the art can make several improvements and refinements to the present application without departing from the principles and concepts of the present application, and these improvements and refinements should also be considered within the scope of protection of the present application.

[0079] Referring to Figure 1 The present application provides a medical AI decision explainability enhancement method, which mainly includes the following steps:

[0080] First, patient case data is obtained. In an embodiment of the present application, patient case data can be obtained from multiple sources such as electronic medical record systems, medical imaging systems, laboratory examination systems, etc. These data usually include patient demographic characteristics (such as age, gender and weight), medical characteristics (such as blood pressure, heart rate and body temperature), medical imaging data (such as CT imaging data and X-ray imaging data) and laboratory examination data (such as blood test data and urine test data).

[0081] Next, based on the patient case data, an explainable diagnosis result of the patient's condition is predicted. The explainable diagnosis result includes two key parts: the patient's condition diagnosis result (i.e. the disease type and the diagnosis confidence) and the explainable diagnosis features corresponding to the patient's condition diagnosis result (i.e. the key features that have the greatest impact on the diagnosis result and their importance).

[0082] Referring to Figure 2 The generation process of the explainable diagnosis result can be divided into the following steps:

[0083] First, the patient data is preprocessed, including standardization (unifying all indicators to the same dimension), missing value processing (using multiple imputation methods to fill in missing data), outlier detection (using the quartile method to identify outliers) and encoding conversion of categorical features.

[0084] Then feature extraction and transformation are performed, the medical image data is subjected to resolution unification processing, a deep learning method based on ResNet architecture is used to extract image features, structured data and image features are fused to form a comprehensive feature vector, and the feature vector is subjected to dimension reduction processing to retain key information.

[0085] Then diagnostic result prediction is performed, the preprocessed comprehensive feature vector is input into the trained deep learning model, and the model outputs the disease type and its probability distribution. Preferably, the confidence threshold is set to 0.75 (this threshold is based on the consideration of high diagnosis requirements in medical diagnosis), high-confidence diagnostic results are screened, and multiple possible disease types are sorted.

[0086] Subsequently, explanatory feature extraction is performed, feature importance analysis techniques (such as SHAP or LIME) are used to identify the features that have the greatest impact on the diagnostic results, and the contribution score of each feature is calculated. Preferably, the feature importance threshold is set to 0.15 (this threshold is determined through experiments and can effectively screen out key features without introducing too much noise), key features are screened, and the impact direction (positive or negative) of each feature on the diagnostic results is determined.

[0087] Finally, the explanatory diagnostic results are integrated, the diagnostic results are associated with the explanatory features, a standardized explanatory diagnostic result data structure is constructed, the explanatory features are sorted by importance, and an explanatory diagnostic result report is generated.

[0088] This explanatory diagnostic result not only contains the diagnostic conclusion, but also contains the key features supporting the conclusion and their importance, laying a foundation for subsequent explainability enhancement.

[0089] Then, according to the first preset mode, an explanatory diagnostic result adversarial sample is generated based on the patient case data and the explanatory diagnostic result. Referring to Figure 3 The generation process of the adversarial sample specifically includes:

[0090] First, according to the first preset mode, a random adversarial sample is generated based on the patient case data and the explanatory diagnostic result, and the input adversarial features of the random adversarial sample are confirmed based on the explainability technology. In a preferred embodiment of the present application, the generation of the random adversarial sample is realized by applying a small random perturbation to the original patient case data, and the perturbation amplitude is controlled within ±5% of the original feature value, to ensure that the generated sample meets the medical rationality constraints.

[0091] Then, based on the interpretive diagnosis result, a preset threshold is determined. In a preferred embodiment of the present application, the preset threshold is set to 75% of the confidence of the original diagnosis result. For example, if the confidence of the original diagnosis result is 0.9, the preset threshold is 0.675. This threshold setting is to ensure that the generated adversarial sample is close enough to the decision boundary to effectively detect the local sensitivity of the model, while not deviating too far to cause the inability to provide meaningful explanations.

[0092] Subsequently, when the difference between the interpretive diagnosis result and the preset threshold is less than the preset threshold, the input adversarial features of the random adversarial sample are updated using the stochastic gradient descent method. In a preferred embodiment of the present application, the learning rate of the stochastic gradient descent method is set to 0.01, and the number of iterations is set to 100. After each iteration, the medical rationality constraint is checked to ensure that the updated sample is still valid.

[0093] Finally, based on the updated input adversarial features, an interpretive diagnosis result adversarial sample is generated. The purpose of the adversarial sample is to explore the sample space near the decision boundary, test the robustness of the model in the boundary region, and help understand the critical point at which the diagnosis result may change.

[0094] Next, according to a second preset manner, an interpretive diagnosis result background sample is generated based on the patient case data and the interpretive diagnosis result. Referring to Figure 4 , the generation process of the background sample specifically includes:

[0095] First, according to the second preset manner, a random background sample is generated based on the patient case data and the interpretive diagnosis result, and the input background features of the random background sample are confirmed based on the explainability technology. In a preferred embodiment of the present application, the generation of the random background sample is achieved by randomly selecting data points from the historical patient database, ensuring that the selected samples cover multiple disease types and feature distributions, and the number of samples is usually 3-5 times the original data.

[0096] Then, according to the interpretive diagnosis features corresponding to the patient's disease diagnosis result, the input background features are screened to obtain input background features related to the interpretive diagnosis features. In a preferred embodiment of the present application, the correlation is calculated using the mutual information (MI) index:

[0097] ,

[0098] Where: is the mutual information value between the feature and the target variable , used to quantify the mutual dependence between the two variables; is the possible value of the feature ; is the target variable the possible values of the feature is a feature the possible values of the feature and the target variable the possible values of the feature the joint probability of is a feature the marginal probability of the marginal probability of is a target variable the marginal probability of the marginal probability of is a natural logarithm function. The mutual information value ranges from non-negative real numbers, and the larger the value, the stronger the correlation between the feature and the target variable. Preferably, the correlation threshold is set to 0.6, and features higher than the threshold are considered to be highly correlated with the explanatory diagnostic feature.

[0099] Finally, according to the first preset screening rule, part of the input background features related to the explanatory diagnostic feature is selected to generate the explanatory diagnostic result background sample. In a preferred embodiment of the present application, the first preset screening rule is to retain the top 60% of the correlation ranking features. This proportion is set to retain enough relevant features to provide rich contextual information while avoiding the introduction of too many secondary features that may interfere with interpretation.

[0100] The purpose of the background sample is to provide more extensive feature space context information to help understand the stability of feature importance in different situations.

[0101] After generating the adversarial sample and the background sample, according to the explanatory diagnostic feature corresponding to the patient's disease diagnosis result, the relevant data is screened from the explanatory diagnostic result adversarial sample data and the explanatory diagnostic result background sample data, respectively.

[0102] Referring to Figure 5 , the medical AI decision interpretability enhancement model is trained based on the relevant data screened for the explanatory diagnostic result adversarial sample. This process specifically includes:

[0103] First, according to the explanatory diagnostic feature corresponding to the patient's disease diagnosis result, a first training sample set is established for the relevant data screened from the explanatory diagnostic result adversarial sample data. In a preferred embodiment of the present application, the screening of relevant data is based on the correlation between the feature and the explanatory diagnostic feature, and the correlation is calculated using the mutual information index, with a correlation threshold set to 0.4.

[0104] Then, according to the explanatory diagnostic features corresponding to the diagnosis result of the patient's disease, the relevant data obtained by screening the anti-explanatory diagnostic result sample data is established. The two training sample sets respectively represent the feature set that has an important influence on the diagnosis result and the feature set that has a smaller influence.

[0105] Finally, the first training sample set and the second training sample set are input into the machine learning model, wherein the machine learning model includes a medical AI decision explainability enhancement model. In a preferred embodiment of the present application, the machine learning model adopts a deep neural network architecture, and different sample weights are assigned to the two training sample sets: the sample weight of the first training sample set (the relevant feature set) is 1.5, and the sample weight of the second training sample set (the non-relevant feature set) is 0.8. This weight distribution enables the model to pay more attention to the features that have an important influence on the diagnosis result.

[0106] Next, based on the relevant data obtained by screening, an AI decision model related to the patient's disease is trained for the explanatory diagnostic result background sample. After obtaining the input background features related to the explanatory diagnostic features, the following is further included:

[0107] According to the second preset screening rule, part of the input background features of the input background features related to the explanatory diagnostic features is selected to generate the explanatory diagnostic result background sample. In a preferred embodiment of the present application, the second preset screening rule is to retain the top 40% of the relevant features. This proportion is set to focus more on the most critical features and provide more accurate explanations.

[0108] Then, according to the explanatory diagnostic features corresponding to the diagnosis result of the patient's disease, the relevant data obtained by screening the explanatory diagnostic result background sample data is established. According to the explanatory diagnostic features corresponding to the diagnosis result of the patient's disease, the data other than the relevant data obtained by screening the explanatory diagnostic result background sample data is established.

[0109] Finally, the third training set sample and the fourth training set sample are input into the medical AI decision explainability enhancement model. In a preferred embodiment of the present application, the sample weight of the third training set sample (the relevant feature set in the background sample) is 1.2, and the sample weight of the fourth training set sample (the non-relevant feature set in the background sample) is 0.5. This weight distribution enables the model to focus on key features while also considering the changes in feature importance in different backgrounds.

[0110] This hierarchical training method enables the model to accurately distinguish between truly important features, significantly improving the explainability of the decision, while providing a structured data basis for subsequent uncertainty modeling.

[0111] After the above steps are completed, the present application can further implement the following steps:

[0112] First, the explanatory features of the data set and the associated model behavior data are obtained. In a preferred embodiment of the present application, the explanatory features refer to the features extracted from the trained model that have a significant impact on the prediction results, and the model behavior data refers to the changes in the prediction results of the model under different input conditions.

[0113] Then, the uncertainty weight set between the explanatory features and the target variable data is constructed. In a preferred embodiment of the present application, the calculation of the uncertainty weight uses the following formula:

[0114]

[0115] Wherein: is the uncertainty weight, representing the degree of certainty of the relationship between the feature value and the target variable; is the value of the explanatory feature; is the value of the target variable; is the Gaussian error distribution parameter, controlling the scale of uncertainty, and the preferred value is 0.1; is the natural exponential function; is the constant of pi. This formula is derived from the Gaussian kernel function, and essentially calculates the standardized measure of the difference between the feature value and the target variable. The uncertainty weight ranges from 0 to 1, and the closer the value is to 1, the more certain the feature's impact on the target variable, and the closer the value is to 0, the less certain the impact.

[0116] Next, according to the uncertainty weight set, the explanatory features and the associated model behavior data are respectively de-weighted to obtain the explainability weight set of the model's explanatory features. In a preferred embodiment of the present application, the formula for de-weighting operation is as follows:

[0117]

[0118] Wherein: is the de-weighted data, which has the same dimension as the original data ; is the original data; is the element-level multiplication operator, indicating multiplication of corresponding elements; is the normalized weight, calculated as to ensure that the sum of all weights is 1; is Gaussian noise, following a normal distribution with mean 0 and variance 0.01, used to increase the robustness of the data. The purpose of this operation is to reduce the impact weight of uncertain features and highlight the role of certain features, thereby improving the accuracy and reliability of the explanation. For a data set with dimension ​​The de-weighting operation will keep the vector dimension unchanged but adjust the weight contribution of each feature.

[0119] Finally, interpretable medical image data is obtained based on the interpretability weights of the model's explanatory features. In a preferred embodiment of the present invention, interpretable medical image data is generated by overlaying the original medical image with a feature importance heat map, which intuitively displays the areas in the image that have a significant impact on the diagnostic results.

[0120] In addition, the present invention can also select key variables as the benchmark for counterfactual explanations based on the uncertainty weight set and the interpretability weight set; generate multiple counterfactual explanations based on the key variables; weight the multiple counterfactual explanations as test data and reversely generate diagnostic results; and visualize the multiple counterfactual explanations to form a visualized counterfactual explanation.

[0121] Reference Figure 7 The visualization processing of counterfactual explanations includes: selecting a visualization type suitable for counterfactual explanations, which includes at least one of feature importance heat maps, feature comparison maps, and diagnostic result change trend maps; designing visualization elements, including color schemes, graphic layouts, and interactive functions; building an interactive interface that allows medical personnel to adjust feature values ​​and observe changes in diagnostic results in real time; integrating counterfactual explanations, feature importance, and visualization elements to form a hierarchical display interface.

[0122] In a preferred embodiment of the present invention, the feature importance heat map adopts a gradient color scheme from blue (low importance) to red (high importance) to intuitively display the distribution of feature importance; the feature comparison chart adopts a bar chart format, simultaneously displaying the original feature values ​​and counterfactual feature values, and marking them with different colors; the diagnosis result change trend chart adopts a line chart format to show the trend of diagnosis probability changes caused by feature value changes.

[0123] This visualization method enables medical personnel to intuitively understand the basis of AI decisions, achieve what-you-see-is-what-you-get explainability, and complete the closed loop of the entire technical solution.

[0124] Finally, the medical AI decision-making explainability enhancement model predicts whether the patient's diagnosis is correct for the patient's condition. In a preferred embodiment of the present invention, the medical AI decision-making explainability enhancement model outputs not only the diagnosis result and confidence level, but also the key features supporting the diagnosis and their importance, as well as counterfactual explanations (how the diagnosis would change if certain features were changed), providing doctors with a comprehensive and intuitive basis for decision-making.

[0125] Reference Figure 8 The present invention also provides a medical AI decision interpretability enhancement system, which includes the following modules:

[0126] The data input module 10 is configured to obtain patient case data. In a preferred embodiment of the present application, the data input module 10 comprises a data receiving unit and a data preprocessing unit. The data receiving unit is configured to receive patient data from multiple sources such as electronic medical record systems, medical imaging systems, laboratory examination systems, etc.; and the data preprocessing unit is configured to perform preprocessing operations such as cleaning, standardization, conversion, etc. on the received data to ensure data quality and consistency.

[0127] The explanatory diagnosis result prediction module 20 is configured to predict an explanatory diagnosis result of a patient condition based on patient case data; wherein the explanatory diagnosis result of the patient condition comprises a patient condition diagnosis result and an explanatory diagnosis feature corresponding to the patient condition diagnosis result. In a preferred embodiment of the present application, the explanatory diagnosis result prediction module 20 comprises a diagnosis prediction unit and a feature extraction unit. The diagnosis prediction unit is configured to predict a disease type and a diagnosis confidence based on preprocessed patient data using a deep learning model; and the feature extraction unit is configured to extract features and their importance that have important influence on the diagnosis result using feature importance analysis techniques such as SHAP or LIME.

[0128] The adversarial sample generation module 30 is configured to generate an explanatory diagnosis result adversarial sample based on patient case data and an explanatory diagnosis result in a first preset manner. In a preferred embodiment of the present application, the adversarial sample generation module 30 comprises a random perturbation unit, a threshold calculation unit, a gradient update unit and a sample generation unit. The random perturbation unit is configured to apply a small random perturbation to the original patient data; the threshold calculation unit is configured to set a preset threshold based on the original diagnosis result confidence; the gradient update unit is configured to update the adversarial feature using a stochastic gradient descent method; and the sample generation unit is configured to generate a final adversarial sample based on the updated feature.

[0129] The background sample generation module 40 is configured to generate an explanatory diagnosis result background sample based on patient case data and an explanatory diagnosis result in a second preset manner. In a preferred embodiment of the present application, the background sample generation module 40 comprises a sample selection unit, a correlation calculation unit and a feature screening unit. The sample selection unit is configured to randomly select data points from a historical patient database as background samples; the correlation calculation unit is configured to calculate the correlation between the background features and the explanatory diagnosis features; and the feature screening unit is configured to select relevant background features according to a preset screening rule.

[0130] The data screening module 50 is configured to screen data related to the explanatory diagnostic feature from the adversarial sample data and the background sample data of the explanatory diagnostic result according to the explanatory diagnostic feature corresponding to the diagnosis result of the patient's disease. In a preferred embodiment of the present application, the data screening module 50 comprises a correlation analysis unit and a data classification unit. The correlation analysis unit is responsible for calculating the correlation between the features in the sample data and the explanatory diagnostic feature; and the data classification unit is responsible for classifying the data into relevant data and irrelevant data according to a correlation threshold.

[0131] The enhanced model training module 60 is configured to train an AI decision model related to the patient's disease based on the relevant data screened from the background sample of the explanatory diagnostic result, and train a medical AI decision explainability enhancement model based on the relevant data screened from the adversarial sample of the explanatory diagnostic result. In a preferred embodiment of the present application, the enhanced model training module 60 comprises a sample set construction unit and a model training unit. The sample set construction unit is responsible for constructing four training sample sets (adversarial sample relevant data set, adversarial sample irrelevant data set, background sample relevant data set, and background sample irrelevant data set); and the model training unit is responsible for training the medical AI decision explainability enhancement model using the four training sample sets.

[0132] The diagnosis prediction module 70 is configured to predict whether the diagnosis result of the patient is the diagnosis result of the patient's disease by the medical AI decision explainability enhancement model. In a preferred embodiment of the present application, the diagnosis prediction module 70 not only outputs the diagnosis result, but also outputs the key features supporting the diagnosis and their importance, and the counterfactual explanation, thereby providing comprehensive decision basis for doctors.

[0133] In addition, the system can further comprise an uncertainty weight modeling module (not shown in the figure) configured to calculate the uncertainty weight of the features and construct an explainability weight set; and a counterfactual explanation generation module (not shown in the figure) configured to generate and visualize the counterfactual explanation.

[0134] The visualization interface of the system of the present application can comprise the following parts: the left side is a patient basic information and diagnosis result area, which displays the basic information of the patient and the AI-predicted disease type and confidence; the middle is a feature importance display area, which displays the features having important influence on the diagnosis result and their importance in the form of a heat map or a column chart; and the right side is a counterfactual explanation area, which allows doctors to adjust the feature values and observe the changes in the diagnosis result in real time.

[0135] The medical AI decision explainability enhancement method and system provided by the application fundamentally solve the black box problem of medical AI, improve the transparency and reliability of AI-assisted diagnosis by innovatively introducing a double-sample collaborative mechanism, combining explainable diagnosis result generation, collaborative application of adversarial samples and background samples, hierarchical screening of multi-level training sample set construction, uncertainty weight modeling and counterfactual explanation visualization, etc.

[0136] The method can provide comprehensive explanations from multiple angles and multiple levels, help doctors understand the basis of AI diagnosis, improve diagnostic accuracy, reduce misdiagnosis rates, promote medical knowledge discovery and medical education, and promote medical AI from research to widespread clinical application, making an important contribution to improving medical quality and efficiency.

[0137] The above description is only the preferred embodiment of the application and is not intended to limit the application. For those skilled in the art, the application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A method for enhancing the explainability of medical AI decisions, characterized by: include: Obtain patient case data; Predicting an explanatory diagnosis result of the patient's condition based on the patient case data, wherein the explanatory diagnosis result of the patient's condition includes the patient's condition diagnosis result and an explanatory diagnostic feature corresponding to the patient's condition diagnosis result; According to a first preset manner, generating an explanatory diagnosis result adversarial sample based on the patient case data and the explanatory diagnosis result; According to a second preset manner, based on the patient case data and the explanatory diagnosis result, an explanatory diagnosis result background sample is generated; According to the explanatory diagnostic feature corresponding to the patient's disease diagnosis result, respectively screening data related to the explanatory diagnostic feature from the explanatory diagnostic result confrontation sample data and the explanatory diagnostic result background sample data; For the background samples of the explanatory diagnostic results, based on the relevant data obtained by screening, an AI decision model related to the patient's condition is trained; For the explanatory diagnostic result adversarial samples, a medical AI decision-making explainability enhancement model is trained based on the relevant data obtained by screening; and The medical AI decision explainability enhancement model is used to predict whether the patient's diagnosis result is the patient's disease diagnosis result.

2. The method for enhancing the explainability of medical AI decisions according to claim 1, characterized in that: According to the first preset method, based on the patient case data and the explanatory diagnosis result, generating the explanatory diagnosis result adversarial sample specifically includes: Generating a random adversarial sample based on the patient case data and the explanatory diagnosis result according to the first preset manner, and confirming the input adversarial features of the random adversarial sample based on the explainability technology; determining a preset threshold based on the explanatory diagnostic result; When the difference between the explanatory diagnosis result and the preset threshold is less than the preset threshold, updating the input adversarial feature of the random adversarial sample using a stochastic gradient descent method; Based on the updated input adversarial features, the explanatory diagnosis result adversarial sample is generated.

3. The method for enhancing the explainability of medical AI decisions according to claim 1, characterized in that: According to the second preset manner, generating the explanatory diagnosis result background sample based on the patient case data and the explanatory diagnosis result specifically includes: According to the second preset manner, a random background sample is generated based on the patient case data and the explanatory diagnosis result, and input background features of the random background sample are confirmed based on an explainability technique; Screening the input background features according to the explanatory diagnostic features corresponding to the patient's disease diagnosis results to obtain input background features related to the explanatory diagnostic features; According to a first preset screening rule, some input background features of the input background features related to the explanatory diagnostic features are selected to generate the explanatory diagnostic result background sample.

4. The method for enhancing the explainability of medical AI decisions according to claim 1, characterized in that: For the explanatory diagnostic result adversarial sample, the medical AI decision interpretability enhancement model is trained based on the relevant data obtained by screening, specifically including: Establishing a first training sample set based on the explanatory diagnostic features corresponding to the patient's disease diagnosis result and the relevant data obtained by screening the adversarial sample data for the explanatory diagnostic result; According to the explanatory diagnostic features corresponding to the patient's disease diagnosis result, the explanatory diagnostic result is screened against the adversarial sample data to obtain data other than relevant data, thereby establishing a second training sample set; Inputting the first training sample set and the second training sample set into a machine learning model; Among them, the machine learning model includes the medical AI decision explainability enhancement model.

5. The method for enhancing the explainability of medical AI decisions according to claim 4, characterized in that: After obtaining the input background features related to the explanatory diagnostic features, the method further includes: selecting, according to a second preset screening rule, a portion of the input background features that are relevant to the explanatory diagnostic features, to generate the explanatory diagnostic result background sample; Establishing a third training set of samples based on the explanatory diagnostic features corresponding to the patient's disease diagnosis results and the relevant data screened from the background sample data of the explanatory diagnostic results; According to the explanatory diagnostic features corresponding to the patient's disease diagnosis result, data other than relevant data is screened from the background sample data of the explanatory diagnostic result to establish a fourth training set sample; The third training set samples and the fourth training set samples are input into the medical AI decision explainability enhancement model.

6. The method for enhancing the explainability of medical AI decisions according to claim 5, characterized in that: The method further comprises: Obtain explanatory features of the dataset and associated model behavior data; Constructing a set of uncertainty weights between the explanatory features and the target variable data; De-weighting the explanatory features and the associated model behavior data according to the uncertainty weight set to obtain an interpretability weight set of the explanatory features of the model; Obtain interpretable medical imaging data based on an interpretability weight set of the explanatory features of the model.

7. The method for enhancing the explainability of medical AI decisions according to claim 6, characterized in that: The method further comprises: Selecting key variables as benchmarks for counterfactual explanations based on the uncertainty weight set and the interpretability weight set; generating multiple counterfactual explanations based on the key variables; weighting the multiple counterfactual explanations as test data to reversely generate a diagnosis result; The multiple counterfactual explanations are visualized to form a visualized counterfactual explanation.

8. The method for enhancing the explainability of medical AI decisions according to claim 7, characterized in that: The visualization process includes: Selecting a visualization type suitable for the counterfactual explanation, the visualization type including at least one of a feature importance heat map, a feature comparison map, and a diagnosis result change trend map; Design visual elements, including color schemes, graphic layouts, and interactive features; Build an interactive interface that allows medical personnel to adjust feature values ​​and observe changes in diagnostic results in real time; The counterfactual explanations, feature importance, and visualization elements are integrated to form a hierarchical display interface.

9. The method for enhancing the explainability of medical AI decisions according to claim 1, characterized in that: The patient case data includes: patient demographic characteristics, including age, sex, and weight; the patient's medical characteristics, including blood pressure, heart rate, and temperature; Patients' medical imaging data, including CT imaging data and X-ray imaging data; The patient's laboratory test data, including blood test data and urine test data.

10. Medical AI decision-making explainability enhancement system, characterized by: The system comprises: Data input module, used to obtain patient case data; An explanatory diagnostic result prediction module is used to predict an explanatory diagnostic result of the patient's condition based on the patient case data; wherein the explanatory diagnostic result of the patient's condition includes the patient's condition diagnosis result and an explanatory diagnostic feature corresponding to the patient's condition diagnosis result; an adversarial sample generation module, configured to generate an explanatory diagnostic result adversarial sample based on the patient case data and the explanatory diagnostic result in a first preset manner; A background sample generating module, configured to generate an explanatory diagnosis result background sample based on the patient case data and the explanatory diagnosis result in a second preset manner; a data screening module, configured to screen data related to the explanatory diagnostic feature corresponding to the patient's disease diagnosis result from the explanatory diagnostic result confrontation sample data and the explanatory diagnostic result background sample data, respectively; An enhanced model training module is used to train an AI decision model related to the patient's condition based on the relevant data obtained by screening for the background samples of the explanatory diagnosis results; and to train a medical AI decision-making interpretability enhancement model based on the relevant data obtained by screening for the adversarial samples of the explanatory diagnosis results; The diagnosis prediction module is used to predict whether the patient's diagnosis result is the patient's disease diagnosis result through the medical AI decision explainability enhancement model.

Images