A multimodal medical AI system decision behavior consistency evaluation method

Abstract

This invention discloses a method for evaluating the consistency of decision-making behavior in a multimodal medical AI system, belonging to the field of medical artificial intelligence evaluation technology. The method first constructs a multimodal clinical pathway knowledge graph integrating images, text, and standard knowledge as an evaluation benchmark, and parses the decision output of the AI ​​to be evaluated into a structured directed graph. Then, it utilizes a cross-modal embedding model to unify the semantic representation of nodes, and through a cross-modal hypergraph neural network, comprehensively calculates the consistency across the structural, semantic, and modal dimensions. An interpretable consistency score is generated based on a dynamic weighted graph edit distance algorithm, and anomalies are automatically detected. Furthermore, this method supports interactive dynamic testing and robustness analysis. This invention achieves automated, comprehensive, and interpretable consistency evaluation of the medical AI decision-making process.

Description

Technical Field

[0001] This invention belongs to the field of medical artificial intelligence model evaluation technology, specifically involving a method for evaluating the consistency of decision-making behavior in a multimodal medical AI system. It is particularly suitable for automated and interpretable consistency quality evaluation of medical AI systems that integrate multiple modal information such as medical images and text for decision-making. Background Technology

[0002] In recent years, large-scale language models and multimodal artificial intelligence have shown great potential in medical diagnosis and treatment decision support. However, multimodal medical AI systems face challenges in reliability and consistency in practical applications, and the correctness of their decision-making processes and their consistency with clinical knowledge urgently need effective evaluation. Currently, the evaluation of medical AI systems mainly relies on methods such as manual annotation and task accuracy. For example, for the evaluation of generative medical AI systems, indicators such as Reference Answer Accuracy (RAAR), Subjective Answer Accuracy (SAAR), and Strict Accuracy (SAR) are often used for quality assessment. However, these evaluation methods have the following shortcomings:

[0003] 1. Lack of structured semantic alignment mechanism: Existing methods mostly measure the output of medical AI systems by the accuracy of question-and-answer, and lack direct judgment on whether the diagnostic conclusions or suggestions given by the AI ​​system are consistent with the medical knowledge system. Especially in complex reasoning scenarios, it is difficult to ensure the consistency of the output with the standard clinical pathway.

[0004] 2. Difficulty in detecting logical flaws in the decision-making process: Traditional assessments focus on the correctness of the results and cannot identify logical errors that may exist in the decision-making process of medical AI systems, such as skipping necessary examinations and drawing conclusions directly, or making inferences based on unrelated information, which are abnormalities in the reasoning process.

[0005] 3. Insufficient robustness testing: Sensitivity to input noise or adversarial interference is often not adequately assessed. Existing evaluations rarely simulate variable fluctuations in clinical scenarios, failing to guarantee that the model will maintain stable decision outputs even when image quality deteriorates or medical record information is biased.

[0006] 4. Lack of cross-modal consistency verification: When medical AI systems need to integrate image and text information, current evaluations lack checks on the consistency of information fusion across different modalities. For example, current methods struggle to promptly detect inconsistencies between image and text conclusions regarding whether the model-generated textual diagnostic conclusions match the image evidence.

[0007] To address the aforementioned issues, a novel evaluation method is urgently needed to integrate multimodal medical knowledge and conduct comprehensive consistency analysis of the decision-making process of medical AI systems, encompassing structural, semantic, and cross-modal aspects. This application proposes a technical solution combining clinical pathway knowledge graphs and cross-modal hypergraph neural networks, aiming to overcome the shortcomings of existing technologies and provide an automatic and comprehensive consistency evaluation of the decision-making behavior of multimodal medical AI systems. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for evaluating the consistency of decision-making behavior in multimodal medical AI systems. This method constructs a multimodal clinical pathway knowledge graph as the evaluation benchmark and utilizes an innovative cross-modal hypergraph neural network and a dynamic weighted graph editing distance algorithm to achieve automated, refined, and interpretable quantitative evaluation of the consistency of decision-making behavior in medical AI systems. It also supports interactive testing and robustness analysis.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A method for evaluating the consistency of decision-making behavior in a multimodal medical AI system includes the following steps:

[0011] S1: Construct a multimodal clinical pathway knowledge graph based on authoritative clinical guidelines and real electronic medical record databases. The knowledge graph is obtained by performing consistency and confidence screening on candidate subgraphs. The candidate subgraphs include clinical pathway nodes coded with standardized medical terms, temporal dependency edges between nodes, and multimodal edges formed by medical images and their text information associated with the nodes.

[0012] S2: Input the target medical record into the medical AI system to be evaluated and output the medical image segmentation results, text description and decision path. Parse the output into a first directed graph with nodes and edges.

[0013] S3: By using a pre-trained multimodal joint coding model, nodes in the knowledge graph and nodes in the first directed graph are mapped to the same semantic space to generate corresponding cross-modal embedding vectors.

[0014] S4: Input the knowledge graph and the directed graph into the graph neural network model and calculate the comprehensive similarity between the two in the three dimensions of structure, semantics and modality;

[0015] Based on the comprehensive similarity and the preset cost function, the minimum cost required to edit the first directed graph to be consistent with the knowledge graph is calculated, and a first consistency score is generated based on the minimum cost.

[0016] S5: Based on the differences between the knowledge graph and the first directed graph, identify and label the first set of anomalies in the decision-making behavior of the medical AI system to be evaluated;

[0017] S6: Perform robustness verification on the decision-making behavior of the medical AI system to be evaluated, and output a structured evaluation report.

[0018] In step S1, a multimodal clinical pathway knowledge graph is constructed based on authoritative clinical guidelines and real electronic medical record databases. The knowledge graph is obtained by performing consistency and confidence screening on candidate subgraphs. The candidate subgraphs include clinical pathway nodes encoded with standardized medical terms, temporal dependency edges between nodes, and multimodal edges formed by medical images and their text information associated with the nodes.

[0019] Clinical pathway nodes are extracted from authoritative clinical guidelines, coded using a standard clinical medical terminology system, and temporal dependency edges representing the order of nodes are established.

[0020] The system integrates medical images and their text information associated with clinical pathway nodes from a real electronic medical record database, identifies the medical images using a standard radiological terminology system, uses an image segmentation model to identify regions of interest in the medical images, and aligns the regions of interest and their corresponding text information with the associated clinical pathway nodes to form multimodal edges.

[0021] In step S2, the target medical record is input into the medical AI system to be evaluated and the system outputs medical image segmentation results, text descriptions and decision paths. The output is then parsed into a first directed graph with nodes and edges.

[0022] Obtain the medical image segmentation results, text descriptions, and decision paths output by the medical AI system to be evaluated for the target medical record;

[0023] Extract regions of interest from medical image segmentation results and filter them. Generate standardized descriptive text for the retained regions of interest and encode them with standardized medical terminology. Output a structured list of regions of interest.

[0024] Analyze the decision path, extract decision nodes, judgment criteria and reasoning steps as nodes, and establish directed edges according to the logical or temporal relationship between steps to construct an initial directed graph;

[0025] The text descriptions of the behavioral nodes in the initial directed graph are semantically linked with the standard medical term entities in the knowledge graph. This generates a link relationship and link confidence score for each behavioral node pointing to the standard medical term entities, and outputs a standardized directed graph.

[0026] In step S3, the nodes in the knowledge graph and the nodes in the first directed graph are mapped to the same semantic space through a pre-trained multimodal joint coding model, generating corresponding cross-modal embedding vectors.

[0027] The nodes in the knowledge graph and the nodes in the directed graph are encoded by a pre-trained multimodal joint encoding model, mapped to the public semantic space through a shared projection layer, and then normalized.

[0028] Contrastive learning is used for training to make the embedding vectors of image regions, text descriptions, and knowledge entities with the same clinical semantics close to each other in the semantic space.

[0029] In step S4, the knowledge graph and the directed graph are input into the graph neural network model, and the comprehensive similarity between the two in the three dimensions of structure, semantics and modality is calculated. Based on the comprehensive similarity and the preset cost function, the minimum cost required to edit the first directed graph to be consistent with the knowledge graph is calculated, and the first consistency score is generated based on the minimum cost.

[0030] The graph neural network model is a cross-modal hypergraph neural network, which enables nodes in the knowledge graph and the directed graph to perceive each other during the information transmission process through a cross-graph attention mechanism;

[0031] The minimum cost required to edit the first directed graph to be consistent with the knowledge graph is calculated using a dynamically weighted graph edit distance algorithm, where the cost of the edit operation is determined by structural differences, semantic differences, and modal differences, and the weight of each difference is dynamically adjusted through a learnable gating function based on clinical prior knowledge.

[0032] Furthermore, the dynamic weighted graph edit distance algorithm introduces a stage alignment penalty term and a causal chain disruption penalty term;

[0033] When the order of decision nodes in the first directed graph is misaligned with the expected stage order of the corresponding nodes in the knowledge graph, a stage alignment penalty term is introduced for the graph editing distance based on the degree of misalignment, reflecting the impact of decision stage deviation on consistency evaluation. When there are missing key preceding nodes, reversed causal order, or subsequent decision nodes generated based on unnecessary conditions in the first directed graph, a causal chain disruption penalty term is introduced for editing operations that disrupt the integrity of the clinical causal chain, reducing the consistency score of such decision-making behavior.

[0034] In step S5, based on the difference between the knowledge graph and the first directed graph, the first set of anomalies in the decision-making behavior of the medical AI system to be evaluated is identified and labeled.

[0035] The first set of anomalies includes at least one of the following: path skipping, cross-modal inconsistency, or redundant nodes;

[0036] Path jumps include missing intermediate nodes marked as necessary in the knowledge graph of the directed graph;

[0037] Cross-modal contradictions include semantic inconsistencies between image evidence and text evidence associated with the same node in a directed graph;

[0038] Redundant nodes include nodes in the directed graph that have low semantic similarity to any node in the knowledge graph and are not necessary.

[0039] In step S6, the robustness of the decision-making behavior of the medical AI system to be evaluated is verified, and a structured evaluation report is output.

[0040] Robustness verification includes:

[0041] Apply perturbations to the target medical record, generate at least one set of perturbations, and then input them.

[0042] The perturbated input is fed into the medical AI system to be evaluated, and its output of perturbated medical image segmentation results, text descriptions and decision paths is obtained and parsed into a second directed graph.

[0043] Repeat steps S3 to S4 to calculate the second consistency score between the second directed graph and the knowledge graph;

[0044] The decrease in the first consistency score and the second consistency score is calculated as a robustness quantification indicator.

[0045] By comparing the second directed graph with the knowledge graph, we identify the new or changed difference nodes and edges that cause the score to drop, and denot them as the second anomaly set.

[0046] The output structured evaluation report includes:

[0047] The system comprises a first consistency score, a second consistency score, a first set of anomalies, a robustness quantification index, and a second set of anomalies; the second set of anomalies is used to locate vulnerable links in the medical AI decision-making process.

[0048] The beneficial effects of this invention are as follows:

[0049] 1. Comprehensive evaluation dimensions: The consistency of the three dimensions of structure, semantics, and modality is uniformly quantified, overcoming the shortcomings of traditional methods that have only one evaluation dimension.

[0050] 2. High interpretability: Through graph edit distance, difference subgraphs, anomaly location reports, etc., it clearly reveals the specific differences between the medical AI system's decision-making and the standard path, not only providing scores, but also pointing out the root cause of the problem.

[0051] 3. Robustness Guarantee: The embedded interactive adversarial testing and decomposed robust optimization mechanism can systematically evaluate the stability of the model in the face of noise and uncertainty, thereby improving the reliability of the evaluation results.

[0052] 4. Automation and Efficiency: It achieves full automation from knowledge construction and AI output parsing to consistency calculation and report generation, which significantly improves evaluation efficiency and is suitable for iterative development and continuous monitoring of medical AI systems. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating a method for evaluating the consistency of decision-making behavior in a multimodal medical AI system, as provided by this invention.

[0055] Figure 2 This is a detailed flowchart of step S4 of the multimodal medical AI system decision behavior consistency evaluation method provided by the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0057] The present invention provides a method for evaluating the consistency of decision-making behavior in a multimodal medical AI system, which mainly includes six steps, and the specific implementation is as follows:

[0058] like Figure 1 As shown, in step S1, a multimodal clinical pathway knowledge graph is constructed based on authoritative clinical guidelines and real electronic medical record databases. The knowledge graph is obtained by performing consistency and confidence screening on candidate subgraphs. The candidate subgraphs include clinical pathway nodes encoded with standardized medical terms, temporal dependency edges between nodes, and multimodal edges formed by medical images and their text information associated with the nodes.

[0059] Preferably, standardized clinical pathway node sequences are extracted from authoritative clinical guidelines such as NCCN or Chinese Medical Association guidelines, such as "chest X-ray examination → suspected nodule diagnosis → pathological biopsy → lung cancer diagnosis → surgical resection and treatment", and encoded using the SNOMED CT standard medical terminology system, and temporal dependency edges are established to form a clinical pathway subgraph.

[0060] From an anonymized real electronic medical record database, medical images and their corresponding radiological terminology sets (RadLex) corresponding to the aforementioned path nodes are integrated. Image segmentation models are used to identify key regions of interest (ROIs) in the images, and natural language processing techniques are used to extract key phrases and medical terminology entities from the reports. Through contrastive learning, multimodal association edges are established between a specific image ROI, its corresponding textual phrase, and the standard medical conceptual entities. These edges are assigned learnable weights to characterize their association strength.

[0061] The candidate subgraphs are subjected to consistency and confidence screening to remove conflicts and duplicates, resulting in the multimodal clinical pathway knowledge graph G. std .

[0062] Given a sequence of candidate nodes Its text label or attribute is ,

[0063]

[0064]

[0065] in:

[0066] :and The corresponding region of interest in the image;

[0067] Text channel model (parameters are) ) on the concept The posterior probability;

[0068] Image channel model (parameters are) ) on the concept The posterior probability;

[0069] The most likely SNOMED CT concept given by the text channel;

[0070] The most likely RadLex concept given by the image channel;

[0071] The confidence level of this node is the larger of the posterior probabilities of the two channels.

[0072] : The encoded set of nodes;

[0073] in:

[0074] ;

[0075] ;

[0076] There is a confidence threshold ,like Then set it to "Uncertain".

[0077] Contrastive learning is performed on positive alignment pairs at the medical record level, embedding the ROI with corresponding text and knowledge entities, increasing the cosine similarity of the positive alignment, and obtaining multimodal edges with learnable weights for subsequent GNN propagation and scoring. Use directly with anomaly detection.

[0078] Formula expression:

[0079] For each case Take positive samples Let the embedding vector be:

[0080]

[0081] in:

[0082] Sample index;

[0083] , , : No. The region of interest in the image, text fragments, and knowledge entities of each sample;

[0084] ROI / Text / Entity Encoder;

[0085] : The corresponding embedding vector of ROI / text / entity;

[0086] Calculate the InfoNCE loss and train to obtain , Normalization method:

[0087]

[0088]

[0089]

[0090] in:

[0091] : InfoNCE loss of the text corresponding to the ROI;

[0092] : ROI corresponding entity's InfoNCE loss;

[0093] : Minimize the total loss by using the parameters;

[0094] The scoring function, in this formula, is the cosine similarity. ;

[0095] Temperature coefficient, the steepness of the scaling score, always greater than 0;

[0096] Edge weights generated:

[0097]

[0098]

[0099] in:

[0100] : The edge weight of the text corresponding to the ROI;

[0101] : The edge weight of the entity corresponding to the ROI;

[0102] The Sigmoid function maps scores to... interval;

[0103] : Scale factor, edge weight steepness;

[0104] : Bias, shift threshold;

[0105] , : respectively represent The specific values ​​to be taken in the two categories of ROI corresponding text and ROI corresponding entity;

[0106] The set of multimodal weighted edge triples, the union of which combines the two sets of edges into a single graph.

[0107] Directed edges such as "examination → diagnosis" and "diagnosis → treatment" are generated according to the guidelines' prerequisite relationships, and clinical importance weights are assigned.

[0108] Rule filtering:

[0109]

[0110] in:

[0111] , Clinical behavioral milestones, including examination, diagnosis, and treatment;

[0112] : A set of temporal rules, where elements have a directed sequential relationship. ;

[0113] : Indicator function, returns 1 if true, and 0 if false;

[0114] : Indicates whether the timing rule exists. As will be known below, a value of 0 will be discarded.

[0115] Weight calculation:

[0116]

[0117] in:

[0118] The importance weight of the sequence relationship. Within the range;

[0119] : Guideline confidence level Within the range;

[0120] , The weighting coefficients for the two signals are usually taken as follows: ;

[0121] Temporal edge weights;

[0122] Time verification and edge generation:

[0123]

[0124]

[0125]

[0126] in:

[0127] Temporal directed edges and their weights;

[0128] , : Node timestamp or topological order index;

[0129] Minimum lag time, always greater than 0;

[0130] : The final set of temporal edges output.

[0131] Overall output knowledge graph :

[0132]

[0133] like Figure 1 As shown, in step S2, the target medical record is input into the medical AI system to be evaluated and the system outputs medical image segmentation results, text descriptions and decision paths. The output is parsed into a first directed graph with nodes and edges.

[0134] Given a target medical record, it is input into the multimodal medical AI system to be evaluated. The system outputs three results: medical image segmentation (usually a probabilistic graph or mask), text description (such as a diagnostic report), and decision path (usually a log or explanatory output of the system's internal logic). The decision path is then parsed, extracting key decision points, judgment criteria, and reasoning steps as nodes, and extracting logical or temporal dependencies between steps as edges, thus constructing an initial directed graph structure. To make the graph machine-readable and traceable, it is represented using machine-readable semantic forms such as RDF or OWL, and each node and edge is assigned a globally unique identifier, which is bound to the corresponding segment in the original output data to achieve full-process traceability.

[0135] Specifically, firstly, the probability maps for each category are thresholded to obtain binary masks. Next, connected component analysis is performed on each binary mask to extract independent connected components as candidate regions of interest (ROIs). These ROIs are then lightly denoised, and non-maximum suppression (NMS) is applied to merge ROIs with high spatial overlap and the same category. For each final determined ROI, the coordinates of its minimum bounding box are calculated, and the mean of the original probabilities within that bounding box is taken as the confidence level of the ROI. Minimum area and minimum confidence thresholds are set to filter out invalid ROIs that may be generated by noise. Finally, a normalized descriptive text containing category, location coordinates, size, and confidence level is generated for each retained ROI, and its corresponding standard medical terminology is encoded according to the RadLex radiology terminology set. A structured list of ROIs is output, providing input for subsequent cross-modal alignment and edge weight calculation.

[0136] Based on decision nodes and combined with the actual timestamp information of cases, all clinical behaviors are treated as nodes. First, time alignment is performed, merging events with the same or highly similar meanings and adjacent times. Then, according to the standard process sequence in clinical guidelines and the actual occurrence sequence in case records, the temporal relationship between events is determined, and these are used as edges to connect them, thereby constructing a directed graph describing the development of the case. During the edge connection process, if a loop that violates the temporal logic appears, the system will calculate the confidence of each edge on the loop and remove the edge with the lowest confidence to ensure that the graph structure is a directed acyclic graph (DAG). Each clinical event node is associated with and bound to its corresponding image ROI and its standardized descriptive text, forming an image-text evidence subgraph, which facilitates the traceability and visualization of diagnostic decisions. Finally, the directed decision graph and its topological sorting sequence are output.

[0137] The text descriptions accompanying each clinical behavior node are preprocessed, including regularization cleaning and synonym merging. Then, a unified text encoder encodes the processed node text into semantic embedding vectors. Simultaneously, standard medical terminology entities in the knowledge graph are isomorphically encoded. In the vector space, the cosine similarity between each node embedding and the knowledge graph entity embedding is calculated as semantic similarity. To improve link accuracy, graph context is introduced as a prior, and candidate entity scores are reweighted. The entity with the highest reweighted score is selected as the link target for that node, and its link confidence is recorded. If the highest score is lower than a preset similarity threshold, the node is marked as an unknown entity, and its candidate entity list is saved for subsequent manual verification and knowledge graph expansion. Finally, link triples are output. ,in:

[0138] Behavioral nodes in the case;

[0139] :predicate;

[0140] : The unique code of the standard ontology item to which it is linked;

[0141] : Confidence of the link.

[0142] This allows the behavior nodes in the original decision-oriented directed graph to be normalized and aligned to a standard medical ontology, forming a final, machine-understandable and reasonable normalized directed graph.

[0143] like Figure 1 As shown, in step S3, the nodes in the knowledge graph and the nodes in the first directed graph are mapped to the same semantic space through a pre-trained multimodal joint coding model, generating corresponding cross-modal embedding vectors.

[0144] A joint encoding model pre-trained on a large medical multimodal dataset is employed. This model typically includes a visual encoder, ViT-Rad (Vision Transformer), and a text encoder, BioClinical-BERT. For entities in the knowledge graph, their textual descriptions are encoded using the text encoder.

[0145] In practical implementation, a weight-sharing dual encoder architecture can be adopted: the image branch is encoded independently, while the text and knowledge entity branches share the same text encoder parameters; the encoded outputs of the three branches are mapped to a 256-dimensional common semantic space through a shared projection layer and then L2 normalized; the training process uses the contrastive learning loss InfoNCE loss, which aims to bring image region embedding vectors, text phrase embedding vectors, and knowledge entity embedding vectors that belong to the same semantic concept (such as "lung nodule") closer together in the common space, and push away unrelated embedding vectors.

[0146] During reasoning, a trained encoder is used to process the knowledge graph G. std And directed graph G ai All nodes generate a normalized set of cross-modal embedding vectors Z.

[0147] Represented as:

[0148] Given batch Encode and project, and then perform L2 normalization:

[0149]

[0150]

[0151]

[0152]

[0153]

[0154] in:

[0155] Batch size;

[0156] , , : respectively the first The first image, the first The text, the first Three modalities for each knowledge entity;

[0157] Image encoder (ViT-Rad) and its parameters ;

[0158] Text encoder (BioClinical-BERT) and its parameters ;

[0159] BioClinical-BERT physical encoder and its parameters ;

[0160] , , These are the latent vectors encoded in three modalities: image, text, and knowledge entity.

[0161] Shared projector head and its parameters , mapped to Fixed 256 dimensions;

[0162] , , : The unnormalized vector after projection;

[0163] : L2 normalized vector.

[0164] Calculate similarity and InfoNCE loss:

[0165]

[0166]

[0167]

[0168]

[0169] Overall goal:

[0170]

[0171] in:

[0172] The dot product similarity of vectors, after L2 normalization, becomes the cosine similarity.

[0173] , , : These are the InfoNCE losses for image-to-text, image-to-knowledge-entity, and text-to-knowledge-entity, respectively.

[0174] Temperature coefficient, always greater than 0;

[0175] , , The weighting coefficients of the three types of InfoNCE loss;

[0176] : Jointly minimize to train three sets of parameters.

[0177] Inference phase output:

[0178]

[0179] The normalized embedding vectors of the three modalities can be directly used for cosine similarity or dot product, as well as subsequent edge weight calculation and consistency evaluation.

[0180] like Figure 1 , Figure 2 As shown, in step S4, the knowledge graph and the directed graph are input into the graph neural network model, and the comprehensive similarity between the two in the three dimensions of structure, semantics and modality is calculated. Based on the comprehensive similarity and the preset cost function, the minimum cost required to edit the first directed graph to be consistent with the knowledge graph is calculated, and the first consistency score is generated based on the minimum cost.

[0181] Knowledge Graph G std Directed graph G ai Its nodes are embedded in a Z-input cross-modal hypergraph neural network.

[0182] Specifically, it includes the following:

[0183] Allow knowledge graph G std And directed graph G ai The nodes pay attention to each other during the information transmission process, so that the final representation of each node incorporates the information of the potential corresponding node in the other graph, which significantly improves the alignment sensitivity and outputs a cross-graph attention matrix A.

[0184] Clinical prior knowledge is quantified into auxiliary vectors, and the weights in the GNN message passing process are dynamically adjusted through a learnable gating function, making the network learning process more targeted.

[0185] This mechanism exists in message passing and cross-graph attention within the graph, systematically embedding domain knowledge and clinical auxiliary quantities into the GNN neural network and matching process, enabling message propagation and subsequent computation of dynamically weighted graph edit distance. They operate under the same set of reliability weights.

[0186]

[0187] in:

[0188] Relationship gating weights;

[0189] Auxiliary vector;

[0190] A relational gating function is a learnable mapping that... Mapped to ;

[0191] : The set of function parameters, which typically includes a weight vector and a bias.

[0192] In the In the message passing of a multi-layered GNN neural network, for nodes... The update formula is as follows:

[0193]

[0194]

[0195] in:

[0196] : The neighborhood set, from ;

[0197] : Conventional structural attention or normalization coefficient;

[0198] , : Weight matrix of GNN neural network;

[0199] : Activation function, using the ReLU function.

[0200] This mechanism introduces a hypergraph structure to model multimodal evidence units, which can directly express the multiple correspondences between multimodalities. When any node is updated in the GNN neural network, it not only obtains the information of a certain neighboring node, but also obtains the joint information of the entire group of neighboring nodes through the hyperedge.

[0201] Similarity calculation includes structural similarity and semantic similarity:

[0202] Structural similarity:

[0203]

[0204] Semantic similarity:

[0205]

[0206] Modal alignment:

[0207]

[0208] in:

[0209] Knowledge Graph One of the nodes;

[0210] Directed graph One of the nodes;

[0211] Matrix represented by nodes Indicated The adjacency set;

[0212] Matrix represented by nodes Indicated The adjacency set;

[0213] : semantic embedding vector;

[0214] : semantic embedding vector;

[0215] : By attention matrix The representation of matching The corresponding cross-modal mutual information normalization reduction;

[0216] , Mutual information before and after the operation;

[0217] : Normalization constant, usually taken as the maximum value within the dataset.

[0218] Add auxiliary variables and use relational gating to generate learnable weights:

[0219]

[0220]

[0221] in:

[0222] The baseline weights obtained by mapping from the clinical necessity level are monotonically non-decreasing.

[0223] Clinically essential level, including ;

[0224] , , These are the uncertainty amplification factor, the local image amplification factor, and the mutual information reduction gating factor, respectively.

[0225] Model uncertainty degree;

[0226] Key pattern matching degree, value The larger the value, the better the match;

[0227] Resource / Taboo Situation Factor: >1 when resources are insufficient or taboos exist, ≈1 when resources are sufficient; constant range. ;

[0228] : Weight lower / upper bound.

[0229] Define atomic distance:

[0230]

[0231]

[0232]

[0233] For any editing operation ,(type ):

[0234]

[0235] in:

[0236] Operational costs;

[0237] :according to from The learnable weights derived from it.

[0238] Calculate the edit distance of the dynamically weighted graph , represented as:

[0239]

[0240] in:

[0241] : The set of all possible editing paths;

[0242] A specific editing path;

[0243] The sum of the costs of all operations along the path;

[0244] : Weight coefficients of stage alignment items;

[0245] : Weighting coefficients of causal penalty terms;

[0246] Two time series The dynamic time-warped distance;

[0247] Limit the acceptable range of misalignment;

[0248] :path Triggered causal dependency violation penalty, often written as .

[0249] Further, a consistency score is calculated.

[0250]

[0251] in:

[0252] : No. Wheel image data;

[0253] : No. Round inspection results;

[0254] : No. Wheel text information;

[0255] Pipeline In the The evidence image parsed from the wheel input. This includes steps such as segmentation, extraction, entity linking, and trajectory generation;

[0256] No. The basic consistency score received in turn :

[0257]

[0258] Temperature scaling is applied to obtain a preliminary score:

[0259]

[0260] in This is a pre-scoring result after temperature scaling. This is the temperature coefficient; the larger the value, the lower the sensitivity to changes in cost.

[0261] Then, a sequence-preserving calibration is performed to output the final consistency score:

[0262]

[0263] in This is a sequence-preserving calibration function.

[0264] like Figure 1 As shown, in step S5, based on the difference between the knowledge graph and the first directed graph, the first set of anomalies in the decision-making behavior of the medical AI system to be evaluated is identified and labeled.

[0265] If there is a directed graph G ai The two nodes are directly connected in the middle, while the knowledge graph G std There is an intermediate node marked "MUST" between these two nodes, and this node does not appear in the directed graph G. ai If the path jump occurs, it is determined that a path jump has occurred.

[0266] Set up entity link mapping (Confidence level) Mapping is possible.

[0267]

[0268]

[0269]

[0270] in:

[0271] Unmatched node;

[0272] Knowledge Graph The directed adjacency indicator function, There exists an edge;

[0273] Directed graph The directed adjacency indicator function, There exists an edge;

[0274] , , Knowledge Graph The three nodes in;

[0275] , Directed graph China and Knowledge Graph The corresponding node;

[0276] Select matching knowledge graph If a node has multiple individuals with the highest scores;

[0277] : Jump detection result, belongs to .

[0278] For a directed graph G ai For each clinical behavior node, calculate the cosine similarity of its embedding vectors for both image and text evidence. If the similarity is below a preset threshold... If further examination reveals a serious mismatch between the semantic types of the textual evidence and the corresponding entities in the knowledge graph, then a cross-modal contradiction is determined to exist.

[0279]

[0280]

[0281]

[0282] in:

[0283] , , : Common space embedding vectors corresponding to ROI, text, and entity, respectively;

[0284] : Similarity function, defaults to cosine similarity. ;

[0285] Low similarity threshold, if This is considered "inconsistency between image and text";

[0286] Inconsistency tolerance, if This is considered "inconsistency between text and entity";

[0287] The category mismatch determination function is true if there is a conflict between the text semantic type and the entity ontology type.

[0288] : Cross-modal contradiction flag, a value of 0 indicates no contradiction, and a value of 1 indicates a contradiction.

[0289] Traversing a directed graph G ai In the knowledge graph G, if a certain node is related to a node in the knowledge graph G... std The maximum semantic similarity of any standard node is below the threshold. If removing the node and its edges from the graph does not affect the connectivity of the critical path in the original graph, then the node is considered redundant.

[0290] right ,like:

[0291]

[0292]

[0293] in:

[0294] : Similarity function, defaults to cosine similarity. ;

[0295] Similarity threshold;

[0296] : Indicates the existence of an edge and ;

[0297] From directed graphs Remove node And the graph following its related edges;

[0298] : indicates deletion It still exists Alternative paths;

[0299] Redundancy indicator variable, a value of 1 indicates that redundancy exists.

[0300] All detected anomalies will be categorized and labeled, and associated with specific nodes and edges to generate a location report.

[0301] Final output: A set of annotations and evidence for path jumps, cross-modal contradictions, or redundant nodes.

[0302] like Figure 1 As shown, in step S6, the robustness of the decision-making behavior of the medical AI system to be evaluated is verified, and a structured evaluation report is output.

[0303] Preferably, the medical AI system can instantly reconstruct directed graphs. And refresh the consistency score and local contribution;

[0304] The cost of consensus in a distributed system:

[0305]

[0306] in:

[0307] In the disruptive budget Edit the distance to the target value in the bar chart below;

[0308] Disturbance strength constraint;

[0309] : Meet the budget The family of perturbation transforms;

[0310] The set of probability distributions over the family of perturbation transforms;

[0311] : A specific perturbation transformation obtained from the distribution sampling.

[0312] The final consistency score after perturbation is calculated and expressed as:

[0313]

[0314] in, This is an uncalibrated score under robust conditions. For temperature coefficient, For order-preserving calibration function, The final consistency score after perturbation.

[0315] It can also define robust reduction metrics:

[0316]

[0317] All evaluation results are automatically populated into a structured report template to generate the final evaluation report.

[0318] The assessment report includes, but is not limited to:

[0319] Overview: Consistency score, robustness, case highlights;

[0320] Anomaly list: type, involved nodes or edges, evidence, severity, confidence level;

[0321] Root cause analysis: Critical paths and contribution heatmap;

[0322] Modal alignment: Visualization and weighting of image-text-knowledge hotspots;

[0323] Improvement suggestion: Rules for each type of anomaly + search generation;

[0324] Reproduction information: model version, hyperparameters, timestamps, and data fingerprints.

[0325] Through the implementation of the above six steps, this invention can systematically complete the consistency assessment of the decision-making behavior of a multimodal medical AI system, providing a powerful technical tool for the optimization, verification, and compliance review of AI models.

[0326] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can be implemented in the form of a computer program product on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0327] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Other embodiments may also be used. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating the consistency of decision-making behavior in a multimodal medical AI system, characterized in that, Includes the following steps: S1: Construct a multimodal clinical pathway knowledge graph based on authoritative clinical guidelines and real electronic medical record databases. The knowledge graph is obtained by performing consistency and confidence screening on candidate subgraphs. The candidate subgraphs include clinical pathway nodes encoded with standardized medical terms, temporal dependency edges between nodes, and multimodal edges formed by medical images and their text information associated with the nodes. S2: Input the target medical record into the medical AI system to be evaluated and output the medical image segmentation results, text description and decision path. Parse the output into a first directed graph with nodes and edges. S3: By using a pre-trained multimodal joint coding model, the nodes in the knowledge graph and the nodes in the first directed graph are mapped to the same semantic space to generate corresponding cross-modal embedding vectors. S4: Input the knowledge graph and the directed graph into the graph neural network model, and calculate the comprehensive similarity between the two in the three dimensions of structure, semantics and modality; Based on the comprehensive similarity and the preset cost function, the minimum cost required to edit the first directed graph to be consistent with the knowledge graph is calculated, and a first consistency score is generated based on the minimum cost. S5: Based on the difference between the knowledge graph and the first directed graph, identify and label the first set of anomalies in the decision-making behavior of the medical AI system to be evaluated; S6: Perform robustness verification on the decision-making behavior of the medical AI system to be evaluated, and output a structured evaluation report.

2. The method for evaluating the consistency of decision-making behavior in a multimodal medical AI system according to claim 1, characterized in that, In step S1 The clinical pathway nodes are extracted from the authoritative clinical guidelines, the nodes are encoded using a standard clinical medical terminology system, and temporal dependency edges representing the order between nodes are established. The medical images and their text information associated with the clinical pathway nodes are integrated from the real electronic medical record database. The medical images are identified using a standard radiological terminology system. An image segmentation model is used to identify the regions of interest in the medical images. The regions of interest and their corresponding text information are aligned with the associated clinical pathway nodes to form multimodal edges.

3. The method for evaluating the consistency of decision-making behavior in a multimodal medical AI system according to claim 1, characterized in that, In step S2 Obtain the medical image segmentation results, text descriptions, and decision paths output by the medical AI system to be evaluated for the target medical record; The regions of interest (ROIs) are extracted from the medical image segmentation results and filtered. Standardized descriptive texts are generated for the retained ROIs and standardized medical terminology is encoded. A structured ROI list is then output. The decision path is analyzed, and decision nodes, judgment criteria, and reasoning steps are extracted as nodes. Directed edges are established based on the logical or temporal relationships between the steps to construct an initial directed graph. The text descriptions of the behavioral nodes in the initial directed graph are semantically linked with the standard medical term entities in the knowledge graph. A link relationship and link confidence score pointing to the standard medical term entities are generated for each behavioral node, and a standardized directed graph is output.

4. The method for evaluating the consistency of decision-making behavior in a multimodal medical AI system according to claim 1, characterized in that, In step S3 The nodes in the knowledge graph and the nodes in the directed graph are encoded by the pre-trained multimodal joint coding model, mapped to the public semantic space through a shared projection layer, and then normalized. Contrastive learning is used for training to make the image region embedding vector, text description embedding vector, and knowledge entity embedding vector of the same clinical semantics close to each other in the semantic space.

5. The method for evaluating the consistency of decision-making behavior in a multimodal medical AI system according to claim 1, characterized in that, In step S4 The graph neural network model is a cross-modal hypergraph neural network, which enables the knowledge graph and the nodes of the directed graph to perceive each other during the information transmission process through a cross-graph attention mechanism; The minimum cost required to edit the first directed graph to be consistent with the knowledge graph is calculated using a dynamically weighted graph edit distance algorithm, wherein the cost of the edit operation is determined by structural differences, semantic differences, and modal differences, and the weight of each difference is dynamically adjusted through a learnable gating function based on clinical prior knowledge.

6. The method for evaluating the consistency of decision-making behavior in a multimodal medical AI system according to claim 5, characterized in that, The dynamic weighted graph edit distance algorithm introduces a phase alignment penalty term and a causal chain disruption penalty term; When the order of decision nodes in the first directed graph is misaligned with the expected stage order of the corresponding nodes in the knowledge graph, the stage alignment penalty term is introduced into the graph edit distance according to the degree of misalignment, reflecting the impact of decision stage deviation on consistency evaluation. When the first directed graph contains missing key preceding nodes, reversed causal order, or subsequent decision nodes generated based on unnecessary conditions, the causal chain disruption penalty term is introduced for editing operations that disrupt the integrity of the clinical causal chain, thereby reducing the consistency score of such decision-making behavior.

7. The method for evaluating the consistency of decision-making behavior in a multimodal medical AI system according to claim 1, characterized in that, In step S5 The first set of anomalies includes at least one of path skipping, cross-modal conflict, or redundant nodes; The path jump includes the absence of intermediate nodes marked as necessary in the knowledge graph in the directed graph; The cross-modal contradictions include semantic inconsistencies between image evidence and text evidence associated with the same node in the directed graph; The redundant nodes include nodes in the directed graph that have low semantic similarity to any node in the knowledge graph and are not necessary.

8. The method for evaluating the consistency of decision-making behavior in a multimodal medical AI system according to claim 1, characterized in that, In step S6, The robustness verification includes: The target medical record is perturbed, and at least one set of perturbations is generated and then input. The perturbated input is fed into the medical AI system to be evaluated, and the perturbated medical image segmentation results, text descriptions and decision paths output by the system are obtained and parsed into a second directed graph. Repeat steps S3 to S4 to calculate the second consistency score between the second directed graph and the knowledge graph; The decrease in the first consistency score and the second consistency score is calculated as a robustness quantification indicator. By comparing the second directed graph with the knowledge graph, the newly added or changed difference nodes and edges that cause the score to drop are identified and denoted as the second anomaly set; The structured assessment report includes: The first consistency score, the second consistency score, the first anomaly set, the robustness quantification index, and the second anomaly set.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images