A method and system for explaining medical decision-making mechanisms based on attention mechanisms

Abstract

This application provides a method and system for explaining medical decision-making mechanisms based on attention mechanisms, relating to the field of smart healthcare technology. It addresses the technical problems of existing medical AI models having opaque decision-making processes and being unable to transform model reasoning logic into structured decision-making basis with clinical semantics. The method includes: acquiring multi-source medical data; extracting multimodal features from the multi-source medical data to obtain multimodal feature vectors; inputting the multimodal feature vectors into a pre-trained medical decision-making model, outputting medical decision results and a feature weight matrix; identifying key features of the multimodal feature vectors based on the feature weight matrix to obtain a key feature set; semantically mapping and associating the key feature set based on a pre-set medical knowledge graph to obtain key decision-making basis; and structurally organizing the medical decision results and key decision-making basis to obtain a medical decision explanation report. This application is used in the process of medical decision explanation.

Description

Technical Field

[0001] This application relates to the field of smart healthcare technology, and in particular to a method and system for explaining medical decision-making mechanisms based on attention mechanisms. Background Technology

[0002] With the widespread application of deep learning technology in the medical field, artificial intelligence models, represented by deep neural networks, have demonstrated excellent performance in tasks such as disease diagnosis, risk assessment, and treatment recommendation. However, these models typically have highly complex internal structures and massive parameters, making their decision-making process like a "black box," which is difficult for clinicians to understand and trust. Existing interpretability methods, such as LIME and SHAP, mostly adopt a post-interpretation strategy, that is, generating interpretations separately after the model completes its predictions. This decoupling approach leads to discrepancies between the interpretations and the model's actual reasoning logic, and it cannot incorporate prior medical knowledge during model training. Furthermore, medical data has multi-source heterogeneous characteristics, including multiple modalities such as text medical records, medical images, and time-series vital signs. Most existing methods are designed for single-modal data and lack the ability to deeply integrate and collaboratively interpret multi-modal information. At the same time, the generated interpretations are often presented in the form of feature importance lists or heatmaps, lacking structured clinical semantic support and difficult to directly translate into actionable diagnostic and treatment criteria. Therefore, how to construct a medical decision interpretation method that can embed interpretability constraints during model training, achieve deep fusion of multimodal data, and generate structured clinical decision-making basis based on medical knowledge graphs has become an urgent technical problem to be solved. Summary of the Invention

[0003] This application provides a method and system for explaining medical decision-making mechanisms based on attention mechanisms, which solves the technical problems that existing medical artificial intelligence models are difficult for clinicians to trust due to the lack of transparency in the decision-making process, and cannot transform the model's reasoning logic into structured decision-making basis with clinical semantics.

[0004] To achieve the above objectives, this application adopts the following technical solution: Firstly, a method for explaining medical decision-making mechanisms based on attention mechanisms is provided, including: Acquire multi-source medical data; the multi-source medical data includes medical text data, medical image data, and time-series monitoring data; Multimodal feature extraction is performed on multi-source medical data to obtain multimodal feature vectors; The multimodal feature vectors are input into a pre-trained medical decision-making model, which outputs medical decision results and feature weight matrices; the medical decision-making model is a deep learning model with an embedded attention mechanism. Key features are identified from multimodal feature vectors based on feature weight matrices to obtain a set of key features; Based on a pre-defined medical knowledge graph, semantic mapping and association are performed on key feature sets to obtain key decision-making criteria; The results of medical decisions and key decision-making evidence are structured and organized to obtain a medical decision interpretation report.

[0005] Based on the above technical solutions, this application provides a medical decision-making mechanism explanation method based on attention mechanisms. By constructing an end-to-end interpretability framework, it organically combines multimodal medical data fusion, attention mechanisms, and medical knowledge graphs, achieving a unified medical decision-making process and interpretable output. First, by extracting multimodal features from multi-source medical data, it fully mines complementary information from text, images, and time-series data, providing comprehensive feature support for subsequent decisions. Second, the medical decision-making model embedded with the attention mechanism simultaneously generates a feature weight matrix while outputting decision results, explicitly presenting the model's focus and achieving an integrated design of "reasoning as explanation." Third, it identifies key features based on the feature weight matrix, ensuring that the explanation focuses on the core factors that contribute most to the decision. Furthermore, it performs semantic mapping and association of key features through a medical knowledge graph, transforming the numerical features the model focuses on into clinically meaningful entities and relationships, generating traceable and verifiable decision-making basis. Finally, it forms a structured medical decision explanation report, enabling clinicians to intuitively understand the model's decision logic and the source of its basis. This application effectively enhances the transparency and credibility of the medical AI decision-making process. The generated explanations conform to clinical thinking habits, possess medical semantic support and evidence traceability capabilities, and provide technical support for the reliable application of artificial intelligence in clinical decision support.

[0006] In conjunction with the first aspect above, in one possible implementation, the medical decision-making model includes a feature embedding layer, a location encoding layer, a multi-head self-attention layer, a feedforward network layer, and an output layer; The feature embedding layer is used to map the input multimodal feature vector to a preset latent space to generate a feature sequence; The positional encoding layer is used to add positional encoding information to the feature sequence to generate a positional feature sequence; the positional encoding information is a learnable positional encoding matrix, used to add positional information to each feature in the feature sequence; The multi-head self-attention layer is used to calculate the attention weight between any two positional features in the positional feature sequence and output the feature weight matrix and the enhanced feature representation. The feedforward network layer is used to perform nonlinear transformation on the enhanced feature representation and output the transformed feature representation; The output layer is used to output medical decision results based on the transformation feature representation.

[0007] In conjunction with the first aspect above, in one possible implementation, the multi-head self-attention layer includes multiple parallel attention heads, a multi-head feature fusion unit, and an attention weight aggregation unit; The attention head includes a query linear transformation layer, a key linear transformation layer, a value linear transformation layer, and a scaled dot product attention calculation unit, which is used to calculate the attention weight between any two positional features in the positional feature sequence through regularization, traverse the positional feature sequence, and output the attention weight matrix and the corresponding weighted feature representation. The multi-head feature fusion unit is used to concatenate and fuse the weighted feature representations output by multiple attention heads to obtain a fused feature representation; the fused feature representation is then residually connected to the positional feature sequence and then subjected to layer normalization to obtain an enhanced feature representation; The attention weight aggregation unit is used to average or weightedly fuse the attention weight matrices output by multiple attention heads to obtain a feature weight matrix.

[0008] In conjunction with the first aspect above, in one possible implementation, the training method of the medical decision-making model includes: Obtain a sample medical dataset; wherein the sample medical dataset includes sample feature vectors and corresponding sample decision results; The sample medical dataset is divided into a training set and a validation set according to a preset ratio; Build an initial decision model based on the Transformer architecture; Based on a pre-defined interpretive loss function, the initial decision model is iteratively trained using a training set to obtain a transitional decision model. The medical decision-making model is obtained by evaluating and optimizing the performance of the transitional decision-making model using a validation set.

[0009] In conjunction with the first aspect above, in one possible implementation, the interpretable loss function includes prediction loss, knowledge graph alignment loss, attention sparsity loss, and attention stability loss; The prediction loss, constructed based on cross-entropy loss or mean squared error loss, is used to measure the difference between the decision prediction results output by the medical decision-making model and the sample decision results. The knowledge graph alignment loss, calculated based on KL divergence, is used to constrain the weight prediction results output by the medical decision-making model to align with the prior entity relationships defined in the medical knowledge graph. The attention sparsity loss, constructed based on L1 regularization or entropy regularization, is used to constrain the sparsity of the weight prediction results. The attention stability loss, constructed based on mean squared error or cosine distance, is used to constrain the consistency of weight prediction results for similar samples; the similar samples are any two sample feature vectors whose similarity is less than a preset similarity threshold.

[0010] In conjunction with the first aspect above, in one possible implementation, the construction method of the medical knowledge graph includes: Construct a knowledge graph framework and acquire medical knowledge data; Natural language processing technology is used to extract entity nodes and entity relationships from medical knowledge data to obtain basic knowledge materials. The basic knowledge materials are mapped onto the knowledge graph framework to obtain the initial knowledge graph; The path confidence weights are obtained by calculating the weights of entity relationships in the initial knowledge graph using the GRADE method. The path confidence weights are mapped onto the initial knowledge graph to obtain the medical knowledge graph.

[0011] In conjunction with the first aspect above, in one possible implementation, the semantic mapping and association of key feature sets based on a preset medical knowledge graph includes: Entity recognition is performed on each key feature in the key feature set to obtain the clinical entity set; The clinical entity set is aligned with the standard concept nodes in the medical knowledge graph to obtain the standard entity set; the standard concept nodes are obtained by standardizing the terminology of the entity nodes. The meta-path algorithm is used to mine the association paths of a standard entity set to obtain a chain of decision evidence. By performing path semantic transformation on the chain of decision-making evidence, key decision-making basis can be obtained.

[0012] In conjunction with the first aspect above, in one possible implementation, the step of mining association paths for the standard entity set using a meta-path algorithm includes: Based on a medical knowledge graph, a graph search algorithm is used to perform path search on a standard entity set to obtain an initial path set. The initial path set is matched and connected based on the preset meta-path template to obtain the combined path set; The combined path set is deduplicated using a similarity algorithm to obtain a candidate path set; Based on the path confidence weights in the medical knowledge graph, the confidence score of each candidate path is calculated by the cumulative multiplication method or the minimum value method, and a set of path confidence scores is obtained. Sort the path confidence scores by size, and mark the candidate paths corresponding to the top K path confidence scores as decision evidence chains; where K is a positive integer.

[0013] In conjunction with the first aspect above, in one possible implementation, the multimodal feature extraction of multi-source medical data includes: Entity information is extracted from medical text data using natural language processing techniques to obtain text feature vectors. Deep visual feature extraction is performed on medical image data using a pre-trained convolutional neural network to obtain image feature vectors; Statistical feature extraction of time-series monitoring data is performed using the sliding window technique to obtain a time-series feature vector. Multimodal feature vectors are obtained by aligning and fusing text feature vectors, image feature vectors, and temporal feature vectors through a fully connected layer.

[0014] Secondly, a medical decision-making mechanism explanation system based on attention mechanism is provided, including a data acquisition module, a feature extraction module, a model reasoning module, a feature recognition module, a semantic mapping module, and a report generation module; The data acquisition module is used to acquire multi-source medical data, including medical text data, medical image data, and time-series monitoring data. The feature extraction module is used to extract multimodal features from multi-source medical data to obtain multimodal feature vectors. The model inference module is used to input multimodal feature vectors into a pre-trained medical decision-making model and output medical decision results and feature weight matrices; the medical decision-making model is a deep learning model with an embedded attention mechanism. The feature recognition module is used to identify key features of multimodal feature vectors based on feature weight matrix to obtain a set of key features; The semantic mapping module is used to perform semantic mapping and association on a set of key features based on a preset medical knowledge graph to obtain key decision-making basis; The report generation module is used to structure and organize medical decision results and key decision-making evidence to obtain a medical decision interpretation report.

[0015] This application provides a method and system for interpreting medical decision-making mechanisms based on attention mechanisms. By constructing a multimodal deep learning model embedded with attention mechanisms, it achieves the integrated generation of the medical decision-making process and interpretable output. During the model training phase, a composite loss function is introduced, including knowledge graph alignment loss, attention sparsity loss, and attention stability loss. This effectively guides the model to focus on key features that conform to clinical logic, ensuring that the attention weight distribution has medical semantics and interpretability. Through an innovative design of a multi-head self-attention layer, the multi-head attention weight matrix is ​​directly output as the feature weight matrix, while simultaneously generating enhanced feature representations for decision reasoning, achieving parallel computation of reasoning and interpretation. After identifying key features based on the feature weight matrix, entity alignment and meta-path mining are further performed using a pre-set medical knowledge graph, connecting scattered clinical features into a decision evidence chain that conforms to clinical reasoning logic, generating a structured medical decision interpretation report. This application organically combines multimodal data fusion, attention mechanisms, knowledge graphs, and interpretability constraints, not only improving the predictive accuracy of the medical decision-making model but also making the generated interpretation clinically understandable, evidence traceable, and results verifiable, significantly enhancing clinicians' trust in and willingness to adopt AI-assisted decision-making systems.

[0016] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a medical decision-making mechanism explanation method based on an attention mechanism, provided for an embodiment of this application; Figure 2 A schematic diagram illustrating the construction process of a medical knowledge graph provided in this application embodiment; Figure 3 This is a system architecture diagram of a medical decision-making mechanism explanation system based on attention mechanism, provided for an embodiment of this application. Detailed Implementation

[0018] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0019] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0020] To address the technical problems of existing medical AI models being difficult for clinicians to trust due to their opaque decision-making processes and their inability to transform model reasoning logic into structured decision-making basis with clinical semantics, this application provides a method for explaining medical decision-making mechanisms based on attention mechanisms. This method includes: Acquire multi-source medical data; the multi-source medical data includes medical text data, medical image data, and time-series monitoring data; Multimodal feature extraction is performed on multi-source medical data to obtain multimodal feature vectors; The multimodal feature vectors are input into a pre-trained medical decision-making model, which outputs medical decision results and feature weight matrices; the medical decision-making model is a deep learning model with an embedded attention mechanism. Key features are identified from multimodal feature vectors based on feature weight matrices to obtain a set of key features; Based on a pre-defined medical knowledge graph, semantic mapping and association are performed on key feature sets to obtain key decision-making criteria; The results of medical decisions and key decision-making evidence are structured and organized to obtain a medical decision interpretation report.

[0021] Based on this, the technical problems of existing medical artificial intelligence models being difficult for clinicians to trust due to the lack of transparency in the decision-making process, and the inability to transform the model's reasoning logic into structured decision-making basis with clinical semantics, are solved.

[0022] like Figure 1 As shown in the embodiment of this application, a method for explaining medical decision-making mechanisms based on attention mechanisms is provided, including: S201. Obtain multi-source medical data.

[0023] The multi-source medical data includes medical text data, medical image data, and time-series monitoring data.

[0024] For example, the hospital information platform connects to multiple business systems to obtain multi-source medical data. Medical text data is collected from HIS and EMR systems via the HL7 FHIR interface, including unstructured documents such as outpatient records, admission records, progress notes, and discharge summaries, as well as structured medical orders and diagnostic information. More than 5,000 text records are collected daily, with each record containing an average of 800 to 1,200 Chinese characters. Medical imaging data is acquired via the PACS system using the DICOM protocol, including CT, MRI, and X-ray images. During CT scans, the DICOM Modality Worklist service automatically retrieves the patient's examination list. A single chest CT scan generates 300 to 500 tomographic images, each with a resolution of 512 x 512 pixels and a grayscale depth of 4096 levels. Approximately 200 GB of new image data is added daily. Time-series monitoring data is collected via GE CARESCAPE. The Gateway collects continuous vital sign waveforms and data streams from bedside monitors, encapsulating them in XML format using the HL7 protocol. It collects parameters such as body temperature, heart rate, respiratory rate, and blood oxygen saturation at a sampling frequency of 1 second. Continuous 24-hour monitoring of a single patient generates over 86,400 raw records. The system supports simultaneous access from up to 512 monitoring devices, collecting over 40 million time-series data points daily. All collected multi-source data are linked and integrated according to the patient's unique identifier, constructing a complete patient dataset including text, images, and time-series modalities. A data cleaning module removes duplicate and abnormal records to ensure data quality.

[0025] In this embodiment, by integrating existing medical information systems and standardized acquisition protocols, the automated and large-scale acquisition of multi-source medical data is achieved, covering the entire process of patient diagnosis and treatment information from clinical documents and medical images to real-time vital signs. This provides a comprehensive, accurate, and traceable data foundation for subsequent multimodal feature extraction and medical decision-making model training, significantly improving the efficiency and consistency of data acquisition.

[0026] S202. Perform multimodal feature extraction on multi-source medical data to obtain multimodal feature vectors.

[0027] In some implementations, the multimodal feature extraction of multi-source medical data includes: Entity information is extracted from medical text data using natural language processing techniques to obtain text feature vectors. Deep visual feature extraction is performed on medical image data using a pre-trained convolutional neural network to obtain image feature vectors; Statistical feature extraction of time-series monitoring data is performed using the sliding window technique to obtain a time-series feature vector. Multimodal feature vectors are obtained by aligning and fusing text feature vectors, image feature vectors, and temporal feature vectors through a fully connected layer.

[0028] For example, firstly, entity information is extracted from medical text data using natural language processing techniques. A named entity recognition model is constructed using a pre-trained language model based on the Transformer architecture, ERNIE-Health, combined with a bidirectional long short-term memory network and a conditional random field. This model identifies entities such as diseases, symptoms, examinations, and medications from admission records and medical records, mapping them to 300-dimensional word vectors. These vectors are then aggregated through an attention mechanism to form a 1024-dimensional text feature vector. For medical image data, a ResNet152 convolutional neural network pre-trained on ImageNet is used to extract deep visual features. 300 to 500 tomographic images from a single chest CT scan are scaled to 224 x 224 pixels and input into the network. Global average pooling is then performed on each image feature to obtain a 2048-dimensional image feature vector. Statistical features were extracted from time-series monitoring data using a sliding window technique. The window size was set to 60 minutes, and the sliding step size to 15 minutes. Forty-eight statistical parameters, including mean, standard deviation, maximum, minimum, slope, and quartiles, were calculated for parameters such as body temperature, heart rate, respiratory rate, and blood oxygen saturation within the window. Continuous monitoring of a single patient over 24 hours generated over 1400 time-series records, with each window's features forming a 960-dimensional time-series feature vector. Finally, the text, image, and time-series feature vectors were mapped to a 512-dimensional common space through fully connected layers for feature alignment. After concatenation, an initial 1536-dimensional fused feature vector was obtained. This was then subjected to nonlinear transformation and dimensionality reduction through two fully connected layers, outputting the final 512-dimensional multimodal feature vector.

[0029] In this embodiment, heterogeneous medical data is transformed into a unified feature vector. Text features retain key clinical entities and their semantic relationships in medical records, image features capture morphological details and spatial distribution patterns of lesions, and temporal features reflect the dynamic evolution of patients' physiological parameters. Feature alignment and fusion eliminate the semantic gap between modalities, enabling subsequent medical decision-making models to make comprehensive judgments using multi-dimensional information. The generated comprehensive feature vector effectively supports the performance improvement of tasks such as disease screening, assisted diagnosis, and risk assessment, laying a data foundation for building an accurate and reliable intelligent medical decision-making system.

[0030] S203. Input the multimodal feature vectors into the pre-trained medical decision-making model and output the medical decision-making results and feature weight matrix.

[0031] The medical decision-making model is a deep learning model with an embedded attention mechanism.

[0032] In some implementations, the medical decision-making model includes a feature embedding layer, a location encoding layer, a multi-head self-attention layer, a feedforward network layer, and an output layer; The feature embedding layer is used to map the input multimodal feature vector to a preset latent space to generate a feature sequence; The positional encoding layer is used to add positional encoding information to the feature sequence to generate a positional feature sequence; the positional encoding information is a learnable positional encoding matrix, used to add positional information to each feature in the feature sequence; The multi-head self-attention layer is used to calculate the attention weight between any two positional features in the positional feature sequence and output the feature weight matrix and the enhanced feature representation. The feedforward network layer is used to perform nonlinear transformation on the enhanced feature representation and output the transformed feature representation; The output layer is used to output medical decision results based on the transformation feature representation.

[0033] It should be noted that the feature embedding layer includes independent embedding layers for different modal data and learnable modality fusion weights.

[0034] For example, the input 768-dimensional multimodal feature vector is composed of 256-dimensional text features, 256-dimensional image features, and 256-dimensional temporal features. The feature embedding layer sets up an independent linear embedding layer for each modality, mapping the 256-dimensional features of each modality to a 128-dimensional latent space. Simultaneously, to further extract local details, the 256-dimensional features of each modality are divided into 8 feature blocks using a 32-dimensional window, for a total of 24 feature blocks. Each feature block is mapped to a 128-dimensional vector through its respective modality's independent embedding layer, forming a 24×128 initial feature sequence. The feature embedding layer also includes a learnable 24×24 modality fusion weight matrix, used to initialize the cross-modal interaction bias of the subsequent attention layer. The positional encoding layer introduces a learnable 24×128 positional encoding matrix, which is added to the feature sequence to obtain the positional feature sequence. The multi-head self-attention layer contains eight parallel attention heads, each mapping a 128-dimensional input to a 64-dimensional query, key, and value matrix. Scaling dot-product attention is then performed to obtain eight 24×24 attention weight matrices and eight 24×64 weighted feature representations. The multi-head feature fusion unit concatenates the eight head outputs to a 24×512 matrix, compresses it back to 24×128, connects it to the original feature residuals, and performs layer normalization to output an enhanced feature representation. Simultaneously, the attention weight aggregation unit averages and fuses the eight head attention weight matrices to obtain the final 24×24 feature weight matrix. The feedforward network layer consists of two fully connected layers with a 512-dimensional intermediate dimension, using GELU activation. After a non-linear transformation of the enhanced feature representation, it remains 24×128. The output layer performs global average pooling on the 24 features to obtain a 128-dimensional vector, which is then mapped to a 4-dimensional probability vector via a fully connected layer, outputting the decision results for four types of diseases, including community-acquired pneumonia.

[0035] In this embodiment, the semantic specificity of text, images, and time-series data is fully preserved through independent modal mapping and learnable fusion weights in the feature embedding layer, while providing prior biases for cross-modal interaction. A multi-head self-attention mechanism captures multiple clinical relationships in parallel, and the attention weight matrix is ​​directly used as an interpretable output, realizing the integration of reasoning and interpretation. Residual connections and layer normalization ensure the stability of deep network training, and the feedforward network enhances nonlinear expressive ability. The output layer combines global pooling to generate the final decision, so that the model not only predicts accurately, but its attention weight matrix can also accurately reflect the contribution of each modality feature to the decision, providing a high-quality interpretable foundation for subsequent key feature identification and decision basis generation.

[0036] In some implementations, the multi-head self-attention layer includes multiple parallel attention heads, a multi-head feature fusion unit, and an attention weight aggregation unit; The attention head includes a query linear transformation layer, a key linear transformation layer, a value linear transformation layer, and a scaled dot product attention calculation unit, which is used to calculate the attention weight between any two positional features in the positional feature sequence through regularization, traverse the positional feature sequence, and output the attention weight matrix and the corresponding weighted feature representation. The multi-head feature fusion unit is used to concatenate and fuse the weighted feature representations output by multiple attention heads to obtain a fused feature representation; the fused feature representation is then residually connected to the positional feature sequence and then subjected to layer normalization to obtain an enhanced feature representation; The attention weight aggregation unit is used to average or weightedly fuse the attention weight matrices output by multiple attention heads to obtain a feature weight matrix.

[0037] It should be noted that the query linear transformation layer is used to map the input positional feature sequence to a query matrix, the key linear transformation layer is used to map the input positional feature sequence to a key matrix, and the value linear transformation layer is used to map the input positional feature sequence to a value matrix. The scaled dot product attention calculation unit is used to calculate a similarity matrix based on the query matrix and the key matrix, scale the similarity matrix and normalize it using the Softmax function to obtain the attention weight matrix of the attention head, and multiply the attention weight matrix with the value matrix to obtain the weighted feature representation of the attention head.

[0038] For example, the input positional feature sequence is 24×128 dimensional, corresponding to 24 feature blocks, each 128 dimensional. Eight parallel attention heads are set up. Internally, each head maps the 128-dimensional input to a 64-dimensional query matrix through a query linear transformation layer, a key linear transformation layer to a 64-dimensional key matrix, and a value linear transformation layer to a 64-dimensional value matrix, resulting in eight sets of 24×64 query, key, and value matrices. A scaling dot product attention calculation unit performs a dot product of each query and the transpose of the key matrix to obtain a 24×24 similarity matrix. After dividing by a scaling factor of 8, it is normalized using Softmax to obtain the attention weight matrix for that head. This weight matrix is ​​then multiplied by the value matrix to obtain the 24×64 weighted feature representation for that head. For a preset clinical relationship type, the first and second attention heads are guided to focus on symptom-disease relationships through regularization. During training, the KL divergence is calculated as a regularization term based on the co-occurrence matrix of 180 symptoms and 120 diseases in the medical knowledge graph. The multi-head feature fusion unit concatenates the 24×64 weighted feature representations output by the eight attention heads into a 24×512 representation, compresses it back to 24×128 through a linear fusion layer to obtain the fused feature representation, and then performs residual concatenation with the original 24×128 positional feature sequence before normalization to output a 24×128 enhanced feature representation. The attention weight aggregation unit averages and fuses the 24×24 attention weight matrices output by the eight attention heads to obtain the final 24×24 feature weight matrix.

[0039] In this embodiment, the multi-head self-attention layer, through parallel attention head design, enables the model to simultaneously capture multiple clinical semantic relationships such as symptom-disease and examination-diagnosis. Some heads are guided by regularization to align with medical prior knowledge, effectively improving the clinical interpretability of attention weights. The multi-head feature fusion unit integrates the diverse information captured by each head, residual connections ensure the stability of the gradient of the deep network, and layer normalization accelerates convergence. The attention weight aggregation unit fuses the multi-head information into a single feature weight matrix, which not only preserves the richness of multi-angle attention information but also provides a unified weight basis for subsequent key feature recognition, realizing the integrated output of reasoning and interpretation.

[0040] In some implementations, the training method of the medical decision-making model includes: Obtain a sample medical dataset; wherein the sample medical dataset includes sample feature vectors and corresponding sample decision results; The sample medical dataset is divided into a training set and a validation set according to a preset ratio; Build an initial decision model based on the Transformer architecture; Based on a pre-defined interpretive loss function, the initial decision model is iteratively trained using a training set to obtain a transitional decision model. The medical decision-making model is obtained by evaluating and optimizing the performance of the transitional decision-making model using a validation set.

[0041] For example, a sample medical dataset is first obtained. This dataset consists of 12,800 respiratory infection patients admitted to the respiratory department of a tertiary hospital from 2021 to 2023. Each sample contains a 768-dimensional multimodal feature vector and a clinically confirmed sample decision result, including 4,320 cases of community-acquired pneumonia, 3,860 cases of influenza A, 2,710 cases of influenza B, and 1,910 cases of novel coronavirus infection. The sample dataset is divided into a training set of 10,240 samples and a validation set of 2,560 samples in an 8:2 ratio. An initial decision model is built based on the Transformer architecture, with a feature embedding dimension of 256, 8 attention heads, a feedforward network intermediate layer dimension of 1024, and 6 encoder layers. The parameters are initialized using Xavier. Training is performed based on a pre-defined interpretable loss function, which is composed of a weighted average of prediction loss, knowledge graph alignment loss, attention sparsity loss, and attention stability loss, with weight coefficients of 1.0, 0.3, 0.1, and 0.2, respectively. Iterative training is performed using the AdamW optimizer with an initial learning rate of 3e. -4 Weight decay 1e -4The batch size is 32, and the maximum number of training epochs is 50. After each epoch, the loss is evaluated on the validation set. Training is terminated early if the loss does not decrease after 5 consecutive epochs, and the optimal model on the validation set is saved as an interim decision model. Based on the validation set performance, the number of attention heads is adjusted to 10, the number of encoder layers to 8, and the learning rate to 2e. -4 The model was then retrained to obtain the final medical decision-making model.

[0042] In this embodiment, the model's attention is guided to align with medical priors through knowledge graph alignment loss, sparsity loss encourages attention to key features, and stability loss enhances interpretability, thus significantly improving interpretability while ensuring prediction accuracy. The early stopping strategy prevents overfitting, validation set evaluation ensures generalization ability, and hyperparameter tuning further improves the model's identification performance in various respiratory infectious diseases, providing a medical artificial intelligence model with accurate prediction and transparent decision-making for clinical deployment.

[0043] In some implementations, the interpretable loss function includes prediction loss, knowledge graph alignment loss, attention sparsity loss, and attention stability loss; The prediction loss, constructed based on cross-entropy loss or mean squared error loss, is used to measure the difference between the decision prediction results output by the medical decision-making model and the sample decision results. The knowledge graph alignment loss, calculated based on KL divergence, is used to constrain the weight prediction results output by the medical decision-making model to align with the prior entity relationships defined in the medical knowledge graph. The attention sparsity loss, constructed based on L1 regularization or entropy regularization, is used to constrain the sparsity of the weight prediction results. The attention stability loss, constructed based on mean squared error or cosine distance, is used to constrain the consistency of weight prediction results for similar samples; the similar samples are any two sample feature vectors whose similarity is less than a preset similarity threshold.

[0044] For example, the prediction loss uses the cross-entropy loss function to calculate the difference between the probability distribution output by the model and the one-hot vector of the true label. The knowledge graph alignment loss is constructed based on KL divergence. First, a disease-symptom prior relation matrix is ​​extracted from the medical knowledge graph. This matrix is ​​4 rows and 180 columns, representing the association strength between 4 diseases and 180 symptoms. After row normalization, the prior distribution is obtained. For each training sample, the corresponding prior distribution row vector is selected according to its true disease label. At the same time, the attention weight subset corresponding to the symptom feature block is extracted from the feature weight matrix output by the model and normalized. The KL divergence between the two is calculated as the alignment loss. The attention sparsity loss uses L1 regularization. The average sparsity loss is obtained by directly summing the absolute values ​​of all elements in the feature weight matrix and dividing by 576, i.e., 24×24. The attention stability loss is constructed based on mean squared error. For each training sample, two augmented samples are generated using a random masking method, requiring the masking ratio to not exceed 15% while ensuring the preservation of key features. The mean squared error between the feature weight matrices output by the two augmented samples after inputting them into the model is calculated. The average mean squared error of all samples within the same batch is taken as the stability loss. The four losses are weighted and summed according to their weight coefficients: prediction loss weight 1.0, knowledge graph alignment loss weight 0.3, attention sparsity loss weight 0.1, and attention stability loss weight 0.2. The total loss is then used for backpropagation to update the model parameters.

[0045] In this embodiment, the knowledge graph alignment loss guides the model's attention weight distribution to align with medical prior knowledge, enabling the model to actively focus on disease-related symptom features during diagnosis, effectively improving the clinical interpretability of attention weights. The attention sparsity loss prompts the model to focus on a few key feature blocks, avoiding attention dispersion and making the generated interpretations more concise and clear. The attention stability loss enhances the consistency of the attention distribution of samples by constraining them, improving the model's robustness to input noise and ensuring that similar cases receive similar attention interpretations. The prediction loss guarantees the model's basic classification performance. The synergistic effect of these four losses ensures that the trained medical decision-making model not only has high accuracy but also that its attention weight matrix stably, sparsely, and logically reflects the decision-making basis, providing a high-quality interpretable foundation for subsequent key feature identification and decision-making basis generation.

[0046] S204. Based on the feature weight matrix, key features are identified from the multimodal feature vectors to obtain a set of key features.

[0047] For example, taking a patient with community-acquired pneumonia as an example, the feature weight matrix output by the medical decision-making model is 24×24 dimensional, corresponding to the attention intensity among 24 feature blocks, including 8 text feature blocks, 8 image feature blocks, and 8 time-series feature blocks. First, the feature weight matrix is ​​summed row-wise to obtain the global importance score of each feature block. The score of the i-th feature block is the sum of the 24 attention weight values ​​in that row. After calculating the 24 importance scores, a quicksort algorithm is used to sort them from highest to lowest score, and the top 8 feature blocks are selected as key feature blocks. The sorting results are as follows: Text block 3 corresponds to a persistent high fever feature score of 2.34; text block 7 corresponds to a cough with yellow sputum feature score of 2.18; image block 5 corresponds to right lower lung consolidation feature score of 1.96; time sequence block 2 corresponds to the rate of temperature rise feature score of 1.87; text block 9 corresponds to dyspnea feature score of 1.79; image block 12 corresponds to air bronchogram feature score of 1.68; time sequence block 4 corresponds to the trend of decreasing blood oxygen saturation feature score of 1.55; and image block 15 corresponds to pleural effusion feature score of 1.43. Mapping these 8 key feature blocks back to the original data space, text block 3 originates from the admission record's chief complaint of persistent high fever for 5 days, image block 5 originates from the right lower lung lobe region in CT images, and time sequence block 2 originates from the temperature monitoring curve showing a 1.8 degree Celsius increase over the past 6 hours, ultimately yielding a key feature set containing 8 key features and their original descriptions.

[0048] In this embodiment, the global importance score is calculated by summing the rows of the feature weight matrix, effectively measuring the contribution of each feature block to the model decision-making process. The score ranking mechanism ensures the objectivity and repeatability of key feature selection. The top 8 feature blocks cover the patient's main clinical manifestations, imaging changes, and physiological parameter variations. The differences in score distribution intuitively reflect the importance level of each feature to the final diagnosis. Mapping key feature blocks back to the original data space establishes a connection between abstract attention weights and specific clinical descriptions, providing directly alignable clinical entity material for subsequent semantic mapping based on medical knowledge graphs.

[0049] S205. Based on the preset medical knowledge graph, perform semantic mapping and association on the key feature set to obtain key decision-making basis.

[0050] In some implementations, the construction method of the medical knowledge graph is as follows: Figure 2 As shown, it includes: S61. Construct a knowledge graph framework and acquire medical knowledge data; S62. Extract entity nodes and entity relationships from medical knowledge data using natural language processing technology to obtain basic knowledge materials; S63. Map the basic knowledge materials onto the knowledge graph framework to obtain the initial knowledge graph; S64. Calculate the weights of entity relationships in the initial knowledge graph using the GRADE method to obtain path confidence weights; S65. Map the path confidence weights onto the initial knowledge graph to obtain the medical knowledge graph.

[0051] It should be noted that the path confidence weight is calculated by combining the evidence level and the timeliness factor. The evidence level refers to the classification of knowledge sources into different levels using the GRADE method or other evidence grading systems. Since the timeliness of knowledge affects its current credibility, the reciprocal of the number of years that the medical knowledge is from the current year is defined as the timeliness factor.

[0052] For example, entity types are first defined as including six major categories: disease, symptom, examination, drug, pathogen, and transmission route; relationship types include 12 types such as manifestation, diagnostic applicability, and treatment applicability. Medical knowledge data was obtained from 126 treatment guidelines downloaded from the official website of the National Health Commission, 89 expert consensus articles published in Chinese Medical Association journals, 34 international guidelines obtained from the WHO official website, and 215 systematic reviews retrieved from CNKI and PubMed, spanning from 2015 to 2024. An ERNIE-Health-based joint extraction framework was used for entity and relationship extraction. After fine-tuning on 20,000 manually annotated corpora, the model achieved an F1 score of 0.92 for entity recognition and 0.87 for relationship extraction. From the above data, 385 disease entities, 572 symptom entities, 246 examination entities, 418 drug entities, 97 pathogen entities, 23 transmission route entities, and 47,000 entity relationships were extracted. The extracted results were aligned to the standard nodes of the Chinese Clinical Medical Terminology System (CCST) through string matching and semantic similarity, constructing an initial knowledge graph containing 1568 entity nodes and 47,000 relation edges. The GRADE method was used to classify each relation into levels of evidence: high-level (0.8), medium-level (0.6), low-level (0.4), and very low-level (0.2). The timeliness factor was calculated as 1 divided by 1 plus the number of years since the current year; for example, the timeliness factor for guidelines published in 2023 was 0.33. The path confidence weight was obtained by multiplying the evidence level weight by 0.7 and adding the timeliness factor by 0.3. For example, the relation weight for the use of oseltamivir in influenza in the 2023 high-level guidelines was 0.8 × 0.7 + 0.33 × 0.3 = 0.66. The calculated path confidence weights were added as attributes to each relation edge, and the final medical knowledge graph was stored in the Neo4j graph database.

[0053] In this embodiment, the comprehensiveness and accuracy of knowledge are ensured through multi-source authoritative data collection and advanced natural language processing technology, covering the core areas of respiratory infectious disease prevention and control. The GRADE method assigns a scientific weight based on the level of evidence to each relation, the timeliness factor enables the graph to dynamically reflect the latest developments in medical knowledge, and the quantification of path confidence weights allows subsequent meta-path mining to filter evidence chains based on credibility, avoiding interference from low-quality or outdated knowledge in decision-making. This provides an authoritative, credible, and quantifiable knowledge foundation for model alignment loss calculation and decision basis generation.

[0054] In some implementations, the semantic mapping and association of key feature sets based on a preset medical knowledge graph includes: Entity recognition is performed on each key feature in the key feature set to obtain the clinical entity set; The clinical entity set is aligned with the standard concept nodes in the medical knowledge graph to obtain the standard entity set; the standard concept nodes are obtained by standardizing the terminology of the entity nodes. The meta-path algorithm is used to mine the association paths of a standard entity set to obtain a chain of decision evidence. By performing path semantic transformation on the chain of decision-making evidence, key decision-making basis can be obtained.

[0055] For example, taking a patient with community-acquired pneumonia as an example, the key feature set includes five key features: persistent high fever, cough with yellow sputum, consolidation in the right lower lung field, dyspnea, and decreased blood oxygen saturation. First, entity recognition is performed using a named entity recognition model based on ERNIE-Health. After fine-tuning on 8600 manually annotated clinical texts, the model achieved an F1 score of 0.93, identifying six clinical entities from the above features: high fever, cough, yellow sputum, pulmonary consolidation, dyspnea, and hypoxemia. The clinical entity set is then aligned with standard concept nodes in a medical knowledge graph. A similarity threshold of 0.85 is set using string matching combined with BERT semantic similarity calculation. High fever is aligned to fever, cough to cough, yellow sputum to sputum, pulmonary consolidation to pulmonary consolidation, dyspnea to dyspnea, and hypoxemia to hypoxemia, resulting in a standard entity set containing these six standard entities. Meta-path algorithms were used to mine association paths in a standard entity set. A bidirectional breadth-first search with a depth of 3 hops was employed in the medical knowledge graph. After meta-path template matching and confidence ranking, three decision evidence chains were obtained. Semantic transformation of these decision evidence chains was performed: Fever → Community-acquired pneumonia was transformed into "Persistent high fever is a common symptom of community-acquired pneumonia" (confidence 0.85); Cough and sputum → Community-acquired pneumonia was transformed into "Cough with yellow sputum is consistent with typical manifestations of community-acquired pneumonia" (confidence 0.82); and Pulmonary consolidation → Community-acquired pneumonia → Hypoxemia was transformed into "Pulmonary consolidation in the right lower lung suggests community-acquired pneumonia and decreased blood oxygenation may be complicated by hypoxemia" (confidence 0.49).

[0056] In this embodiment, entity recognition transforms key features of interest to the model into computable clinical entities. Entity alignment eliminates the diversity of clinical expressions, unifying features from different sources under a standard medical concept system. The meta-path algorithm mines the associated paths connecting multiple entities in the knowledge graph, linking scattered key features into a complete chain of evidence that conforms to clinical reasoning logic. The semantically transformed natural language description enables doctors to directly understand the model's decision-making logic, significantly improving the clinical acceptability and trustworthiness of medical decision interpretation.

[0057] In some implementations, the step of mining association paths for the standard entity set using a meta-path algorithm includes: Based on a medical knowledge graph, a graph search algorithm is used to perform path search on a standard entity set to obtain an initial path set. The initial path set is matched and connected based on the preset meta-path template to obtain the combined path set; The combined path set is deduplicated using a similarity algorithm to obtain a candidate path set; Based on the path confidence weights in the medical knowledge graph, the confidence score of each candidate path is calculated by the cumulative multiplication method or the minimum value method, and a set of path confidence scores is obtained. Sort the path confidence scores by size, and mark the candidate paths corresponding to the top K path confidence scores as decision evidence chains; where K is a positive integer.

[0058] For example, taking a patient with community-acquired pneumonia as an example, the standard entity set includes nine entities: fever, cough, pulmonary consolidation, hypoxemia, dyspnea, sputum production, air bronchogram, pleural effusion, and elevated white blood cell count. First, a bidirectional breadth-first search algorithm is used to search for paths for each entity in the medical knowledge graph stored in Neo4j. The maximum search depth is set to 3 hops, and the time limit is 500 milliseconds. Starting from the nine entities, a total of 128 initial paths are obtained. Matching and connection are then performed based on eight preset meta-path templates, including symptom-disease, symptom-disease-examination, and disease-complication, resulting in 42 combined paths after matching and filtering. The Jaccard similarity algorithm is then used to deduplicate the combined paths. A deduplication threshold of 0.7 is set; paths with a similarity higher than 0.7 are considered redundant, and only the one with higher confidence is retained. After deduplication, 23 candidate paths are obtained. Based on the path confidence weights of each relation edge in the medical knowledge graph, a cumulative multiplication method is used to calculate the confidence score of each candidate path. For example, the path "Fever → Community-acquired Pneumonia" contains one relation edge with a weight of 0.85, resulting in a score of 0.85. The path "Pulmonary Consolidation → Community-acquired Pneumonia → Hypoxemia" contains three relation edges with weights of 0.82, 0.79, and 0.75, resulting in a cumulative score of 0.49. All 23 candidate paths are sorted from highest to lowest confidence score, and the top 3 paths are selected as the decision evidence chains: "Fever → Community-acquired Pneumonia" (0.85), "Cough → Community-acquired Pneumonia" (0.81), and "Pulmonary Consolidation → Community-acquired Pneumonia → Hypoxemia" (0.49).

[0059] In this embodiment, a bidirectional breadth-first search is used to efficiently traverse the graph, ensuring the discovery of potential associations within milliseconds. Meta-path template matching filters out random paths without clinical significance, retaining candidate paths that conform to medical logic. Jaccard deduplication eliminates semantic redundancy, avoiding repeated output of multiple similar paths. The cumulative multiplication confidence calculation fully considers the evidence level and timeliness of each relation edge on the path, giving priority to high-confidence paths. The final ranking selects the top K paths as the decision evidence chain, ensuring that the explanations provided to doctors are concise and reliable, and enabling scattered standard entities to be linked into a complete clinical reasoning chain through graph semantic relationships.

[0060] S206. The results of medical decisions and key decision-making basis are organized in a structured manner to obtain a medical decision interpretation report.

[0061] For example, taking a patient with community-acquired pneumonia as an example, the medical decision-making model outputs a 4-dimensional probability vector: community-acquired pneumonia probability 0.87, influenza A probability 0.06, influenza B probability 0.04, and novel coronavirus infection probability 0.03. Monte Carlo Dropout sampling for 20 iterations yields a 95% confidence interval of 0.82 to 0.91. Key feature identification yields 8 features and their importance scores: persistent high fever 2.34, cough with yellow sputum 2.18, right lower lung consolidation 1.96, rate of temperature rise 1.87, dyspnea 1.79, air bronchogram 1.68, decreased blood oxygen saturation 1.55, and pleural effusion 1.43. The key decision-making criteria generated three chains of evidence: fever → community-acquired pneumonia (confidence 0.85), corresponding to the description that persistent high fever is a common symptom; cough and sputum → community-acquired pneumonia (confidence 0.82), corresponding to the description that cough with yellow sputum is consistent with typical presentation; pulmonary consolidation → community-acquired pneumonia → hypoxemia (confidence 0.49), corresponding to the description that consolidation in the right lower lung suggests disease and decreased blood oxygenation may be complicated by hypoxemia. The structured report was organized in JSON format, including patient anonymity identifier, consultation time, primary diagnosis and probability confidence interval, key feature tables linked to the original data, hierarchical display of the evidence chains including semantic descriptions, confidence levels, and graph paths, and ECharts generated feature importance bar charts and directed graphs of the evidence chains. The final output included interactive PDF and HTML versions for doctors to view and provide feedback.

[0062] In this embodiment, numerical results, key features, and evidence chains are organically integrated to form a well-structured and comprehensive medical decision interpretation report. The decision results provide diagnostic probabilities and confidence intervals, enabling doctors to assess the reliability of predictions. Key features transform attention weights into understandable clinical descriptions and link them to original data to ensure traceability of evidence. The evidence chain is transformed into natural language descriptions with accompanying confidence levels and source citations, making the reasoning logic verifiable. Visual charts intuitively present the importance of features and the evidence path, enhancing report readability. This provides doctors with a complete information chain from prediction to basis to original data, making the medical AI decision-making process transparent, explainable, and verifiable.

[0063] Based on the above technical solutions, this application provides a medical decision-making mechanism explanation method based on attention mechanisms. By constructing a multimodal deep learning model embedded with attention mechanisms, it achieves the integrated generation of the medical decision-making process and interpretable output. During the model training phase, a composite loss function is introduced, including knowledge graph alignment loss, attention sparsity loss, and attention stability loss. This effectively guides the model to focus on key features that conform to clinical logic, ensuring that the attention weight distribution has medical semantics and interpretability. Through the innovative design of a multi-head self-attention layer, the multi-head attention weight matrix is ​​directly output as the feature weight matrix, while simultaneously generating enhanced feature representations for decision reasoning, achieving parallel computation of reasoning and interpretation. After identifying key features based on the feature weight matrix, a pre-set medical knowledge graph is further used for entity alignment and meta-path mining, connecting scattered clinical features into a decision evidence chain that conforms to clinical reasoning logic, generating a structured medical decision explanation report. This application organically combines multimodal data fusion, attention mechanisms, knowledge graphs, and interpretability constraints, which not only improves the predictive accuracy of medical decision-making models, but also makes the generated explanations clinically understandable, evidence traceable, and results verifiable, significantly enhancing clinicians' trust in and willingness to adopt AI-assisted decision-making systems.

[0064] In one possible implementation, this application embodiment also provides a medical decision-making mechanism explanation system 100 based on an attention mechanism, such as... Figure 3 As shown, the system 100 includes a data acquisition module 10, a feature extraction module 20, a model inference module 30, a feature recognition module 40, a semantic mapping module 50, and a report generation module 60; The data acquisition module 10 is used to acquire multi-source medical data, including medical text data, medical image data, and time-series monitoring data. The feature extraction module 20 is used to extract multimodal features from multi-source medical data to obtain multimodal feature vectors; The model inference module 30 is used to input multimodal feature vectors into a pre-trained medical decision-making model and output medical decision results and feature weight matrix; the medical decision-making model is a deep learning model with an embedded attention mechanism. The feature recognition module 40 is used to perform key feature recognition on the multimodal feature vector based on the feature weight matrix to obtain a key feature set; The semantic mapping module 50 is used to perform semantic mapping and association on a set of key features based on a preset medical knowledge graph to obtain key decision-making basis; The report generation module 60 is used to structure and organize the medical decision results and key decision-making basis to obtain a medical decision interpretation report.

[0065] For example, the system is deployed in the data center of a tertiary hospital. The data acquisition module 10 connects to the hospital's HIS, PACS, and monitoring systems. Using the HL7 FHIR interface, it collects over 5,000 pieces of medical text data daily, including admission records and progress notes. It also collects approximately 200GB of CT images daily via the DICOM protocol, containing 512x512 pixel tomographic images. Furthermore, it collects over 40 million points of time-series data daily, such as body temperature and heart rate, via the GECARESCAPE Gateway at a sampling frequency of 1 second. The feature extraction module 20 uses the ERNIE-Health model to identify entities in the text data and maps them to 256-dimensional feature vectors. For the image data, it uses ResNet152 to extract 2048-dimensional features. For the time-series data, it uses a sliding window with a 60-minute step size and a 15-minute interval to extract 960-dimensional statistical features. After alignment and fusion by a fully connected layer, it outputs a 768-dimensional multimodal feature vector. The model inference module 30 inputs 768-dimensional features into a pre-trained medical decision-making model. This model, based on the Transformer architecture, contains 8 attention heads and 6 layers of encoders, outputting the probabilities of 4 types of diseases, including community-acquired pneumonia, and a 24×24 feature weight matrix. The feature recognition module 40 sorts the feature weight matrix by row summation and selects the top 8 feature blocks as the key feature set. For example, key features of a pneumonia patient include a score of 2.34 for persistent high fever and a score of 1.96 for consolidation in the right lower lung. The semantic mapping module 50, based on a medical knowledge graph containing 1568 entity nodes and 47,000 relation edges, uses a meta-path algorithm to perform entity recognition alignment and path mining on the key features, obtaining 3 decision evidence chains, such as fever → community-acquired pneumonia with a confidence level of 0.85. The report generation module 60 integrates the decision outcome probability of 0.87 and confidence interval of 0.82 to 0.91, the list of key features and the semantic description of the evidence chain into JSON format, and generates a feature importance bar chart and a directed graph of the evidence chain through ECharts. Finally, it outputs interactive versions of the medical decision interpretation report in PDF and HTML for clinicians to view.

[0066] Based on the above technical solution, the modular design realizes full-process automation of multi-source medical data acquisition, multimodal feature extraction, interpretable model reasoning, key feature identification, knowledge graph semantic mapping, and structured report generation. It provides clinicians with complete and traceable information from model prediction results to key feature evidence and knowledge graph evidence chain, which significantly improves the transparency of medical artificial intelligence decision-making and clinical adoption.

[0067] This application proposes a systematic solution to the fundamental contradiction between the "black box" nature of models and the clinical interpretability requirements in medical decision-making scenarios by constructing a closed-loop architecture that integrates "multimodal data acquisition - feature extraction - interpretable model reasoning - key feature identification - knowledge graph semantic mapping - structured report generation".

[0068] Specifically, firstly, heterogeneous medical data is uniformly represented through multimodal feature extraction, laying the data foundation for subsequent interpretable analysis. Then, a medical decision-making model with an embedded attention mechanism is designed, simultaneously generating a feature weight matrix while outputting decision results, making the focus of attention within the model explicit. Based on this, key feature identification is used to screen out the core features that contribute most to the decision, and a medical knowledge graph is introduced for semantic mapping and association, transforming numerical feature weights into clinically meaningful entity relationships and evidence chains. This allows scattered key features to be linked into a complete reasoning path through pre-defined clinical relationship types, solving the problem that single features cannot form clinical logic. Finally, a structured medical decision interpretation report is generated, including decision results, key features, evidence chains, and visual charts, integrating multi-source information into a presentation format that conforms to doctors' cognitive habits. In the entire architecture, the feature weight matrix serves as a bridge connecting model reasoning and semantic mapping, enabling abstract computational weights to establish a traceable association with specific medical knowledge; the introduction of knowledge graphs provides verifiable medical semantic support for the reasoning basis; each step is interconnected and indispensable, jointly solving the problems of opaque model decision-making processes, lack of clinical semantic support for explanations, and untraceable reasoning basis in existing technologies, achieving a complete transformation from "black box prediction" to "explainable clinical decision-making basis," the technical effect of which cannot be achieved by any single step or simple combination.

[0069] In the implementation process, each step of the method provided in this embodiment can be completed by the integrated logic circuit in the processor or by software instructions. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0070] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0071] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0072] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. An attention mechanism-based medical decision mechanism explanation method, characterized in that, include: Acquire multi-source medical data; the multi-source medical data includes medical text data, medical image data, and time-series monitoring data; Multimodal feature extraction is performed on multi-source medical data to obtain multimodal feature vectors; The multimodal feature vectors are input into a pre-trained medical decision-making model, which outputs medical decision results and feature weight matrices; the medical decision-making model is a deep learning model with an embedded attention mechanism. Key features are identified from multimodal feature vectors based on feature weight matrices to obtain a set of key features; Based on a pre-defined medical knowledge graph, semantic mapping and association are performed on key feature sets to obtain key decision-making criteria; The results of medical decisions and key decision-making evidence are structured and organized to obtain a medical decision interpretation report.

2. The method of claim 1, wherein, The medical decision-making model includes a feature embedding layer, a location encoding layer, a multi-head self-attention layer, a feedforward network layer, and an output layer; The feature embedding layer is used to map the input multimodal feature vector to a preset latent space to generate a feature sequence; The positional encoding layer is used to add positional encoding information to the feature sequence to generate a positional feature sequence; the positional encoding information is a learnable positional encoding matrix, used to add positional information to each feature in the feature sequence; The multi-head self-attention layer is used to calculate the attention weight between any two positional features in the positional feature sequence and output the feature weight matrix and the enhanced feature representation. The feedforward network layer is used to perform nonlinear transformation on the enhanced feature representation and output the transformed feature representation; The output layer is used to output medical decision results based on the transformation feature representation.

3. The method of claim 2, wherein, The multi-head self-attention layer includes multiple parallel attention heads, a multi-head feature fusion unit, and an attention weight aggregation unit; The attention head includes a query linear transformation layer, a key linear transformation layer, a value linear transformation layer, and a scaled dot product attention calculation unit, which is used to calculate the attention weight between any two positional features in the positional feature sequence through regularization, traverse the positional feature sequence, and output the attention weight matrix and the corresponding weighted feature representation. The multi-head feature fusion unit is used to concatenate and fuse the weighted feature representations output by multiple attention heads to obtain a fused feature representation; the fused feature representation is then residually connected to the position feature sequence and then subjected to layer normalization to obtain an enhanced feature representation; The attention weight aggregation unit is used to average or weightedly fuse the attention weight matrices output by multiple attention heads to obtain a feature weight matrix.

4. The method of claim 1, wherein, The training methods for the medical decision-making model include: Obtain a sample medical dataset; wherein the sample medical dataset includes sample feature vectors and corresponding sample decision results; The sample medical dataset is divided into a training set and a validation set according to a preset ratio; Build an initial decision model based on the Transformer architecture; Based on a pre-defined interpretive loss function, the initial decision model is iteratively trained using a training set to obtain a transitional decision model. The medical decision-making model is obtained by evaluating and optimizing the performance of the transitional decision-making model using a validation set.

5. The method of claim 4, wherein, The explanatory loss function includes prediction loss, knowledge graph alignment loss, attention sparsity loss, and attention stability loss; The prediction loss, constructed based on cross-entropy loss or mean squared error loss, is used to measure the difference between the decision prediction results output by the medical decision-making model and the sample decision results. The knowledge graph alignment loss, calculated based on KL divergence, is used to constrain the weight prediction results output by the medical decision-making model to align with the prior entity relationships defined in the medical knowledge graph. The attention sparsity loss, constructed based on L1 regularization or entropy regularization, is used to constrain the sparsity of the weight prediction results. The attention stability loss, constructed based on mean squared error or cosine distance, is used to constrain the consistency of weight prediction results for similar samples; the similar samples are any two sample feature vectors whose similarity is less than a preset similarity threshold.

6. The method of claim 1, wherein, The construction methods of the medical knowledge graph include: Construct a knowledge graph framework and acquire medical knowledge data; Natural language processing technology is used to extract entity nodes and entity relationships from medical knowledge data to obtain basic knowledge materials. The basic knowledge materials are mapped onto the knowledge graph framework to obtain the initial knowledge graph; The path confidence weights are obtained by calculating the weights of entity relationships in the initial knowledge graph using the GRADE method. The path confidence weights are mapped onto the initial knowledge graph to obtain the medical knowledge graph.

7. The method of claim 6, wherein the medical decision mechanism explanation method based on attention mechanism is characterized in that, The semantic mapping and association of key feature sets based on a pre-defined medical knowledge graph includes: Entity recognition is performed on each key feature in the key feature set to obtain the clinical entity set; The clinical entity set is aligned with the standard concept nodes in the medical knowledge graph to obtain the standard entity set; the standard concept nodes are obtained by standardizing the terminology of the entity nodes. The meta-path algorithm is used to mine the association paths of a standard entity set to obtain a chain of decision evidence. By performing path semantic transformation on the chain of decision-making evidence, key decision-making basis can be obtained.

8. The method of claim 7, wherein, The process of mining association paths for a standard entity set using a meta-path algorithm includes: Based on a medical knowledge graph, a graph search algorithm is used to perform path search on a standard entity set to obtain an initial path set. The initial path set is matched and connected based on the preset meta-path template to obtain the combined path set; The combined path set is deduplicated using a similarity algorithm to obtain a candidate path set; Based on the path confidence weights in the medical knowledge graph, the confidence score of each candidate path is calculated by the cumulative multiplication method or the minimum value method, and a set of path confidence scores is obtained. Sort the path confidence scores by size, and mark the candidate paths corresponding to the top K path confidence scores as decision evidence chains; where K is a positive integer.

9. The method of claim 1, wherein, The multimodal feature extraction of multi-source medical data includes: Entity information is extracted from medical text data using natural language processing techniques to obtain text feature vectors. Deep visual feature extraction is performed on medical image data using a pre-trained convolutional neural network to obtain image feature vectors; Statistical feature extraction of time-series monitoring data is performed using the sliding window technique to obtain a time-series feature vector. Multimodal feature vectors are obtained by aligning and fusing text feature vectors, image feature vectors, and temporal feature vectors through a fully connected layer.

10. An attention mechanism-based medical decision mechanism explanation system applied to the attention mechanism-based medical decision mechanism explanation method according to any one of claims 1-9, characterized in that, It includes a data acquisition module, a feature extraction module, a model inference module, a feature recognition module, a semantic mapping module, and a report generation module; The data acquisition module is used to acquire multi-source medical data, including medical text data, medical image data, and time-series monitoring data. The feature extraction module is used to extract multimodal features from multi-source medical data to obtain multimodal feature vectors; The model inference module is used to input multimodal feature vectors into a pre-trained medical decision-making model and output medical decision results and feature weight matrices; the medical decision-making model is a deep learning model with an embedded attention mechanism. The feature recognition module is used to identify key features of multimodal feature vectors based on feature weight matrices to obtain a set of key features; The semantic mapping module is used to perform semantic mapping and association on a set of key features based on a preset medical knowledge graph to obtain key decision-making basis; The report generation module is used to structure and organize the medical decision results and key decision-making basis to obtain a medical decision interpretation report.

Images