A Method and System for Interpretable Analysis of Active Ingredients in Traditional Chinese Medicine Based on Attention Weight Tracing

Abstract

This invention discloses a method and system for interpretability analysis of active ingredients in traditional Chinese medicine (TCM) based on hierarchical attention weight tracing, belonging to the interdisciplinary field of artificial intelligence and TCM informatics. The system includes an attention extractor, a calibrator, an importance propagator, a compatibility analyzer, and a visualization generator. The original drug-level attention weights extracted from the dual-encoder Transformer model are jointly calibrated based on entropy and gradient. The importance predicted by the model is propagated backward layer by layer to the drug level and the chemical component level of each drug through the calibrated attention matrix and hidden state semantic relationships. Finally, by analyzing the attention path and semantic interaction, the compatibility contribution value between any two drugs is calculated. This provides a hierarchical interpretation from drug to chemical component, quantitatively reveals the synergistic or antagonistic relationships between drugs, and provides objective evaluation through multi-dimensional indicators such as fidelity and completeness.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and traditional Chinese medicine informatics, specifically a method and system for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing. Background Technology

[0002] With the successful application of deep learning technology, especially the Transformer architecture, in natural language processing and biomedicine, it has demonstrated great potential in tasks such as predicting the efficacy of traditional Chinese medicine (TCM) prescriptions. However, deep neural networks are often considered "black box" models, and their internal decision-making processes are difficult to understand. In TCM, a field that heavily relies on empirical knowledge and theoretical systems, clinicians and researchers not only need to know the model's predictions but also need to understand the basis upon which those predictions are made. This lack of interpretability severely hinders the practical application and trust building of artificial intelligence models in TCM clinical decision support, prescription optimization, and new drug development.

[0003] Currently, various methods have been proposed for research on the interpretability of deep learning models, mainly including the following categories:

[0004] The first category is gradient-based methods, such as SaliencyMap and Integrated Gradients. These methods assess the importance of features by calculating the gradient of the model output with respect to the input features. However, in the analysis of traditional Chinese medicine formulas, the input features (drugs, ingredients) are often discretized, and the gradient information may suffer from high noise and instability. Furthermore, it is difficult to directly map the gradient to drug or ingredient entities with clear semantics, resulting in poor readability of the interpretation results.

[0005] The second category is perturbation-based methods, such as LIME and SHAP. These methods estimate feature importance by locally approximating or repeatedly perturbing the input features and observing the changes in the output. Their drawback is the high computational cost, requiring hundreds or even thousands of model forward propagations, resulting in low efficiency. For traditional Chinese medicine formulas, which consist of multiple herbs, each containing numerous chemical components, the perturbation space grows exponentially, leading to poor feasibility and potentially unstable results.

[0006] The third category is attention-based methods. The attention mechanism in Transformer models naturally provides correlation weights between features, and some studies have attempted to directly use the original attention weights as a basis for interpretability. However, existing research shows that the original attention weights are not always strictly consistent with feature importance and may be affected by various factors such as model structure and training dynamics, which may lead to misleading interpretations if used directly.

[0007] In the specific application scenario of traditional Chinese medicine prescription analysis, existing interpretability methods face a series of unique challenges, highlighting their inherent defects and shortcomings:

[0008] First, the explanations are too granular and cannot match the hierarchical cognitive structure of traditional Chinese medicine (TCM). TCM prescriptions have a multi-level system of "prescription-drug-component". Most existing methods can only provide explanations at a single level (such as the drug level), and cannot further trace the importance of the drug to its specific active ingredients, making it difficult to meet the research needs from macroscopic pharmacological properties to microscopic mechanisms.

[0009] Secondly, ignoring drug interactions fails to reveal compatibility rules. The core principle of traditional Chinese medicine formulas lies in "compatibility," that is, the synergistic (mutual reinforcement, mutual enhancement) or antagonistic (mutual aversion, mutual restraint) effects between drugs. Existing methods mainly calculate the independent contribution of each input feature, and cannot quantify the additional effects produced by any combination of two or more drugs, thus failing to provide insights into understanding the principles of formula composition and optimizing compatibility.

[0010] In summary, existing interpretability techniques suffer from problems such as coarse granularity and neglect of compatibility relationships when dealing with traditional Chinese medicine formulas that have complex hierarchical structures and strong correlations. Therefore, there is an urgent need for a new interpretability analysis method that can provide explanations that are hierarchically clear, quantitatively correlated, consistent with traditional Chinese medicine theory, and objectively assessable. Summary of the Invention

[0011] To address the above problems, this invention provides an interpretability analysis method for active ingredients in traditional Chinese medicine based on attention weight tracing, which solves the problems of not being able to match the hierarchical cognitive structure of traditional Chinese medicine and not being able to reveal the compatibility rules.

[0012] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0013] An interpretability analysis method for active ingredients in traditional Chinese medicine based on attention weight tracing includes the following steps:

[0014] S1. Obtain the raw data generated by the pre-trained dual encoder Transformer model after forward propagation of the input Chinese medicine prescription. The raw data includes the raw drug-level attention weight matrix, the raw component-level attention weight matrix corresponding to each drug, the drug-level hidden state, and the component-level hidden state corresponding to each drug.

[0015] S2. Based on the attention entropy and gradient information, the original drug-level attention weight matrix is ​​jointly calibrated, attention heads with entropy values ​​within a preset threshold range are selected, and the absolute values ​​of the gradients of the attention weights are weighted and fused using a loss function to generate the calibrated drug-level attention weight matrix.

[0016] S3. Calculate the importance of the output layer based on the model's predicted logit value for the target category and the derivative of the softmax function. Then, normalize this importance through the classification layer weight vector and propagate it to the CLS label of the drug-grade encoder to obtain the CLS label importance.

[0017] S4. Based on the calibrated drug-level attention weight matrix, drug-level hidden state, and the importance of the CLS marker, the importance flow from the CLS marker to the position of each drug is calculated through the attention flow mechanism to obtain the independent importance score of each drug.

[0018] S5. Based on the independent importance scores of each drug, the component-level attention weight matrix corresponding to each drug, and the semantic similarity between the drug hidden state and the hidden states of each component, calculate the importance scores of each chemical component in each drug.

[0019] S6. Based on the calibrated drug-level attention weight matrix, drug-level hidden state, and the importance of the CLS label, the importance of the combined effect of any two drugs is calculated by analyzing the attention path and semantic interaction function, and then the compatibility contribution value between the drugs is obtained by the difference between the drug's importance score and its independent importance score.

[0020] Furthermore, it also includes: S7, generating a multi-level visualization analysis report containing drug importance ranking, key component list, and compatibility contribution heatmap based on the calculated independent importance score of the drug, chemical component importance score, and compatibility contribution value; and performing a quantitative evaluation of the visualization analysis report, the quantitative evaluation including at least: calculating the explanatory fidelity index through a progressive masking experiment, calculating the explanatory integrity index by retaining key drugs, and calculating the explanatory consistency index by comparing the drug importance ranking with the ranking based on traditional Chinese medicine a priori theories.

[0021] Further, step S2 includes: calculating the entropy value of each attention head k in the original drug-level attention weight matrix. Set the lower limit threshold for entropy. and upper limit threshold The lower threshold The value range is 0.5 to 1.0, and the upper limit threshold is... The value range is 2.2~2.5; if satisfy If the value is 1, retain the attention head and set its corresponding mask value to 1; otherwise, discard the attention head and set its corresponding mask value to 0. Calculate the weight of the task loss function L for each attention head. The absolute value of the gradient; the original weights of all attention heads are weighted and normalized according to the mask value and the absolute value of the gradient to obtain the calibrated drug-grade attention weight matrix.

[0022] Furthermore, the specific formula for calculating the independent importance score of each drug using the attention flow mechanism in step S4 is as follows: ,in ,in, This represents the attention weight of the CLS label for the i-th drug. This represents the attention weight of the j-th drug towards the i-th drug. For a learnable parameter matrix, This indicates that the hidden state vectors of the j-th drug and the i-th drug are concatenated. This represents the independent importance score of the i-th drug.

[0023] Furthermore, step S5 includes: utilizing alignment indexes and component aggregation weights to determine drug importance. Calculate the importance of each component assigned to the i-th drug. : ;in The independent importance score for the i-th drug is... Let be the attention weight of the k-th component in the component-level encoder used to generate the final representation of drug i. Let k be the hidden state of the k-th component of drug i. Let represent the final hidden state of drug i. These are the preset temperature parameters.

[0024] Furthermore, step S6 includes: calculating the attention path weights connecting the CLS markers via drug i and drug j based on the calibrated drug-level attention weight matrix. Based on the hidden states of drug i and drug j, the interaction strength is calculated using the semantic interaction function Φ; the importance of the combined effect of drug i and drug j is then calculated. ,in Assign importance to the CLS; calculate the compatibility contribution value. ,in and These are the independent importance scores for drugs i and j, respectively.

[0025] Furthermore, the formula for calculating the semantic interaction function Φ is as follows: ,in This represents the sigmoid function. and For learnable parameters, This is the preset attenuation index.

[0026] Furthermore, step S7 includes masking drugs sequentially in descending order of importance, observing the decreasing curve of the predicted probability, and calculating the area under the curve for masking in descending order of importance. ; Calculate the area under the curve for randomly masked drugs. Define loyalty score ; Retain the k most important drugs and calculate the ratio of the predicted probability P(y|top-kdrugs) when drugs are retained to the predicted probability P(y|alldrugs) when all drugs are present: Completeness(k) = P(y|top-kdrugs) / P(y|alldrugs).

[0027] Furthermore, step S7 includes obtaining the prior importance ranking based on the theory of monarch, minister, assistant, and guide, and calculating the correlation coefficient between drug importance and prior importance ranking as a consistency score.

[0028] Furthermore, the calculation of the CLS marker importance value in step S3 is specifically as follows: ,in, This represents the model's predicted logit value for the target class c. For the softmax function about The derivative of This is the weight vector corresponding to category c in the classification layer.

[0029] The attention-weighted traceability-based interpretability analysis system for active ingredients in traditional Chinese medicine, applied to any of the methods described above, includes:

[0030] Attention extractor is used to obtain the original drug-level attention weight matrix, original component-level attention weight matrix, drug-level hidden state and component-level hidden state generated by the pre-trained dual encoder Transformer model after forward propagation of the input Chinese herbal medicine prescription.

[0031] A calibrator, connected to the attention extractor, is used to jointly calibrate the original drug-level attention weight matrix based on attention entropy and gradient information to generate a calibrated drug-level attention weight matrix.

[0032] The importance propagator, connected to the attention extractor and calibrator respectively, is used to calculate the independent importance score of each drug by using the calibrated drug-level attention weight matrix, drug-level hidden state and CLS label importance propagated from the model output layer, through the attention flow mechanism, and to calculate the importance score of each chemical component based on the component-level attention weight matrix and component hidden state corresponding to each drug.

[0033] A compatibility analyzer, connected to the calibrator and the importance propagator, is used to calculate the synergistic effect importance and compatibility contribution value between any two drugs based on the calibrated drug-level attention weight matrix, drug-level hidden state and CLS label importance.

[0034] A visualization generator, connected to the importance propagator and compatibility analyzer respectively, is used to generate multi-level visualization analysis reports based on the drug importance score, component importance score, and compatibility contribution value.

[0035] The beneficial effects of this invention are:

[0036] 1. By designing calibration, propagation, and interaction analysis algorithms, the predictive importance of the model output is traced back and rationally allocated to different levels (drugs, components) and combinations (compatibility) of the input, forming a structured explanatory basis. This achieves a two-level hierarchical explanation from drugs to chemical components, enabling quantitative analysis of synergistic or antagonistic effects between drugs, and verifying the reliability of the explanation through objective evaluation indicators. Data visualization techniques are used to convert the data into graphical charts; simultaneously, evaluation experiments based on model behavior (fidelity, completeness) and domain knowledge (consistency) are designed to cross-validate the explanation results.

[0037] 2. By employing a joint calibration strategy based on entropy and gradient, attention heads with low information content (excessively high entropy) or potential overfitting (excessively low entropy) are effectively filtered out, and attention heads with a greater impact on prediction loss are given higher weights, thereby refining a purer and more relevant attention signal. Simultaneously, during importance propagation, a modulation term based on semantic similarity is introduced, ensuring that importance allocation depends not only on weight values ​​but also on the semantic correlation between features.

[0038] 3. A quantitative calculation framework for "compatibility contribution" is defined. By analyzing the second-order paths in the attention matrix and fusing deep interaction information from the drug's latent states, the additional effect produced when any two drugs coexist can be calculated. A three-pronged quantitative evaluation index—fidelity, integrity, and consistency—is designed. Fidelity is verified through system perturbation experiments to determine whether the interpretation truly reflects the model's internal logic; integrity assesses the degree of coverage of key information; and consistency is verified by comparing the algorithm results with prior theories such as the "principal, assistant, adjuvant, and guide" principles. Attached Figure Description

[0039] Figure 1 The flowchart shows the interpretability analysis method of active ingredients in traditional Chinese medicine based on attention weight tracing.

[0040] Figure 2 Comparison of attention calibration before and after using Xiao Chai Hu Tang;

[0041] Figure 3Thermographs contribute to the compatibility of Xiao Chai Hu Tang;

[0042] Figure 4 Here is an example of an interpretability analysis report;

[0043] Figure 5 This is a block diagram of a system for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing. Detailed Implementation

[0044] To enable those skilled in the art to better understand the technical solution, the present invention will be described in detail below with reference to embodiments. The description in this part is only exemplary and explanatory, and should not be used to limit the scope of protection of the present invention in any way.

[0045] Example 1

[0046] See attached document Figure 1 To be continued Figure 4 This embodiment uses the classic formula "Xiao Chai Hu Tang" as an example to demonstrate the entire process of hierarchically interpreting the predictive results of the formula's indication for "common cold" using the method of the present invention. The formula contains seven herbs: Bupleurum, Scutellaria baicalensis, Pinellia ternata, Ginseng, Licorice, Ginger, and Jujube.

[0047] S1. Obtain the raw data generated by the pre-trained dual encoder Transformer model after forward propagation of the input Chinese herbal medicine prescription. The raw data includes the raw drug-level attention weight matrix, the raw component-level attention weight matrix corresponding to each drug, the drug-level hidden state, and the component-level hidden state corresponding to each drug.

[0048] In this embodiment, the seven herbs of the target formula "Xiao Chai Hu Tang" (Bupleurum, Scutellaria baicalensis, Pinellia ternata, Ginseng, Licorice, Ginger, and Jujube) and their known chemical components are transformed into an input sequence that the model can process. The input sequence is a concatenation of standardized drug encoding vectors and component encoding vectors. The input sequence is fed into a dual-encoder Transformer model that has been trained using 100,000 data points labeled with traditional Chinese medicine formulas and indications. The model performs a complete forward propagation inference, and the final output layer gives a predicted probability of 0.94 for the indication of "common cold". During the inference process, the system synchronously captures and stores the following intermediate layer raw output data: an original drug-level attention weight matrix with a dimension of 8×8 (containing 1 CLS label and 7 drugs); 7 drug-level hidden state vectors with a dimension of 768; the component-level attention weight matrix corresponding to each drug (taking the principal drug "Bupleurum" as an example, its matrix dimension is 138×138, corresponding to its 138 known chemical components); and the set of all component-level hidden state vectors for each drug (taking Bupleurum as an example, the set contains 138 component hidden state vectors with a dimension of 768).

[0049] S2. Based on attention entropy and gradient information, the original drug-level attention weight matrix is ​​jointly calibrated, attention heads with entropy values ​​within a preset threshold range are selected, and the absolute values ​​of the gradients of the attention weights are weighted and fused using a loss function to generate the calibrated drug-level attention weight matrix.

[0050] Calculate the original drug-level attention weight matrix The entropy value of each attention head k Set the lower limit threshold for entropy. and upper limit threshold ,like satisfy Then retain the attention head and obtain its corresponding mask value. Set the value to 1; otherwise, discard the attention head and set the corresponding mask value. =0; calculate the task loss function L for each attention head weight. The absolute value of the gradient, i.e. The original weights of all attention heads are weighted, summed, and normalized based on the mask value and the absolute value of the gradient, specifically as follows:

[0051] The calibrated drug-level attention weight matrix is ​​obtained. .

[0052] Specifically, the system extracts the 8-head attention matrix. A head-by-head analysis was performed. The entropy values ​​of the eight attention heads were calculated as follows: [0.85, 1.32, 2.45, 1.78, 0.95, 1.91, 2.60, 1.55]. Based on the threshold... and After screening, the entropy values ​​of the 2nd (1.32), 4th (1.78), 6th (1.91), and 8th (1.55) attention heads are within the threshold range, and their mask values ​​are set to 1; the mask values ​​of the remaining 4 heads are set to 0. Subsequently, the system performs one backpropagation to calculate the absolute gradient of the loss function with respect to the weights of each attention head. Let the absolute gradient values ​​of the 2nd, 4th, 6th, and 8th heads be 0.12, 0.08, 0.15, and 0.10, respectively. Finally, according to the weighting formula, gradient-weighted fusion is performed only on the 4 heads with a mask value of 1 to generate a calibrated 8×8 attention matrix. Compared to the original average attention, this matrix is ​​more prominent at key drug positions (such as the weight of CLS on Bupleurum and Scutellaria), and the noise distribution is reduced.

[0053] S3. Calculate the importance of the output layer based on the model's predicted logit value for the target category and the derivative of the softmax function. Then, normalize this importance through the classification layer weight vector and propagate it to the CLS label of the drug-grade encoder to obtain the CLS label importance.

[0054] In this embodiment, the predicted logit value of the target category "cold" is extracted from the model output. ; Calculate the softmax function with respect to derivative =0.18; Calculate the importance of the output layer. Read the weight vector corresponding to the "cold" category in the classification layer. Calculate its L2 norm =1.2, and calculate the sum of the norms of all class weight vectors. .

[0055] S4. Based on the calibrated drug-level attention weight matrix, drug-level hidden states, and the importance of the CLS markers, the importance flow from the CLS markers to the positions of each drug is calculated using an attention flow mechanism to obtain the independent importance score of each drug. According to the formula... Calculate the independent importance score for each drug, where ,in, This represents the attention weight of the CLS label for the i-th drug. This represents the attention weight of the j-th drug towards the i-th drug. For a learnable parameter matrix, This indicates that the hidden state vectors of the j-th drug and the i-th drug are concatenated. This represents the independent importance score of the i-th drug.

[0056] Specifically, the system iterates through the seven herbs for calculation. Taking the calculation of the importance of "Bupleurum" (i=1) as an example: from... Take out Calculate the sum of importance flows from all locations j (including CLS and other drugs) to Bupleurum. This calculation involves, and the corresponding concatenated hidden states and parameter matrices The operation. Calculated to obtain =1.85, then =0.574. After calculating for all drugs, normalization was performed to obtain the final independent importance score vector for each drug. Bupleurum 0.312, Scutellaria baicalensis 0.245, Pinellia ternata 0.156, Ginseng 0.098, Licorice 0.089, Ginger 0.056, Jujube 0.044.

[0057] S5. Based on the independent importance scores of each drug, the component-level attention weight matrix corresponding to each drug, and the semantic similarity between the drug hidden state and the hidden states of each component, calculate the importance scores of each chemical component in each drug.

[0058] In this embodiment, alignment indexes and component aggregation weights are used to determine drug importance. Calculate the importance of each component assigned to the i-th drug. : ;in The independent importance score for the i-th drug is... Let be the attention weight of the k-th component in the component-level encoder used to generate the final representation of drug i. Let k be the hidden state of the k-th component of drug i. Let represent the final hidden state of drug i. The preset temperature parameter is set to 1.5.

[0059] Specifically, taking the principal herb "Bupleurum" as an example, component-level importance propagation is performed. The system loads the component-level attention matrix corresponding to Bupleurum. Obtain the component aggregation attention weight vector used to generate the final drug representation of "Bupleurum". The latent states of Bupleurum and its 138 components were calculated sequentially. The cosine similarity was calculated. After calculation, the top 5 components of Bupleurum chinense in terms of importance and their scores were obtained: saikosaponin a 0.287, saikosaponin d 0.234, quercetin 0.156, isorhamnetin 0.121, and kaempferol 0.098.

[0060] S6. Based on the calibrated drug-level attention weight matrix, drug-level hidden state, and the importance of the CLS label, the importance of the combined effect of any two drugs is calculated by analyzing the attention path and semantic interaction function, and then the compatibility contribution value between the drugs is obtained by the difference between the drug's importance score and its independent importance score.

[0061] Based on the calibrated drug-level attention weight matrix, the attention path weights connecting the CLS markers via drug i and drug j are calculated. Based on the hidden states of drug i and drug j, the interaction strength is calculated using the semantic interaction function Φ. ,in This represents the sigmoid function. and For learnable parameters, The preset attenuation index is set to 2; calculate the combined effect importance of drug i and drug j. ,in Assign importance to the CLS; calculate the compatibility contribution value. ,in and These are the independent importance scores for drugs i and j, respectively.

[0062] Specifically, let's take the analysis of the compatibility between "Bupleurum" (i=1) and "Scutellaria" (j=2) as an example. Calculate the path weight. =0.032; Calculate the interaction strength ; Calculate the importance of the combined effect: ; Calculate compatibility contribution A positive value indicates a synergistic effect between the two drugs. The system iterates through and calculates all 21 drug combinations, generating a 7×7 compatibility contribution matrix.

[0063] S7. Based on the calculated drug independent importance score, chemical component importance score, and compatibility contribution value, generate a multi-level visualization analysis report including drug importance ranking, key component list, and compatibility contribution heatmap; and quantitatively evaluate the visualization analysis report.

[0064] In this embodiment, the system invokes a visualization engine to generate an analysis report. The report includes: a drug importance bar chart (visually displaying the importance scores of the 7 drugs ranked); a key component table (listing the top-3 most important components and their scores for each drug); and a compatibility contribution heatmap (a 7×7 matrix, with color intensity representing synergistic or antagonistic strength, red for synergy and blue for antagonism). The evaluation module performs three quantitative assessments.

[0065] Loyalty: By Drugs are masked in descending order of scores, the model's predicted probability decreases, and the area under the curve is calculated. =0.85; Perform multiple random sequential occlusions and calculate the average area under the curve. =0.50; Loyalty score =0.35.

[0066] Completeness: Only the most important herb (Bupleurum) was retained, and the model prediction probability retention rate was P(y|top-1drugs) / P(y|alldrugs)=0.41;

[0067] The top 3 drugs (Bupleurum, Scutellaria, and Pinellia) were retained, with a retention rate of P(y|top-3drugs) / P(y|alldrugs)=0.756.

[0068] Consistency: Based on the "principal, assistant, adjuvant, and guide" theory, the prior importance order of Xiao Chai Hu Tang is: [Bupleurum, Scutellaria, Pinellia, Ginseng, Licorice, Ginger, Jujube]. This aligns with the algorithm's conclusion. The Spearman correlation coefficient was calculated based on the ranking, yielding a consistency score of 0.893.

[0069] This embodiment utilizes a hierarchical attention weight tracing algorithm to decompose the model's prediction of the efficacy of Xiao Chai Hu Tang (Minor Bupleurum Decoction) in treating colds into contributions from drugs, chemical components, and drug compatibility. The drug importance ranking derived by the algorithm highly aligns with the traditional "principal, assistant, adjuvant, and guide" theory (consistency 0.893). The key components of Bupleurum (Saikosaponins a and d) are consistent with modern pharmacological research findings, and the strong synergistic effect of the Bupleurum-Scutellaria baicalensis pair (0.087) conforms to the TCM theory of "mutual reinforcement." This demonstrates that the method of this invention can provide reliable, detailed, and interpretable results consistent with TCM theory, effectively enhancing the credibility and practicality of deep learning models in the field of TCM.

[0070] Example 2

[0071] See attached document Figure 5 The attention-weighted traceability-based interpretability analysis system for active ingredients in traditional Chinese medicine is deployed on one or more servers and includes the following modules connected via a data bus:

[0072] Attention extractor is used to obtain the original drug-level attention weight matrix, original component-level attention weight matrix, drug-level hidden state and component-level hidden state generated by the pre-trained dual encoder Transformer model after forward propagation of the input Chinese herbal medicine prescription.

[0073] A calibrator, connected to the attention extractor, is used to jointly calibrate the original drug-level attention weight matrix based on attention entropy and gradient information to generate a calibrated drug-level attention weight matrix.

[0074] The importance propagator, connected to the attention extractor and calibrator respectively, is used to calculate the independent importance score of each drug by using the calibrated drug-level attention weight matrix, drug-level hidden state and CLS label importance propagated from the model output layer, through the attention flow mechanism, and to calculate the importance score of each chemical component based on the component-level attention weight matrix and component hidden state corresponding to each drug.

[0075] A compatibility analyzer, connected to the calibrator and the importance propagator, is used to calculate the synergistic effect importance and compatibility contribution value between any two drugs based on the calibrated drug-level attention weight matrix, drug-level hidden state and CLS label importance.

[0076] A visualization generator, connected to the importance propagator and compatibility analyzer respectively, is used to generate multi-level visualization analysis reports based on the drug importance score, component importance score, and compatibility contribution value.

[0077] The modules are executed in sequence as described above. Users submit traditional Chinese medicine prescription data through the web interface or client. The system automatically completes the full-process analysis from S1 to S7 in the background and returns the final visualization report to the user.

[0078] It should be noted that, in this document, the terms "comprising," "including," and any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Specific examples have been used in this document to illustrate the principles and implementation methods of the present invention. These examples are merely for the purpose of helping to understand the method and core ideas of the present invention. The above descriptions are only preferred embodiments of the present invention. It should be pointed out that, due to the limitations of written expression and the objective existence of infinite specific structures, those skilled in the art can make several improvements, modifications, or variations without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, variations, or combinations, or the direct application of the concept and technical solution of the present invention to other situations without modification, should all be considered within the scope of protection of the present invention.

Claims

1. A method for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing, characterized in that, Includes the following steps: S1. Obtain the raw data generated by the pre-trained dual encoder Transformer model after forward propagation of the input Chinese medicine prescription. The raw data includes the raw drug-level attention weight matrix, the raw component-level attention weight matrix corresponding to each drug, the drug-level hidden state, and the component-level hidden state corresponding to each drug. S2. Based on the attention entropy and gradient information, the original drug-level attention weight matrix is ​​jointly calibrated, attention heads with entropy values ​​within a preset threshold range are selected, and the absolute values ​​of the gradients of the attention weights are weighted and fused using a loss function to generate the calibrated drug-level attention weight matrix. S3. Calculate the importance of the output layer based on the model's predicted logit value for the target category and the derivative of the softmax function. Then, normalize this importance through the classification layer weight vector and propagate it to the CLS label of the drug-grade encoder to obtain the CLS label importance. S4. Based on the calibrated drug-level attention weight matrix, drug-level hidden state, and the importance of the CLS marker, the importance flow from the CLS marker to the position of each drug is calculated through the attention flow mechanism to obtain the independent importance score of each drug. S5. Based on the independent importance scores of each drug, the component-level attention weight matrix corresponding to each drug, and the semantic similarity between the drug hidden state and the hidden states of each component, calculate the importance scores of each chemical component in each drug. S6. Based on the calibrated drug-level attention weight matrix, drug-level hidden state, and CLS label importance, the importance of the combined effect of any two drugs is calculated by analyzing the attention path and semantic interaction function, and then the compatibility contribution value between the drugs is obtained by the difference between the drug's importance score and its independent importance score. S7. Based on the calculated drug independent importance score, chemical component importance score and compatibility contribution value, generate a multi-level visualization analysis report including drug importance ranking, key component list and compatibility contribution heatmap. The visualization analysis report is quantitatively evaluated, and the quantitative evaluation includes at least: calculating the explanatory fidelity index through a progressive masking experiment, calculating the explanatory integrity index by retaining key drugs, and calculating the explanatory consistency index by comparing the drug importance ranking with the ranking of traditional Chinese medicine a priori theories.

2. The method for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing according to claim 1, characterized in that, Step S2 includes: calculating the entropy value of each attention head k in the original drug-level attention weight matrix. Set the lower limit threshold for entropy. and upper limit threshold ,like satisfy If the value is 1, retain the attention head and set its corresponding mask value to 1; otherwise, discard the attention head and set its corresponding mask value to 0. Calculate the weight of the task loss function L for each attention head. The absolute value of the gradient; the original weights of all attention heads are weighted and normalized according to the mask value and the absolute value of the gradient to obtain the calibrated drug-grade attention weight matrix.

3. The method for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing according to claim 1, characterized in that, The specific formula for calculating the independent importance score of each drug using the attention flow mechanism in step S4 is as follows: ,in ,in, This represents the attention weight of the CLS label for the i-th drug. This represents the attention weight of the j-th drug towards the i-th drug. For a learnable parameter matrix, This indicates that the hidden state vectors of the j-th drug and the i-th drug are concatenated. This represents the independent importance score of the i-th drug.

4. The method for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing according to claim 3, characterized in that, Step S5 includes: using alignment indexes and component aggregation weights to determine drug importance. Calculate the importance of each component assigned to the i-th drug. : ;in The independent importance score for the i-th drug is... Let be the attention weight of the k-th component in the component-level encoder used to generate the final representation of drug i. Let be the hidden state of the k-th component of drug i. Let represent the final hidden state of drug i. These are the preset temperature parameters.

5. The method for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing according to claim 4, characterized in that, Step S6 includes: calculating the attention path weights connecting the CLS markers via drug i and drug j based on the calibrated drug-level attention weight matrix. Based on the hidden states of drug i and drug j, the interaction strength is calculated using the semantic interaction function Φ; the importance of the combined effect of drug i and drug j is then calculated. , among which The importance of the CLS marker is determined by... Represent the final hidden state of drug j; calculate the compatibility contribution value. ,in and These are the independent importance scores for drugs i and j, respectively.

6. The method for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing according to claim 5, characterized in that, The formula for calculating the semantic interaction function Φ is as follows: ,in This represents the sigmoid function. Represented as The transpose operation of this matrix, and For learnable parameters, This is the preset attenuation index.

7. The method for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing according to claim 1, characterized in that, Step S7 includes sequentially masking drugs in descending order of importance, observing the decreasing curve of the predicted probability, and calculating the area under the curve for the drugs masked in descending order of importance. ; Calculate the area under the curve for randomly masked drugs. Define loyalty score ; Retain the k most important drugs and calculate the ratio of the predicted probability P(y|top-kdrugs) when drugs are retained to the predicted probability P(y|alldrugs) when all drugs are present: Completeness(k) = P(y|top-kdrugs) / P(y|alldrugs).

8. The method for interpretability analysis of active ingredients in traditional Chinese medicine based on attention weight tracing according to claim 7, characterized in that, Step S7 includes obtaining the prior importance ranking based on the theory of monarch, minister, assistant and guide, and calculating the correlation coefficient between drug importance and prior importance ranking as a consistency score.

9. A system for interpretable analysis of active ingredients in traditional Chinese medicine based on attention weight tracing, applied to the method described in any one of claims 1 to 8, characterized in that, include: Attention extractor is used to obtain the original drug-level attention weight matrix, original component-level attention weight matrix, drug-level hidden state and component-level hidden state generated by the pre-trained dual encoder Transformer model after forward propagation of the input Chinese herbal medicine prescription. A calibrator, connected to the attention extractor, is used to jointly calibrate the original drug-level attention weight matrix based on attention entropy and gradient information to generate a calibrated drug-level attention weight matrix. The importance propagator, connected to the attention extractor and calibrator, is used to calculate the independent importance score of each drug by using the calibrated drug-level attention weight matrix, drug-level hidden state and CLS label importance propagated from the model output layer, through the attention flow mechanism, and to calculate the importance score of each chemical component based on the component-level attention weight matrix and component hidden state corresponding to each drug. A compatibility analyzer, connected to the calibrator and the importance propagator, is used to calculate the synergistic effect importance and compatibility contribution value between any two drugs based on the calibrated drug-level attention weight matrix, drug-level hidden state and CLS label importance. A visualization generator, connected to the importance propagator and compatibility analyzer, is used to generate a visualization analysis report based on the drug's independent importance score, component importance score, and compatibility contribution value.

Images