Traditional Chinese medicine prescription efficacy prediction system and method based on double-encoder transformer

Abstract

The application discloses a traditional Chinese medicine prescription efficacy prediction system and method based on a double-encoder Transformer, and belongs to the technical field of cross between artificial intelligence and informationization of traditional Chinese medicine. A double-layer embedding representation module containing a drug embedding layer and a chemical component embedding layer is constructed; a component-level Transformer encoder and a content-gated attention mechanism are used to extract component aggregation representations of each drug; the representations are dynamically integrated into the input of a drug-level encoder through a component information injection module; a compatibility perception bias matrix is introduced in the drug-level encoder to learn the compatibility relationship between drugs; finally, based on the comprehensive representation of the prescription output by the drug-level encoder, the probability distribution of the disease indication and the targeted tissue is synchronously output through a multi-task prediction module. The application realizes collaborative modeling of the hierarchical structure of traditional Chinese medicine prescriptions, improves the dynamic nature and prediction accuracy of the feature representation, and supports multi-task learning and the interpretability of the prediction results.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and information technology of traditional Chinese medicine, specifically a system and method for predicting the efficacy of traditional Chinese medicine prescriptions based on dual encoder Transformer. Background Technology

[0002] Traditional Chinese medicine prescription efficacy assessment mainly relies on physicians' long-term clinical experience and classical literature records. While this approach has value, it has inherent limitations such as strong subjectivity, difficulty in standardization and quantification, and low efficiency in knowledge transmission, making it difficult to adapt to the needs of large-scale, precise modern pharmaceutical research and development and clinical decision support.

[0003] With the evolution of artificial intelligence technology, utilizing machine learning methods to assist in the prediction of the efficacy of traditional Chinese medicine prescriptions has become an important research direction. Existing technical solutions can be mainly divided into the following categories:

[0004] The first category is based on traditional machine learning algorithms, such as Support Vector Machines (SVM) and Random Forest. These methods typically require pre-converting the Chinese herbal medicines and their chemical components into fixed-length feature vectors before training a classifier for efficacy classification. However, a significant drawback is their limited feature representation capability: static, isolated vector representations cannot characterize the complex synergistic or antagonistic interactions between components after drug combination, and they struggle to flexibly handle situations where the number of input formula drugs varies (i.e., variable-length sequences).

[0005] The second category is based on simple neural network structures, such as multilayer perceptrons (MLPs) or convolutional neural networks (CNNs). These methods have improved the ability to automatically learn features to some extent, but for data such as traditional Chinese medicine prescriptions that have inherent sequentiality and contextual dependencies, they still lack the ability to effectively model long-term dependencies in the sequence and cannot fully learn the order of drugs in the prescription and the global associations across drugs.

[0006] The third category is methods based on the Transformer architecture, especially models employing a single encoder. Transformers, with their self-attention mechanism, can effectively capture global dependencies within sequences, achieving success in fields such as natural language processing. However, when directly applied to traditional Chinese medicine (TCM) formulas, a single Transformer encoder struggles to simultaneously and effectively model their inherent multi-level structural features. Specifically, TCM formulas have a clear two-level hierarchy of "drugs-components": each drug is a collection of multiple chemical components, and the formula itself is composed of multiple drugs according to specific compatibility rules. When processing this hierarchical structure, a single encoder cannot clearly distinguish and collaboratively learn the compatibility relationships at the drug level and the microscopic interactions at the component level, resulting in insufficient utilization of structural information.

[0007] In summary, existing technical solutions generally suffer from the following common technical defects and challenges in achieving intelligent prediction of the efficacy of traditional Chinese medicine prescriptions:

[0008] Rigid Feature Representation: Existing methods typically cannot achieve dynamic, context-aware feature representation. The efficacy of the same drug may differ in different prescription combinations, but existing static representation methods cannot reflect this difference in efficacy caused by changes in the combination environment.

[0009] Insufficient hierarchical modeling: It failed to effectively model the hierarchical structure of "drug-component" in traditional Chinese medicine prescriptions and lacked a dedicated mechanism for collaborative learning of macroscopic compatibility rules between drugs and microscopic interactions between components.

[0010] Lack of model interpretability: Most deep learning-based prediction models are like "black boxes," and their prediction results lack intuitive explanations. It is difficult to trace which key drugs or core components play a decisive role in the final prediction, which seriously hinders their practical application in clinical auxiliary diagnosis and medication decision support systems that require high reliability.

[0011] Therefore, there is an urgent need for a new method and system for intelligent prediction of the efficacy of traditional Chinese medicine prescriptions that can overcome the above-mentioned defects and possess hierarchical structural modeling, dynamic context awareness, and good interpretability. Summary of the Invention

[0012] To address the above problems, this invention provides a system and method for predicting the efficacy of traditional Chinese medicine prescriptions based on dual encoder Transformer, which solves the problems of rigid feature representation, insufficient hierarchical structure modeling, and lack of model interpretability.

[0013] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0014] The TCM prescription efficacy prediction system based on dual encoder Transformer includes: a data preprocessing module, which receives the input list of TCM prescription drug names, converts the drug names into corresponding drug identifiers to form a drug sequence, and obtains the list of chemical component identifiers corresponding to the drug identifiers according to the preset drug-component mapping relationship to form a chemical component sequence.

[0015] A dual encoder model module, connected to the data preprocessing module, receives drug sequences and chemical component sequences; the dual encoder model module includes:

[0016] The drug embedding layer stores a first embedding matrix that maps drug identifiers in the drug sequence to drug embedding vectors.

[0017] A chemical component embedding layer stores a second embedding matrix that maps chemical component identifiers in the chemical component sequence to component embedding vectors.

[0018] A component-level Transformer encoder, connected to the chemical component embedding layer, processes the component embedding vector and extracts the component aggregate representation of the drug through a multi-head self-attention mechanism with content gating factors and a feedforward neural network.

[0019] The component information injection module is connected to the drug embedding layer, the component-level Transformer encoder, and the drug-level Transformer encoder. It aggregates and represents the components of the drug, and superimposes them onto the drug embedding vector corresponding to the drug through a learnable projection matrix and injection intensity parameters to generate an enhanced drug representation.

[0020] A drug-grade Transformer encoder processes the enhanced drug representation sequence and extracts a comprehensive formula-level representation by introducing a multi-head self-attention mechanism of a compatibility-aware bias matrix and a feedforward neural network.

[0021] The multi-task prediction module is connected to the drug-level Transformer encoder, receives the comprehensive representation at the formula level, and performs calculations through independent disease classification units and tissue classification units respectively, outputting the predicted probability distribution of disease indications and the predicted probability distribution of target tissues corresponding to the traditional Chinese medicine formula.

[0022] Furthermore, the embedding matrices of the drug embedding layer and the chemical component embedding layer are initialized using an adaptive strategy based on spectral decomposition, and the initialization formula is as follows: ,in, This represents the initialized embedding matrix. , , This represents the transpose of the left singular matrix, the singular value diagonal matrix, and the right singular matrix obtained from the singular value decomposition of the drug-component co-occurrence matrix. This represents the learnable scaling parameter matrix. This represents the frequency vector of drug or chemical components appearing in the training corpus. The smoothing constant is represented; the rows of the drug-component co-occurrence matrix correspond to drugs, and the columns correspond to chemical components. The element values ​​represent the strength of the co-occurrence relationship between a specific drug and a specific chemical component in the training corpus.

[0023] Furthermore, the formula for calculating the content gating factor in the component-level Transformer encoder is as follows: ,in, As a content gating factor, , , These represent the query, key, and value vector matrices in the self-attention mechanism, respectively. , , This represents the learnable parameters.

[0024] Furthermore, the formula for calculating the compatibility-aware bias matrix in the drug-grade Transformer encoder is as follows: in, This represents the compatibility-aware bias matrix. This represents the drug representation matrix of the current layer. Represents a learnable bilinear transformation matrix. This indicates the temperature scaling parameter.

[0025] Furthermore, during training, the multi-task prediction module uses a total loss function... Loss Predicted by Disease Organizational forecasting of losses and regularization loss The loss term is composed of dynamic weights based on uncertainty, and the calculation formula is as follows: ,in, For the total loss function, , , represent the loss weights for the disease prediction task, the organization prediction task, and the regularization term, respectively. The task belongs to the set {disease, tissue, reg}.

[0026] A method for predicting the efficacy of traditional Chinese medicine prescriptions based on dual encoder Transformer includes the following steps:

[0027] S1: Receive the input list of Chinese herbal medicine prescriptions, convert the list into a sequence of drug identifiers through the data preprocessing module, and obtain the chemical component identifier sequence corresponding to each drug;

[0028] S2: The drug identifier sequence is mapped to a drug embedding vector sequence through the drug embedding layer, and the chemical component identifier sequence is mapped to a component embedding vector sequence through the chemical component embedding layer;

[0029] S3: Encode the component embedding vector sequence of each drug through a component-level Transformer encoder to extract the component aggregate representation of each drug;

[0030] S4: Through the component information injection module, the component aggregation representation of each drug is injected into its corresponding drug embedding vector to generate an enhanced drug representation sequence.

[0031] S5: Encode the enhanced drug representation sequence using a drug-level Transformer encoder to extract a comprehensive representation at the prescription level;

[0032] S6: The comprehensive representation at the prescription level is processed by the disease classification unit and tissue classification unit of the multi-task prediction module to generate the disease indication prediction probability distribution and the target tissue prediction probability distribution, respectively.

[0033] Furthermore, in step S3, the component-level Transformer encoder employs a multi-head self-attention mechanism that includes a content gating factor. Specifically, the calculation includes: linearly projecting the input component embedding vector sequence to obtain a query Q, key K, and value V vector matrix; and calculating the content gating factor. ; Calculate the component attention weights, the formula for component attention weights is: ,in, This is a learnable relative position bias matrix. The dimension of the key vector K is represented; the attention output is processed through residual connections and layer normalization, and a feedforward network with a gated linear unit structure is used for feature transformation. Finally, the hidden state corresponding to the classification label is taken as the component aggregation representation.

[0034] Furthermore, in step S4, the specific operation of injecting the component information includes: aggregating and representing the components extracted in step S3. Through a learnable projection matrix Perform a linear transformation; through learnable injection intensity parameters. The scaling transformation result yields the injected information. ; Injected information and original drug embedding vector Element-by-element addition generates an enhanced drug representation.

[0035] Furthermore, in step S5, the drug-grade Transformer encoder introduces a multi-head self-attention mechanism for the compatibility-aware bias matrix. Specifically, the calculation includes: linearly projecting the enhanced drug representation sequence to obtain a vector matrix of query Q1, key K1, and value V1; and calculating the compatibility-aware bias matrix. ; Calculate the compatibility-sensing bias matrix Calculate drug-grade attention weights ,in, This is the relative position offset matrix. To match the perception bias matrix, The dimension of the key vector K1 is represented; through residual connections, layer normalization, and gated feedforward network processing, the hidden state corresponding to the classification label is finally taken as the comprehensive representation at the prescription level.

[0036] Furthermore, in step S6, the training and inference of the multi-task prediction module includes: normalizing the comprehensive representation at the prescription level through layers, and then inputting it into the disease classification unit and the tissue classification unit respectively; the disease classification unit outputs the logits value of the disease indication through linear transformation: The tissue classification unit outputs the logits value of the target tissue through a linear transformation: The logits are transformed into a probability distribution using the softmax function, and optimized during training using a dynamically weighted loss function. ,in, , , represent the loss weights for the disease prediction task, the organization prediction task, and the regularization term, respectively. The task belongs to the set {disease, tissue, reg}. This is a comprehensive representation.

[0037] The beneficial effects of this invention are as follows: it achieves accurate modeling of the hierarchical structure of traditional Chinese medicine formulas, significantly improving the biological rationality of predictions. The collaborative design of component-level Transformer encoders and drug-level Transformer encoders accurately simulates the natural hierarchical relationship of "chemical components → single herbs → compound formulas". The component-level encoder learns the interactions between the components within the drug, while the drug-level encoder learns the compatibility rules between the drugs in the formula. This hierarchical modeling makes the features captured by the model more consistent with the internal logic of traditional Chinese medicine theory, thereby greatly improving the accuracy and reliability of efficacy prediction.

[0038] A dynamic context-aware drug representation was generated, overcoming the limitations of static representations. Through a component information injection mechanism, the dynamic representation of each herb, calculated based on its actual composition, is integrated into its drug embedding. The same herb will obtain different vector representations in different prescriptions, thus more accurately reflecting its role in specific formulation contexts.

[0039] This model employs a multi-task prediction head to simultaneously predict disease indications and targeted tissues, and innovatively utilizes dynamic loss weighting based on uncertainty. This allows the model to automatically balance the learning difficulty and importance of different tasks during training, enabling related tasks to share underlying features and mutually reinforce each other. Attached Figure Description

[0040] Figure 1 This is a system architecture diagram of a traditional Chinese medicine prescription efficacy prediction system based on a dual-encoder Transformer.

[0041] Figure 2 The flowchart shows a method for predicting the efficacy of traditional Chinese medicine prescriptions based on a dual-encoder Transformer.

[0042] Figure 3 Heatmap of attention for the six-position Rehmannia pill;

[0043] Figure 4 Attention distribution diagram of Rehmannia glutinosa components. Detailed Implementation

[0044] To enable those skilled in the art to better understand the technical solution, the present invention will be described in detail below with reference to embodiments. The description in this part is only exemplary and explanatory, and should not be used to limit the scope of protection of the present invention in any way.

[0045] Example 1

[0046] This embodiment provides a traditional Chinese medicine prescription efficacy prediction system based on a dual-encoder Transformer. The system is deployed on a server equipped with a GPU accelerator, which includes a central processing unit, memory, a solid-state drive, and a network interface.

[0047] See attached document Figure 1 The system includes the following modules:

[0048] Data Preprocessing Module: This module receives a list of traditional Chinese medicine (TCM) formula names submitted by users via an API interface. Internally, it contains a pre-built standard TCM name dictionary, which includes 5632 drug entries and their unique identifiers. The preprocessing module first maps the input names to corresponding drug identifier sequences. Then, based on a pre-stored drug-chemical component mapping dictionary, it queries the list of chemical component identifiers corresponding to each drug identifier. This dictionary associates 5632 drugs with 26348 chemical components. Finally, the module outputs aligned batch tensors of drug identifier sequences and chemical component identifier sequences.

[0049] Dual encoder model module: This module is the core computing unit of the system and is loaded into the server's GPU memory.

[0050] Drug embedding layer: This layer maintains a drug embedding matrix with dimensions of 5632×768. Each row of this matrix corresponds to a 768-dimensional dense vector representation of a drug identifier.

[0051] Chemical composition embedding layer: This layer maintains a chemical composition embedding matrix with dimensions of 26348×768. Each row of this matrix corresponds to a 768-dimensional dense vector representation of a chemical composition identifier.

[0052] The embedding matrices of the drug embedding layer and the chemical component embedding layer are initialized using an adaptive strategy based on spectral decomposition. The initialization formula is as follows: ,in, This represents the initialized embedding matrix. , , This represents the transpose of the left singular matrix, the singular value diagonal matrix, and the right singular matrix obtained from the singular value decomposition of the drug-component co-occurrence matrix. This represents the learnable scaling parameter matrix. This represents the frequency vector of drug or chemical components appearing in the training corpus. The smoothing constant is represented; the rows of the drug-component co-occurrence matrix correspond to drugs, and the columns correspond to chemical components. The element values ​​represent the strength of the co-occurrence relationship between a specific drug and a specific chemical component in the training corpus.

[0053] Component-level Transformer Encoder: This encoder consists of four stacked layers with identical structures. Each layer contains an 8-head self-attention sublayer and a feedforward neural network sublayer. The self-attention sublayer employs a content gating mechanism, with its content gating factor calculated based on the mean vector of the query matrix Q and the maximum vector of the key matrix K. The hidden layer of the feedforward neural network sublayer has a dimension of 2048 and uses a gated linear unit activation function. The input of this encoder is a sequence of chemical component embeddings, and the output is a 768-dimensional vector representing the aggregated components for each drug. The formula for calculating the content gating factor in the component-level Transformer encoder is as follows: ,in, As a content gating factor, , , These represent the query, key, and value vector matrices in the self-attention mechanism, respectively. , , This represents the learnable parameters.

[0054] Component Information Injection Module: This unit contains a 768×768-dimensional learnable projection matrix and a learnable scalar injection intensity parameter. Its function is to aggregate the component representation output by the component-level encoder, transform it through projection and tanh activation function, and then proportionally superimpose it onto the initial embedding vector of the corresponding drug.

[0055] Drug-grade Transformer Encoder: This encoder also consists of four stacked layers with identical structures, each containing an 8-head self-attention network and a 2048-dimensional feedforward network. Its self-attention calculation incorporates a compatibility-aware bias matrix, obtained by performing a bilinear transformation on the current drug representation matrix and multiplying it by a temperature scaling parameter of 0.1. The encoder's input is a drug representation sequence enhanced with component information injection, and its output is a comprehensive formula-level representation, i.e., a 768-dimensional hidden state vector corresponding to the classification label. The formula for calculating the compatibility-aware bias matrix in the drug-grade Transformer encoder is as follows: in, This represents the compatibility-aware bias matrix. This represents the drug representation matrix of the current layer. Represents a learnable bilinear transformation matrix. This indicates the temperature scaling parameter.

[0056] Multi-task prediction module: This module contains two parallel linear classifiers.

[0057] Disease Classification Unit: This unit contains a 768×155 dimensional weight matrix and a 155 dimensional bias vector. It takes the comprehensive formula representation output by the drug-level encoder as input and outputs the predicted scores for 155 disease indications.

[0058] Tissue Classification Unit: This unit contains a 768×28 dimensional weight matrix and a 28-dimensional bias vector. It takes the same formula as input and outputs predicted scores for 28 target tissues.

[0059] The outputs of the two classification units are normalized using the Softmax function to obtain the final probability distribution. During training, the multi-task prediction module uses the following total loss function: Loss Prediction Based on Disease Organizational forecasting of losses and regularization loss The loss term is composed of dynamic weights based on uncertainty, and the calculation formula is as follows: ,in, , , represent the loss weights for the disease prediction task, the organization prediction task, and the regularization term, respectively. For the total loss function, The task belongs to the set {disease, tissue, reg}.

[0060] API Service Module: This module is implemented based on the FastAPI framework and provides an HTTP RESTful endpoint " / predict". Clients send JSON requests containing a list of drug names to this endpoint, and the system sequentially calls the modules mentioned above to process them, encapsulating the returned disease prediction probabilities and tissue prediction probabilities into a JSON response.

[0061] Example 2

[0062] See attached document Figure 2 To be continued Figure 4 A method for predicting the efficacy of traditional Chinese medicine prescriptions based on a dual-encoder Transformer, the process of which includes at least steps S1-S6:

[0063] S1. Receive the input list of Chinese herbal medicine prescriptions, convert the list of medicines into a sequence of drug identifiers through the data preprocessing module, and obtain the chemical component identifier sequence corresponding to each drug.

[0064] S2. The drug identifier sequence is mapped to a drug embedding vector sequence through a drug embedding layer, and the chemical component identifier sequence is mapped to a component embedding vector sequence through a chemical component embedding layer;

[0065] S3. Encode the component embedding vector sequence of each drug using a component-level Transformer encoder to extract the component aggregate representation of each drug.

[0066] S4. Through the component information injection module, the component aggregation representation of each drug is injected into its corresponding drug embedding vector to generate an enhanced drug representation sequence.

[0067] S5. Encode the enhanced drug representation sequence using a drug-level Transformer encoder to extract a comprehensive representation at the prescription level;

[0068] S6. The comprehensive representation at the prescription level is processed by the disease classification unit and tissue classification unit of the multi-task prediction module to generate the disease indication prediction probability distribution and the target tissue prediction probability distribution, respectively.

[0069] Step S1 includes at least steps S110-S130:

[0070] S110. Receive the input list of Chinese herbal medicine prescriptions, perform drug name normalization and identifier mapping processing, and obtain a set of drug identifier sequences and chemical component identifier sequences.

[0071] The list of traditional Chinese medicine (TCM) prescriptions is a collection of standard names of Chinese medicinal materials, derived from clinical prescriptions or database queries. Specifically, the list of TCM prescriptions is used as input, and each item is matched against a pre-set standard TCM terminology. The standard TCM terminology contains 5632 entries, each recording the standard Chinese name, Latin name, and corresponding unique numerical identifier of the medicinal material. Successfully matched drug names are converted into their corresponding numerical identifiers and arranged in the order of the input list, generating a drug identifier sequence D. For drug names that fail to match, the system records an anomaly and triggers a manual review process. Further, based on a pre-stored drug-chemical component mapping dictionary, all chemical component identifiers contained in each drug identifier in sequence D are searched. The drug-chemical component mapping dictionary associates 5632 kinds of Chinese medicinal materials with 26348 kinds of chemical components, and the mapping relationship comes from the TCMSP database and the HERB database. After the search, a set of chemical component identifiers {C1, C2, ..., Cn} corresponding one-to-one with the drug identifier is generated, where n is the number of drugs. Each chemical component identifier is one of 26348 unique numerical codes. After this section is processed, the drug identifier sequence D and the chemical component identifier sequence set {C1,C2,…,Cn} are used as structured output.

[0072] In this embodiment, the input list of traditional Chinese medicine (TCM) formulas is based on the classic formula "Liuwei Dihuang Wan" and includes "Shu Dihuang" (prepared Rehmannia root), "Shan Zhu Yu" (Cornus officinalis), "Shan Yao" (Dioscorea opposita), "Ze Xie" (Alisma plantago-aquatica), "Fu Ling" (Poria cocos), and "Mu Dan Pi" (Paeonia suffruticosa). Each drug name in the list is matched against a pre-set standard TCM terminology list containing 5632 entries. After matching, "Shu Dihuang" is mapped to drug identifier 101, "Shan Zhu Yu" to 205, "Shan Yao" to 308, "Ze Xie" to 412, "Fu Ling" to 519, and "Mu Dan Pi" to 623, generating the drug identifier sequence D=101,205,308,412,519,623.

[0073] Furthermore, based on the pre-stored drug-chemical component mapping dictionary, the list of chemical component identifiers contained in each identifier in sequence D is searched. Taking the identifier 101 of "Rehmannia glutinosa" as an example, its corresponding chemical component list includes identifiers of active ingredients such as "Citrin" and "Rehmannia glutinosa glycoside". Finally, a set of chemical component identifier sequences {C1,C2,C3,C4,C5,C6} is generated, where C1 corresponds to the component list of "Rehmannia glutinosa".

[0074] S120. Perform sequence padding and alignment on the drug identifier sequence and the chemical component identifier sequence set to generate aligned batch tensor data;

[0075] Specifically, during sequence padding, a maximum sequence length is first set. For drug sequences, the maximum length is set to 32. For chemical component sequences, the maximum number of components per drug is set to 128. For sequences shorter than the maximum length, a specific padding identifier is added to the end of the sequence; for sequences longer than the maximum length, the tail is truncated. After padding, all drug sequences are unified into a one-dimensional tensor of length 32, and all chemical component sequences are unified into a two-dimensional tensor of shape [number of drugs, 128]. During alignment, a corresponding attention mask matrix is ​​generated. For drug sequence masks, the value at the padding position is 0, and the value at the actual drug position is 1. For chemical component sequence masks, a mask vector of length 128 is generated independently for each drug component sequence, with the value at the actual component position being 1 and the value at the padding position being 0. The above operations ensure that input prescriptions of different lengths can be processed in batches. During processing, the original sequence length and padding position information are recorded and written into batch metadata. After this section is processed, three outputs are generated: the aligned drug identifier tensor, the aligned chemical component identifier tensor, and the corresponding drug mask matrix and component mask matrix.

[0076] S130. Based on the batch tensor data and mapping dictionary, perform embedding matrix initialization and table lookup operations to generate drug embedding vector sequence and component embedding vector sequence.

[0077] Specifically, embedding matrix initialization is performed upon system startup. The drug embedding layer maintains a 5632×768-dimensional drug embedding matrix, and the chemical component embedding layer maintains a 26348×768-dimensional chemical component embedding matrix. Both matrices are initialized using an adaptive strategy based on spectral decomposition. The initialization process first constructs a drug-component co-occurrence matrix, where rows correspond to drugs, columns to chemical components, and element values ​​represent the co-occurrence frequency of a specific drug and a specific chemical component in the training corpus. Singular value decomposition is then performed on this matrix. The initialization formula is: , ,in Let Σ be the square root of the singular valued diagonal matrix Σ, and freq be the frequency vector of the drug or ingredient in the training corpus. A learnable 768-dimensional scaled parameter vector, This is a smoothing constant. This formula ensures that the embedding vectors initially contain potential association information between the drug and its components. After initialization, a lookup operation is performed. Based on the drug identifier tensor output in step S120, the corresponding row vectors are indexed from the drug embedding matrix to generate a drug embedding vector sequence of shape [batch size, sequence length, 768]. Based on the chemical component identifier tensor, the corresponding row vectors are indexed from the chemical component embedding matrix to generate a component embedding vector sequence of shape [batch size, drug quantity, 128, 768]. The lookup operation ignores the vectors corresponding to the filler identifiers, and their embedding vectors remain zero. After this processing step, the drug embedding vector sequence and the component embedding vector sequence are output as input to the subsequent encoder.

[0078] Steps S2-S6 are executed consecutively during the model forward propagation, with the following internal cooperation: the product drug embedding vector sequence and component embedding vector sequence from step S2 are directly input into step S3; the output of step S3 is input into step S4; the output of step S4 is input into step S5; and the output of step S5 is input into step S6. Each step shares the same batch dimension and mask matrix to ensure data flow consistency.

[0079] Step S3 includes at least steps S310-S330:

[0080] S310. Obtain the component embedding vector sequence and component mask matrix, generate query, key, and value vector matrices through linear projection, and calculate the gating factor through the content gating mechanism.

[0081] Specifically, a component embedding vector sequence of shape [batch size, drug quantity, 128, 768] and the corresponding component mask matrix are input into a component-level Transformer encoder. This encoder consists of four stacked layers with identical structures, each containing a multi-head self-attention and feedforward network. First, for the component sequence corresponding to a single drug, projection is performed using three independent linear transformation matrices (all 768×768) to generate a query matrix, a key matrix, and a value matrix. The calculation process includes: calculating the mean vector of the query matrix in the feature dimension and the maximum vector of the key matrix in the feature dimension; concatenating the two vectors; performing a linear transformation using a learnable gating weight matrix and a gating bias, and applying the Sigmoid function to generate a gating signal; simultaneously, transforming the value matrix using a learnable value transformation matrix and applying the tanh function to generate a content basis; subsequently, calculating the final content gating factor. :

[0082] ,

[0083] in, As a content gating factor, , , These represent the query, key, and value vector matrices in the self-attention mechanism, respectively. , , This represents the learnable parameters. The mechanism dynamically modulates the propagation strength of each feature dimension based on global statistics of the current sequence.

[0084] S320. Combining relative position bias and content gating factor, perform multi-head self-attention calculation and residual connection to generate attention output features; specifically, perform multi-head attention calculation. The projected query matrix, key matrix, and value matrix are split into 8 heads. For each head, calculate the scaled dot product attention score: score = (query head matrix · key head matrix transpose) / √(key vector dimension). Based on this, add a learnable relative position bias matrix. Apply a component mask matrix, setting the score corresponding to the filling position to a very small negative value. Apply the Softmax function to the score matrix to obtain the normalized attention weight matrix. Multiply the weight matrix with the value head matrix to obtain the output of that head. Concatenate the outputs of the 8 heads along the feature dimension and obtain the self-attention output through an output linear projection matrix. Multiply the content gating factor element-wise with the self-attention output to achieve gating. Then, residual connections and layer normalization are performed: The Dropout rate is 0.1.

[0085] S330. Apply a gated feedforward network to the features after layer normalization, perform residual connection and layer normalization again, and extract the hidden state of the classification label as the component aggregation representation.

[0086] Specifically, the feedforward network employs a gated linear unit structure. For the input x (i.e. The calculation process is as follows: ,in, , This is the weight matrix. For the gated weight matrix, , , The corresponding bias is used. After calculating the feedforward network output, a second residual connection and layer normalization are performed. The above process S310-S330 is repeated in the four coding layers. For each drug, after four layers of coding, the 768-dimensional hidden state vector corresponding to the first position in the final output sequence (i.e., the preset classification label CLS) is taken as the component aggregation representation of that drug. This process is performed on all drugs in the batch to obtain the component aggregation representation matrix.

[0087] In this embodiment, the S310-S330 process was repeated for the other five herbs, including Cornus officinalis, to finally obtain the component aggregation representation matrix of the six herbs. See Appendix for details. Figure 4 In this process, the model assigned high attention weights to the key active ingredients of Rehmannia glutinosa, namely, catalpol and rehmannia glycoside.

[0088] Step S4 is the injection of component information, which is completed in a single step:

[0089] S4. Polymerize and represent the extracted components. Through learnable projection matrices Perform a linear transformation; through learnable injection intensity parameters. The scaling transformation result yields the injected information. ; Injected information and original drug embedding vector Element-by-element addition generates an enhanced drug representation. The same operation is performed on the other five herbs, including "Cornus officinalis," to generate enhanced drug representation sequences.

[0090] Step S5 includes at least steps S510-S530:

[0091] S510. Obtain the enhanced drug representation sequence and drug mask matrix, generate query, key, and value vector matrices through linear projection, and calculate the compatibility-aware bias matrix.

[0092] Specifically, the enhanced drug representation sequence and drug mask matrix are input into a drug-level Transformer encoder. This encoder also consists of four layers. First, linear projection is performed to obtain the query matrix, key matrix, and value matrix. Next, the compatibility-aware bias matrix is ​​calculated. The calculation process is as follows: Compatibility-aware bias matrix in, This represents the compatibility-aware bias matrix. This represents the drug representation matrix of the current layer. Represents a learnable bilinear transformation matrix. The temperature scaling parameter is represented. The bilinear transformation matrix is ​​a learnable 768×768 matrix, and the temperature scaling parameter is fixed at 0.1. The calculated compatibility perception bias matrix is ​​a square matrix whose elements reflect the strength of the compatibility relationship between drug pairs, serving as a bias term for attention calculation. In this embodiment, this matrix encodes the compatibility relationship between drugs; for "Liuwei Dihuang Wan," the calculated... In the matrix, the bias value between "Rehmannia glutinosa" and "Cornus officinalis" is 0.08, indicating that the model has captured the close relationship between the two as core components (principal drugs).

[0093] S520 combines relative position bias, compatibility perception bias and masking to perform multi-head self-attention computation and residual connection to generate attention output features;

[0094] Specifically, multi-head attention calculation is performed. For each attention head, a score is calculated: score = , This is the relative position offset matrix. To match the perception bias matrix, is the dimension of the key vector. The relative position bias matrix is ​​a learnable relative position bias matrix of the drug sequence. A drug mask matrix is ​​applied to mask the padding positions. Then, a standard process of Softmax, multiplication of the value matrix, multi-head concatenation, and linear projection is performed to obtain the final self-attention output. ,in, This is the relative position offset matrix. To match the perception bias matrix, To match the upper and lower key vector dimensions, residual connections and layer normalization are performed. In this embodiment, in the final attention weight matrix, "Rehmannia glutinosa" and "Cornus officinalis" receive the highest attention weights, which is consistent with their positioning as principal herbs in traditional Chinese medicine theory; "Dioscorea opposita" and "Poria cocos" receive medium weights as assistant herbs; and "Alisma plantago-aquatica" and "Paeonia suffruticosa" receive lower weights as adjuvant herbs.

[0095] S530. Apply a gated feedforward network to the features after layer normalization, perform residual connection and layer normalization again, and extract the hidden state of the classification label as the comprehensive representation of the prescription level.

[0096] Specifically, the feedforward network structure is exactly the same as in step S330, using gated linear units. This process loops through four encoding layers. Finally, the 768-dimensional hidden state vector corresponding to the first position (CLS) in the output sequence of the fourth layer is taken as the comprehensive representation of the entire formula. In this embodiment, after four layers of encoding including a gated feedforward network, the 768-dimensional hidden state vector corresponding to the first classification label (CLS) in the final output sequence is taken as the comprehensive representation of the entire "Liuwei Dihuang Wan" formula.

[0097] Step S6 includes at least steps S610-S630:

[0098] S610. Obtain the comprehensive representation of the prescription level, and after hierarchical normalization, input the disease classification unit and tissue classification unit respectively for linear transformation;

[0099] Specifically, the comprehensive representation of the prescription levels is processed through layer normalization. The normalized representation is then input into two parallel linear classifiers. Disease classification unit calculation: The disease weight matrix It is a 768×155 weight matrix, with disease bias. It is a 155-dimensional bias vector, and the output disease prediction score is a 155-dimensional vector corresponding to 155 disease indications. Tissue classification unit calculation: The organization weight matrix It is a 768×28 weight matrix, with organizational bias. It is a 28-dimensional bias vector, and the output tissue prediction score is a 28-dimensional vector, corresponding to 28 target tissues.

[0100] S620. Apply the Softmax function to the output of the linear transformation to generate a probability distribution;

[0101] Specifically, the disease prediction score vector is normalized using the Softmax function, converting the values ​​into probabilities. The calculation process is as follows: Disease prediction probability distribution = softmax(disease prediction score), Tissue prediction probability distribution = softmax(tissue prediction score). The disease prediction probability distribution is a 155-dimensional probability vector, where each element represents the prediction probability of the corresponding disease. The tissue prediction probability distribution is a 28-dimensional probability vector, where each element represents the prediction probability of the corresponding tissue. In this embodiment, the calculation results for "Liuwei Dihuang Wan" show that the Top-5 disease indications and their confidence levels are: "Xiao Ke Zheng" (diabetes) 0.234, "Zanthropy" 0.186, "Tinnitus" 0.142, "Low back pain" 0.098, and "Insomnia" 0.076. Applying the Softmax function, we obtain a 28-dimensional predicted probability distribution of the target tissue. The top-5 target tissues and their confidence levels were: "Kidney" 0.312, "Liver" 0.245, "Spleen" 0.156, "Blood" 0.089, and "Brain" 0.067. These predictions are highly consistent with the traditional efficacy of Liuwei Dihuang Wan (Six-Ingredient Rehmannia Pill) of "nourishing Yin and tonifying the kidneys".

[0102] S630. During the training phase, multi-task loss is calculated based on predicted probability and true label, and joint optimization is performed using uncertainty weighting.

[0103] Specifically, during model training, the loss needs to be calculated. For disease prediction tasks, the cross-entropy loss function is used to calculate the disease loss. For tissue prediction tasks, the cross-entropy loss function is also used to calculate the tissue loss. The total loss function is: ,in, , , These represent the loss weights for the disease prediction task, the tissue prediction task, and the regularization term, respectively. The regularization loss is an L2 regularization loss term used to prevent overfitting. The disease loss weights and tissue loss weights are not fixed values ​​but are automatically learned based on task uncertainty. For each prediction task, its loss weights... The task belongs to the set {disease, tissue, reg}, where the task uncertainty parameter is... This is a learnable scalar parameter, initially set to 0, representing the homoscedastic uncertainty of the task. During training, the model parameters, classifier parameters, and uncertainty parameters are optimized together by the AdamW optimizer, enabling the model to automatically balance the learning difficulty and importance of different tasks. The inference phase directly outputs the disease prediction probability distribution and tissue prediction probability distribution obtained in step S620.

[0104] It should be noted that, in this document, the terms "comprising," "including," and any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Specific examples have been used in this document to illustrate the principles and implementation methods of the present invention. These examples are merely for the purpose of helping to understand the method and core ideas of the present invention. The above descriptions are only preferred embodiments of the present invention. It should be pointed out that, due to the limitations of written expression and the objective existence of infinite specific structures, those skilled in the art can make several improvements, modifications, or variations without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, variations, or combinations, or the direct application of the concept and technical solution of the present invention to other situations without modification, should all be considered within the scope of protection of the present invention.

Claims

1. A traditional Chinese medicine prescription efficacy prediction system based on dual encoder Transformer, characterized in that, include: The data preprocessing module receives the input list of Chinese herbal medicine prescription drug names, converts the drug names into corresponding drug identifiers to form a drug sequence, and obtains the list of chemical component identifiers corresponding to the drug identifiers according to the preset drug-component mapping relationship to form a chemical component sequence. A dual encoder model module, connected to the data preprocessing module, receives drug sequences and chemical component sequences; The dual encoder model module includes: The drug embedding layer stores a first embedding matrix that maps drug identifiers in the drug sequence to drug embedding vectors. A chemical component embedding layer stores a second embedding matrix that maps chemical component identifiers in the chemical component sequence to component embedding vectors. A component-level Transformer encoder, connected to the chemical component embedding layer, processes the component embedding vector and extracts the component aggregate representation of the drug through a multi-head self-attention mechanism with content gating factors and a feedforward neural network. The component information injection module, connected to the drug embedding layer and the component-level Transformer encoder, aggregates and represents the components of the drug, and superimposes them onto the drug embedding vector corresponding to the drug through a learnable projection matrix and injection intensity parameters to generate an enhanced drug representation sequence. A drug-grade Transformer encoder, connected to the component information injection module, processes the enhanced drug representation sequence and extracts a comprehensive representation at the prescription level by introducing a multi-head self-attention mechanism of the compatibility-aware bias matrix and a feedforward neural network. The multi-task prediction module is connected to the drug-level Transformer encoder of the dual encoder model module. It receives the comprehensive representation at the formula level and performs calculations through independent disease classification units and tissue classification units respectively, and outputs the predicted probability distribution of disease indications and the predicted probability distribution of target tissues corresponding to the traditional Chinese medicine formula. The embedding matrices of the drug embedding layer and the chemical component embedding layer are initialized using an adaptive strategy based on spectral decomposition, and the initialization formula is as follows: ,in, This represents the initialized embedding matrix. , , This represents the transpose of the left singular matrix, the singular value diagonal matrix, and the right singular matrix obtained from the singular value decomposition of the drug-component co-occurrence matrix. This represents the learnable scaling parameter matrix. This represents the frequency vector of drug or chemical components appearing in the training corpus. The smoothing constant is represented; the rows of the drug-component co-occurrence matrix correspond to drugs, and the columns correspond to chemical components. The element values ​​represent the strength of the co-occurrence relationship between a specific drug and a specific chemical component in the training corpus. The formula for calculating the content gating factor in the component-level Transformer encoder is as follows: ,in, As a content gating factor, , , These represent the query, key, and value vector matrices in the self-attention mechanism, respectively. , , Indicates learnable parameters; The formula for calculating the compatibility-aware bias matrix in the drug-grade Transformer encoder is as follows: in, This represents the compatibility-aware bias matrix. This represents the drug representation matrix of the current layer. Represents a learnable bilinear transformation matrix. Indicates the temperature scaling parameter; The multi-task prediction module, during training, uses a total loss function... Loss Predicted by Disease Organizational forecasting of losses and regularization loss The loss term is composed of dynamic weights based on uncertainty, and the calculation formula is as follows: ,in, , , represent the loss weights for the disease prediction task, the organization prediction task, and the regularization term, respectively. For the total loss function, The task belongs to the set {disease, tissue, reg}.

2. A method for predicting the efficacy of traditional Chinese medicine prescriptions based on the dual-encoder Transformer-based system for predicting the efficacy of traditional Chinese medicine prescriptions as described in claim 1, characterized in that, Includes the following steps: S1: Receive the input list of Chinese herbal medicine prescriptions, convert the list into a sequence of drug identifiers through the data preprocessing module, and obtain the chemical component identifier sequence corresponding to each drug; S2: The drug identifier sequence is mapped to a drug embedding vector sequence through the drug embedding layer, and the chemical component identifier sequence is mapped to a component embedding vector sequence through the chemical component embedding layer; S3: Encode the component embedding vector sequence of each drug through a component-level Transformer encoder to extract the component aggregate representation of each drug; S4: Through the component information injection module, the component aggregation representation of each drug is injected into its corresponding drug embedding vector to generate an enhanced drug representation sequence. S5: Encode the enhanced drug representation sequence using a drug-level Transformer encoder to extract a comprehensive representation at the prescription level; S6: The comprehensive representation at the prescription level is processed by the disease classification unit and tissue classification unit of the multi-task prediction module to generate the disease indication prediction probability distribution and the target tissue prediction probability distribution, respectively.

3. The method for predicting the efficacy of traditional Chinese medicine prescriptions based on dual encoder Transformer according to claim 2, characterized in that, In step S3, the component-level Transformer encoder employs a multi-head self-attention mechanism that includes a content gating factor. Specifically, the calculation includes: linearly projecting the input component embedding vector sequence to obtain a query Q, key K, and value V vector matrix; and calculating the content gating factor. ; Calculate the component attention weights, the formula for component attention weights is: ,in, This is a learnable relative position bias matrix. The dimension of the key vector K is represented; the attention output is processed through residual connections and layer normalization, and a feedforward network with a gated linear unit structure is used for feature transformation. Finally, the hidden state corresponding to the classification label is taken as the component aggregation representation.

4. The method for predicting the efficacy of traditional Chinese medicine prescriptions based on dual encoder Transformer according to claim 2, characterized in that, In step S4, the specific operation of injecting component information includes: aggregating and representing the components extracted in step S3. Through learnable projection matrices Perform a linear transformation; through learnable injection intensity parameters. The scaling transformation result yields the injected information. ; Injected information and original drug embedding vector Element-by-element addition generates an enhanced drug representation.

5. The method for predicting the efficacy of traditional Chinese medicine prescriptions based on dual encoder Transformer according to claim 2, characterized in that, In step S5, the drug-grade Transformer encoder introduces a multi-head self-attention mechanism for the compatibility-aware bias matrix. Specifically, the calculation includes: linearly projecting the enhanced drug representation sequence to obtain a vector matrix of query Q1, key K1, and value V1; and calculating the compatibility-aware bias matrix. Calculate drug-grade attention weights ,in, This is the relative position offset matrix. To match the perception bias matrix, The dimension of the key vector K1 is represented; through residual connections, layer normalization, and gated feedforward network processing, the hidden state corresponding to the classification label is finally taken as the comprehensive representation at the prescription level.

6. The method for predicting the efficacy of traditional Chinese medicine prescriptions based on dual encoder Transformer according to claim 2, characterized in that, In step S6, the training and inference of the multi-task prediction module includes: normalizing the comprehensive representation at the prescription level through layers and inputting it into the disease classification unit and the tissue classification unit respectively; the disease classification unit outputs the logits value of the disease indication through linear transformation, and the tissue classification unit outputs the logits value of the target tissue through linear transformation; the logits are converted into a probability distribution using the softmax function and optimized by a dynamic weighted loss function during training.

Images