A Method for Predicting the Flavors of Chinese and Western Medicines Based on Deep Learning and Chemical Language

Abstract

This invention provides a method for predicting the taste of traditional Chinese and Western medicines based on deep learning and chemical language. The method includes: S1: Initializing a pre-trained chemical language model, ChemBERTa, based on the RoBERTa architecture, as a semantic encoder to simplify the linear input canonical expression of molecules; S2: Constructing a standardized drug taste dataset covering the taste information of both traditional Chinese and Western medicines; S3: Building a basic pre-trained model for taste prediction based on the model framework in S1; S4: Employing a transfer learning strategy, retraining and domain-adapting the basic pre-trained model in S3 using the drug taste dataset constructed in S2 to obtain an intelligent drug taste prediction model; S5: Using the model in S4 to predict the taste of drug molecules. This method achieves deep semantic modeling of drug tastes and is a universal, interpretable, and scalable intelligent drug taste identification technology applicable to both traditional Chinese and Western medicines, contributing to the analysis of compatibility rules between traditional Chinese and Western medicines and precise clinical medication.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of modernization of traditional Chinese medicine and artificial intelligence, specifically involving an intelligent prediction method for the molecular flavors of Chinese and Western medicines based on a deep learning Transformer architecture and a chemical language pre-trained model. Background Technology

[0002] The theory of medicinal properties in Traditional Chinese Medicine (TCM) is a core component of the TCM system. The concept of "medicinal flavor" describes the scope of action and compatibility of drugs, serving as a crucial basis for clinical prescription design and efficacy regulation. Medicinal flavor not only influences the functional location and pharmacological effects of drugs in the body but also determines their synergistic and antagonistic relationships within compound prescriptions, significantly impacting the overall balance of therapeutic efficacy. However, current determination of medicinal flavor largely relies on traditional experience and literature records, lacking systematic and quantifiable modern research methods, making it difficult to meet the clinical needs of precise treatment. Especially with the increasing prevalence of combined TCM and Western medicine, the lack of clear medicinal flavor classification for Western medicines prevents their inclusion in compatibility assessments based on medicinal property theory, easily leading to biased medicinal properties and even causing antagonistic effects, disordered action, or cumulative toxic side effects, severely hindering the development of integrated TCM and Western medicine treatment.

[0003] In recent years, computer-aided drug molecule property prediction technology has provided a new path to solving this problem. Traditional methods mainly rely on experimental data and empirical formulas, or machine learning models based on molecular descriptors / fingerprints, including random forests, support vector machines, and K-nearest neighbors. While these methods reduce reliance on experiments to some extent, they are insufficient in capturing the complex structural features of drug molecules and their deep nonlinear relationships with drug properties, resulting in limited prediction accuracy and generalization ability.

[0004] With the rapid development of artificial intelligence technology, deep learning models based on natural language processing have provided powerful tools for pattern recognition of complex molecular properties. Unlike descriptor-based methods, models such as ChemBERTa borrow from the BERT (Bidirectional Encoder Representations from Transformers) architecture and are specifically designed for processing chemical language. This model is pre-trained on 77 million unlabeled compound SMILES sequences in the PubChem library and can automatically learn molecular fragments, bond connection patterns, and contextual semantic relationships. Compared with traditional molecular descriptors, it can capture higher-dimensional and more complex chemical structural features, providing significant advantages for molecular property prediction tasks.

[0005] However, existing research largely focuses on single-classification tasks or simple molecular attribute predictions, lacking a dedicated intelligent prediction framework for multi-classification and multi-labeling (single or multiple herbs) of traditional Chinese medicine (TCM) flavors. Furthermore, existing models struggle to accurately express the structural associations and semantic relationships between compound flavors, resulting in insufficient interpretability and limited extrapolation, making it difficult to balance scientific rigor with clinical applicability. Therefore, there is an urgent need to establish a systematic and quantifiable intelligent method for identifying medicinal flavors, unifying TCM and Western medicine molecules within the framework of pharmacological theories, and achieving a unified method for analyzing the flavors of both TCM and Western medicines. This would provide crucial technical support for evaluating the compatibility and efficacy optimization of TCM and Western medicine combinations.

[0006] The purpose of this invention is to overcome the problems of existing technologies, such as the lack of systematic and quantifiable methods for determining the taste of Chinese medicine, the difficulty in incorporating Western medicine into the theoretical framework of Chinese medicine properties, and the insufficient ability of existing computational models to capture the deep relationship between complex molecular structures and taste attributes. This invention proposes an intelligent prediction method and system for the taste of Chinese and Western medicine molecules based on deep learning and chemical language models. Summary of the Invention

[0007] To achieve the above objectives, this invention provides a method for predicting the flavors of traditional Chinese and Western medicines based on deep learning and chemical language, comprising the following steps:

[0008] S1: Load and initialize the ChemBERTa model (named deepchem / chemberta-77m-mlm), a pre-trained chemical language model based on the RoBERTa architecture. The deepchem / chemberta-77m-mlm model based on the RoBERTa architecture was pre-trained on the PubChem 77M molecular corpus with a large scale of masked language model and used as the basic semantic encoder for the chemical molecule SMILES structure.

[0009] S2: Obtain the taste and structural data of drug molecules from the virtual taste database, molecular taste database, organic small molecule bioactivity database, bitter taste database, pungent taste database, and plant molecular taste database. Perform deduplication and Pubchem ID matching, and merge the databases to construct a drug taste dataset including sour, bitter, sweet, pungent, salty, and bland taste subsets. Randomly divide it into three subsets: training set, validation set, and test set.

[0010] S3: After the final output layer of the ChemBERTa basic pre-trained model based on the RoBERTa architecture obtained in S1, add a linear fully connected classification layer. The input dimension of the linear classification layer is consistent with the hidden layer dimension of the pre-trained model, and the output dimension is the number of medicinal flavor categories, thus obtaining the basic pre-trained model for medicinal flavor prediction based on the RoBERTa architecture of ChemBERTa.

[0011] S4: Transfer learning is performed on the drug flavor dataset obtained in S2 and the pre-trained drug flavor prediction model based on the RoBERTa architecture obtained in S3. The loss function is calculated through the backpropagation algorithm to fine-tune the model parameters, so that the model can be adapted from general chemical language understanding ability to specific molecular drug flavor classification task. Its output changes from predicting the masked token to predicting the probability distribution of six types of drug flavors, thus obtaining a Chinese and Western medicine flavor prediction model based on deep learning and chemical language.

[0012] S5: The SMILES structure of the target drug molecule is analyzed by the Chinese and Western medicine flavor prediction model based on deep learning and chemical language obtained in S4. For each input molecule, a six-dimensional probability vector is output, which corresponds to the predicted probability of six flavors: sour, bitter, sweet, pungent, salty and bland. The final classification result is the flavor category with the highest probability value.

[0013] The deepchem / chemberta-77m-mlm model used in S1 of this invention is an open-source, widely used pre-trained model in chemical research. Based on the RoBERTa architecture and its accompanying AutoTokenizer, its core is a deep bidirectional network consisting of 12 attention heads and 6 layers of Transformer encoders, with a hidden state dimension of 384. This model was pre-trained on 77 million standardized SMILES sequences in the PubChem database, covering an extremely broad chemical structure space, ensuring the universality and diversity of the learned representations. Through pre-training, the model has learned deep contextual representations of the chemical language SMILES, possessing a powerful ability to understand the semantics of SMILES sequences, including: parsing molecular substructures, understanding inter-atomic connections, and learning implicit associations between certain chemical fragments and potential properties.

[0014] Based on the pre-trained model described above, by adding a linear classification layer on top of the pre-trained model and making adjustments, an efficient model can be created for the molecular medicinal flavor classification task.

[0015] In a preferred embodiment of the present invention, the acquisition path for the drug flavor data in S2 includes: acquiring data from the following databases: Virtual Taste Database (https: / / insilico-cyp.charite.de / VirtualTaste), Molecular Taste Database (https: / / zenodo.org / records / 6528835), Organic Small Molecule Bioactivity Data (PubChem, https: / / pubchem.ncbi.nlm.nih.gov), Bitter Taste Database (https: / / bitterdb.agri.huji.ac.il / dbbitter.php), Spicy Taste Database (http: / / www.pungentdb.org.cn / home), and Plant Molecular Taste Database (https: / / plantmoleculartastedb.org / index.php);

[0016] Dataset collection: A virtual taste database was used to collect sour, bitter, sweet, and salty tastes; a molecular taste database was used to collect sour, bitter, sweet, spicy, salty, and bland tastes; PubChem was used to collect sour, bitter, sweet, spicy, and bland tastes; a bitter taste database was used to collect bitter tastes; a spicy taste database was used to collect spicy tastes; and a plant molecular medicinal taste database was used to collect molecules with sour, bitter, sweet, and spicy tastes. After downloading and deduplicating the above molecules with clear sour, bitter, sweet, spicy, salty, and bland tastes, all of them were matched with PubChem IDs to remove molecules that could not be matched, ensuring the accuracy and traceability of the data.

[0017] Data integration: By merging the datasets from the virtual taste database, molecular taste database, PubChem, bitter taste database, pungent taste database, and plant molecular medicinal taste database, a dataset of 3686 chemical components covering 6 medicinal tastes was constructed, including 464 sour tastes, 984 bitter tastes, 972 sweet tastes, 337 pungent tastes, 617 salty tastes, and 312 bland tastes.

[0018] In a preferred embodiment, the drug flavor dataset described in S2 needs to be preprocessed and data augmented before pre-training, that is,

[0019] Preprocessing: Data containing SMILES and medicinal taste labels collected from virtual taste database, molecular taste database, PubChem, bitter taste database, spicy taste database, and plant molecular medicinal taste database are preprocessed, including: normalizing SMILES using RDkit and mapping text labels to numerical IDs.

[0020] Dataset partitioning: The data was divided into training, validation and test sets in an 8:1:1 ratio using stratified sampling.

[0021] Data Augmentation: To address the class imbalance problem, data augmentation was performed on classes with fewer samples in the training set (such as "spicy" and "mild") using the SMILES randomization algorithm based on RDKit. Specifically, this method was used to generate random, chemically equivalent SMILES representations for the minority samples. Multiple new samples were generated for each minority class sample using this method until the total number of samples in all classes equaled the number of samples in the largest class. After augmentation, the number of minority class samples increased significantly, and the model's understanding of molecular structure invariance was improved.

[0022] In a preferred embodiment of the present invention, the linear classification layer described in S3 employs the Softmax activation function to output the probability distribution of the six medicinal flavors; during model training, the cross-entropy loss function is set as the objective function to evaluate the deviation between the predicted results and the true labels; simultaneously, the AdamW optimizer is used to update parameters, and by decoupling the weight decay and adaptive learning rate mechanism, all trainable parameters in the model are jointly optimized, including all Transformer layers of the pre-trained model and the newly added linear classification layer, thereby minimizing the loss function and enhancing the convergence stability of the model.

[0023] In a preferred embodiment, the Softmax activation function is: ,

[0024] in, For the model to the first Linear output resembling medicinal flavor; For the input molecule to belong to the first Predicted probability of medicinal flavor.

[0025] The cross-entropy loss function is: ,

[0026] in, For the first The true label of the medicinal flavor; used to measure the difference between the classification prediction and the true label.

[0027] In a preferred embodiment, the adaptive initial learning rate of the AdamW optimizer is set to 5 × 10⁻⁶. -4 The weight decay rate is set to 1×10 -4 The optimizer updates the parameters of both the classification layer and all layers of the pre-trained model simultaneously, following the following parameter update rules:

[0028] ,

[0029] in, for The parameters for the next iteration For learning rate, This is the weight decay coefficient. and These are the first-order and second-order moment estimates, respectively. Set to 1×10 -8 This is used to avoid the denominator being zero. The above optimization strategies can effectively improve the model's training efficiency, convergence speed, and generalization performance.

[0030] In a preferred embodiment, the transfer learning strategy described in S4 of this invention involves structural expansion and parameter fine-tuning based on a pre-trained model. Specifically, a linear classification layer is added after the final output layer of the pre-trained model ChemBERTa. This classification layer is a fully connected structure, with its input dimension matching the hidden layer dimension of the pre-trained model. Its output dimension represents the number of the six medicinal flavor labels required by this invention, used to map the general chemical language representation to the probability distribution of specific medicinal flavor categories. A learning rate of 5×10⁻⁶ is used during training. -4 AdamW optimizer (weight decay rate 1×10) -4 Meanwhile, it updates all layers of the pre-trained Transformer model and the newly added classification layer, thereby promoting the rapid migration of the model from "general chemical knowledge" to "specific medicinal flavor identification task", and transforming the model's output from the prediction of mask tokens to the classification output of six-dimensional medicinal flavor probabilities.

[0031] In a preferred implementation, S4 involves fine-tuning the model using an enhanced medicinal flavor dataset. During this fine-tuning, all parameters of ChemBERTa are adjusted according to the classification task objective using backpropagation and gradient descent to better adapt the model to the flavor classification task. The model output is fed into a Softmax function and a linear classification layer to complete the multi-class classification task. Ultimately, the model learns to accurately associate the input SMILES sequences with their corresponding flavor labels.

[0032] In a preferred embodiment, in the transfer learning described in S4, 100 epochs are set. After each epoch, the model performance is evaluated using a validation set, and multiple metrics such as accuracy, precision, recall, Matthews coefficient, and ROC-AUC are calculated. When the accuracy performance on the validation set exceeds the previous best record, the current model is saved. Early stopping and learning rate reduction strategies are adopted to prevent overfitting and optimize the training process, resulting in a pre-trained model for predicting the flavors of Chinese and Western medicines based on deep learning and chemical language with the best generalization ability.

[0033] More preferably, the early stopping strategy is set based on the validation set accuracy with a patience value of 5; the learning rate reduction strategy is set based on the validation set accuracy with a factor of 0.5 and a patience value of 5.

[0034] After fine-tuning, the best model saved during the validation process is used to perform a final performance evaluation on the test set to obtain an unbiased estimate of the model's generalization ability. All model parameters, word segmenters, training logs, evaluation metrics, and intermediate results are saved for subsequent analysis and reproduction.

[0035] To evaluate the performance of the developed model, a comprehensive set of performance metrics were employed, including accuracy, precision, recall, Matthews coefficient, and ROC AUC score. Classification reports and confusion matrices were also generated for each category to facilitate in-depth analysis of the predictive performance for different medicinal flavor categories.

[0036] To verify the advancement and practical value of the method of this invention, various traditional machine learning models, including random forest models and fully connected neural network models, were trained and compared on the same computing platform (GPU RTX 5070Ti 16GB ×1, CPU Intel® Ultra 7 processor 265K @ 5.50GHz, memory 96GB, storage 1TB). All the models compared were implemented using Python Scikit-learn and PyTorch, and were trained and evaluated under the same experimental settings to further demonstrate the performance advantage of the Transformer model in this invention.

[0037] This invention classifies molecular medicinal flavors using the principle of maximizing probability. The model outputs a six-dimensional probability vector for each input molecule, corresponding to the predicted probabilities of six medicinal flavors: sour, bitter, sweet, pungent, salty, and bland. The final flavor classification result is the category with the highest probability value.

[0038] Secondly, this invention also provides a system for predicting the flavors of traditional Chinese and Western medicines using the aforementioned method based on deep learning and chemical language. The system includes the following modules:

[0039] (1) Drug Flavor Dataset Module, which is used to acquire the drug flavor dataset and update, enhance and divide it into sub-datasets;

[0040] (2) Chinese and Western medicine flavor prediction module, the Chinese and Western medicine flavor prediction pre-training module is the Chinese and Western medicine flavor prediction model based on deep learning and chemical language obtained after the deepchem / chemberta-77m-mlm model based on the RoBERTa architecture described in S1, the structure construction in S3 and the transfer learning and fine-tuning in step S4.

[0041] (3) Drug molecule input module, which is used to input the drug molecule name of the drug flavor to be predicted and enter the Chinese and Western medicine flavor prediction module for prediction.

[0042] The beneficial technical effects of this invention are reflected in:

[0043] Firstly, in the deep learning and chemical language-based system for predicting the flavors of traditional Chinese and Western medicines, and the method for using this system to predict the flavors of traditional Chinese and Western medicines provided in this invention, the system integrates multiple professional databases, including virtual flavor databases, molecular flavor databases, PubChem databases, bitter flavor databases, pungent flavor databases, and plant molecular flavor databases. This system constructs a standardized molecular dataset covering six categories of flavors: sour, bitter, sweet, pungent, salty, and bland, with a total of 3686 compounds. This dataset possesses both scale advantages and ensures the reliability and consistency of label sources. The dataset systematically statistically analyzes and coordinates the sample size and proportions for different flavor categories, effectively addressing the long-standing lack of high-quality benchmark datasets in the field of flavor research. The construction of this dataset provides a solid data foundation for model training, significantly improving the scientific rigor, stability, and generalization ability of the prediction model, enabling the model to exhibit good reproducibility and comparability under different data sources and experimental conditions.

[0044] Secondly, this invention effectively addresses the problem of uneven distribution of medicinal flavor data categories by introducing a SMILES-based data augmentation strategy. To address the scarcity of samples for some medicinal flavor categories, this invention utilizes the RDKit tool during model training to randomize the SMILES structures of molecules. While maintaining strict chemical equivalence, the traversal order and starting atoms are randomly changed, generating 3-10 semantically consistent but differently arranged SMILES sequences for each molecule. Augmented samples participate in model training alongside the original samples, maintaining consistent labels. This significantly expands the number of minority class samples without introducing chemical noise, resulting in a more balanced data distribution. This augmentation strategy improves the model's robustness to different representations, prevents bias towards high-frequency categories, and further enhances the model's generalization ability in multi-class medicinal flavor prediction tasks.

[0045] Furthermore, this invention does not train the model from scratch. Instead, it uses ChemBERTa, a masked language model pre-trained on a massive dataset of PubChem compound SMILES sequences, as the base model. This fully leverages ChemBERTa's learned deep chemical language features to achieve effective transfer of chemical knowledge. By adding a linear classification layer on top of the pre-trained model and fine-tuning it using the medicinal flavor dataset constructed in this invention, the model can quickly adapt to multi-classification tasks related to medicinal flavors, significantly reducing its dependence on the scale of labeled data, accelerating model convergence, and achieving predictive performance superior to traditional machine learning methods. Particularly in capturing the deep, implicit relationships between molecular structures and complex medicinal flavor attributes, the transfer-learned model performs exceptionally well, significantly improving the accuracy, stability, and interpretability of medicinal flavor prediction. Attached Figure Description

[0046] Figure 1 A flowchart of a method for predicting the flavors of traditional Chinese and Western medicines based on deep learning and chemical language;

[0047] Figure 2 A schematic diagram of the structure of the ChemBERTa-based model for predicting the flavor of Chinese and Western medicines. Detailed Implementation

[0048] The present invention will be further described below with reference to specific embodiments, and the advantages and features of the present invention will become clearer as a result of the description. However, these embodiments are merely exemplary and do not constitute any limitation on the scope of protection defined by the claims of the present invention.

[0049] Figure 1 A flowchart of the method for predicting the taste of Chinese and Western medicines based on deep learning and chemical language according to the present invention is provided. The following specific embodiments illustrate the construction process, performance evaluation, and implementation of the prediction model for the taste of Chinese and Western medicines based on deep learning and chemical language of the present invention. Those skilled in the art can easily understand other advantages and functions of the present invention from the content disclosed in this specification.

[0050] Example 1: Construction of a Prediction Model for the Flavors of Chinese and Western Medicines Based on Deep Learning and Chemical Language

[0051] I. Introduction and Architecture Description of the ChemBERTa Model

[0052] 1. Model Introduction and Functional Positioning

[0053] The ChemBERTa model (model name: deepchem / chemberta-77m-mlm) is built on the RoBERTa framework and is a deep bidirectional Transformer encoder for chemical language modeling, specifically designed to learn contextual semantic representations of chemical structures from SMILES strings. Following the RoBERTa design specifications, this model can bidirectionally encode input molecular sequences, capturing the dependencies between atoms, functional groups, and chemical bonds. As a general-purpose chemical language model, it does not directly perform medicinal flavor prediction tasks but acts as a deep feature extractor, providing high-level semantic features for downstream molecular attribute prediction. Its main value lies in providing a powerful starting point for feature extraction in downstream tasks. By inputting the SMILES string of a molecule, the model can output the [CLS] vector in the last hidden state or the average representation of all tokens, serving as a high-dimensional semantic embedding for the molecule, used in subsequent classification or regression tasks.

[0054] The overall architecture of the ChemBERTa model consists of an input layer, a stacked Transformer encoder layer, and an output layer. The specific architecture is as follows:

[0055] (1) Input layer

[0056] The model's input layer employs a Byte-Pair Encoding (BPE) tokenizer optimized for chemical language features, combined with an embedding structure. The tokenizer first decomposes the input SMILES string (e.g., "CC(=O)O") into chemically meaningful sub-word units, such as ["C", "C", "(", "=O", ")", "O"], enabling the model to learn the semantic information of chemical fragments such as carbon atoms, carbonyl groups, and aromatic bonds at the molecular structure level. Subsequently, each token is mapped to a 384-dimensional dense vector through the embedding layer, with absolute positional encoding added to reflect the relative positions of tokens in the sequence. The final input vector is obtained by adding the token embeddings and positional embeddings, forming the initial input features for each token. The maximum input length of the model is set to 128 to ensure uniform input sequence length and computational stability.

[0057] (2) Transformer encoder stacking

[0058] The core computational part of the model consists of six stacked Transformer encoder layers. Each layer includes:

[0059] a) Multi-head self-attention mechanism: 12 attention heads are set up with a hidden dimension of 384, so the dimension of each attention head is 32. This mechanism allows each token in the sequence to establish dependencies with other tokens, thereby capturing complex molecular structural context.

[0060] b) Feedforward neural network layer: A two-layer fully connected network is used. The first layer expands the dimension from 384 to 3072 and the activation function is GELU. The second layer projects the dimension back from 3072 to 384.

[0061] c) Residual Connections and Layer Normalization: Each sub-layer (self-attention layer and feedforward layer) incorporates residual connections and layer normalization structures to stabilize deep network training and mitigate gradient vanishing. The calculation formulas are as follows: ,

[0062] in, For the first SubLayer represents the attention layer or feedforward layer.

[0063] (3) Output layer

[0064] After layer-by-layer information fusion and context modeling through a 6-layer Transformer encoder, the model's output layer generates a 384-dimensional context-aware vector for each token in the input sequence. During pre-training, the model predicts randomly masked tokens using a masking language modeling task, thereby learning the structural relationships and semantic connections within molecules. In downstream tasks, the hidden states corresponding to the [CLS] markers in the last layer can be directly used as the semantic representation of the entire molecule and input into subsequent classification layers to achieve tasks such as flavor prediction or other molecular attribute determination.

[0065] 2. Corpus Sources and Semantic Learning Mechanism of the ChemBERTa Model

[0066] The ChemBERTa model (model name: deepchem / chemberta-77m-mlm) underwent large-scale self-supervised pre-training on a publicly available chemical molecular corpus to learn the contextual rules of chemical language and the semantics of molecular structures. This pre-training process utilized the 77M SMILES dataset from the PubChem public database. This dataset contains approximately 77 million standardized organic small molecule SMILES strings, covering a wide range of structural types including alkanes, aromatics, heterocycles, metal complexes, amino acids, and drug molecules, exhibiting extremely high chemical diversity and representativeness.

[0067] During pre-training, the model does not rely on traditional manual feature descriptors, but directly learns the linguistic patterns of SMILES symbol sequences. The model first constructs a chemical language vocabulary using the byte-pair encoding (BPE) algorithm. This vocabulary is a set of sub-word units automatically learned from a large-scale SMILES corpus, such as "C", "Cl", "Br", "=O", "(", ")", etc., which can decompose complex molecular structures into chemically meaningful fragments, enabling the model to recognize local structural information such as carbon chains, substituents, and functional groups.

[0068] The pre-training task employs a masked language modeling mechanism, consistent with the training method of the BERT model in natural language processing. Specifically, given a SMILES string (e.g., "acetaminophen CC(=O)Nc1ccc(O)cc1"), approximately 15% of the token positions are randomly selected for masking, for example, changing the input to "CC(=O)Nc1ccc[MASK]cc1". The model's goal is to predict the original token masked by the [MASK] tag (which should be "O" in this case). Through this self-supervised task, the model learns how to infer omitted structural units within a chemical context, thereby obtaining the potential dependencies between atoms, chemical bonds, and functional groups.

[0069] Through the aforementioned pre-training, the model can automatically capture the grammatical rules and chemical structural semantic information of SMILES sequences, learn deep representation capabilities of molecular structural composition, functional group pairings, and topological features, and form a unified chemical language embedding space. The deep learning results of this model at the chemical language level can be directly transferred to the task of medicinal flavor prediction, providing a solid feature foundation for unified semantic modeling, medicinal flavor recognition, and combined drug mechanism analysis of traditional Chinese medicine and Western medicine.

[0070] 3. Construction and Augmentation of Drug Flavor Dataset

[0071] This embodiment is based on the medicinal flavor characteristics information of drugs and natural molecules reported in existing public databases. It systematically integrates data from multiple authoritative sources, including: virtual flavor database, molecular flavor database, PubChem, bitter flavor database, pungent flavor database, and plant molecular medicinal flavor database. The databases are merged to construct a medicinal flavor dataset that includes six categories of medicinal flavor labels: sour, bitter, sweet, pungent, salty, and bland.

[0072] (1) Specific data collection path for medicinal taste datasets: Sour, bitter, sweet and salty tastes are collected using a virtual taste database; sour, bitter, sweet, spicy, salty and bland tastes are collected using a plant molecular medicinal taste database; sour, bitter, sweet, spicy and bland tastes are collected using PubChem; bitter taste is collected using a bitter taste database; spicy taste is collected using a spicy taste database; and molecules of sour, bitter, sweet and spicy tastes are collected using a plant molecular medicinal taste database.

[0073] (2) Data Integration: To ensure the consistency and traceability of the same compound across different databases, this embodiment uses PubChem ID as a unified identifier for cross-database matching. Specifically, the SMILES strings of compounds from each database are processed using the API provided by PubChem to obtain a unique PubChem ID for each compound. When compounds from multiple databases correspond to the same PubChem ID, they are considered the same molecule and their medicinal flavor label information is merged to avoid duplicate statistics. If a compound cannot match a valid PubChem ID (e.g., due to incomplete structural formulas or non-standardized expressions), it is removed. Through this process, the integrated medicinal flavor data can be guaranteed to have high accuracy, consistency, and traceability, providing a reliable data foundation for subsequent model training. After cleaning and merging the virtual taste database, molecular taste database, PubChem, bitter taste database, pungent taste database, and plant molecular medicinal taste database, a dataset of 3686 chemical components covering 6 medicinal tastes was constructed, including 464 sour tastes, 984 bitter tastes, 972 sweet tastes, 337 pungent tastes, 617 salty tastes, and 312 bland tastes.

[0074] (3) Data set partitioning: Stratified sampling is used to divide the data into training set, validation set and test set in a ratio of 8:1:1.

[0075] (4) Data Augmentation: To address the class imbalance problem, this embodiment introduces the SMILES randomization algorithm (Randomized SMILESAugmentation) based on RDKit to augment data for categories with fewer samples in the training set (such as "spicy" and "bland"). The core of the SMILES randomization augmentation mechanism lies in structural invariance, that is, although different random SMILES have different symbol arrangements, their corresponding chemical topological structures are completely consistent. This algorithm generates multiple equivalent SMILES expressions while maintaining the chemical structure by changing the molecular traversal starting point and encoding order. For example, for the molecule "C6H5OH (phenol)," its canonical SMILES is "c1ccc(cc1)O," which can generate multiple equivalent forms after randomization, such as "Oc1ccccc1" and "c1cc(O)ccc1." In this embodiment, 3-10 random SMILES structures are generated for each molecule in the "spicy" and "bland" categories until the total number of samples is close to that of the most popular category ("bitter"). Using this method, the number of samples in the spicy category increased from 337 to approximately 950, and the number of samples in the mild category increased from 312 to approximately 940, achieving a basic balance in the number of samples between the categories.

[0076] II. Implementation of Transfer Learning and Training for ChemBERTa-Based Herbal Flavor Prediction Model

[0077] 1. Model structure adjustment

[0078] During the transfer learning phase, the output layer of the ChemBERTa model was structurally modified. Specifically, a fully connected linear classification layer was added after the last Transformer encoding layer of the original model. This layer has an input dimension of 384 and an output dimension of 6, corresponding to the six medicinal taste categories: sour, bitter, sweet, pungent, salty, and bland. This layer outputs the probability distributions of the six medicinal taste categories after applying a Softmax activation function.

[0079] ,

[0080] in, For the model to the first Linear output resembling medicinal flavor; For the input molecule to belong to the first Predicted probability of medicinal flavor.

[0081] The model uses the cross-entropy loss function as the optimization objective:

[0082] ,

[0083] in, For the first The true label of the medicinal flavor; used to measure the difference between the classification prediction and the true label.

[0084] The optimization algorithm uses AdamW with a learning rate of 1×10⁻⁶. -4 and weight decay coefficient 1×10 -4 The training parameters are updated to ensure stable convergence of the model with a limited number of samples and to prevent overfitting. The transfer learning strategy employs a full model fine-tuning approach, which involves simultaneously updating the parameters of the pre-trained backbone layer and the newly added classification layer during training. The parameter update rules are as follows: ,

[0085] in, for The parameters for the next iteration For learning rate, This is the weight decay coefficient. and These are the first-order and second-order moment estimates, respectively. Set to 1×10 -8 This mechanism is used to avoid zero denominators. While maintaining the characteristic representation of chemical language, it stably adjusts the model parameters, enabling the model to quickly adapt to the medicinal flavor classification task.

[0086] 2. Input data encoding

[0087] Each input SMILES molecule sample is first sub-encoded using the BPE segmenter built into the ChemBERTa model, with a maximum input length of 128 to ensure consistency and computational efficiency for molecule input sequences of different lengths. During encoding, the SMILES string is decomposed into several chemically meaningful tokens (such as "C", "(", "=O", ")", "O"). Each token is mapped to a 384-dimensional dense vector through an embedding layer and then input into a Transformer encoding stack for contextual feature extraction. The model encodes layer by layer through a 6-layer multi-head self-attention mechanism and a feedforward network structure, outputting a high-dimensional molecule vector containing global semantic information. For flavor classification, the hidden state corresponding to the [CLS] label is taken as the input of the overall molecular semantic representation to the classification layer, and finally the predicted probability of each flavor category is output.

[0088] 3. Training parameter settings

[0089] The model training process employs batch stochastic gradient descent with a batch size of 16 and a total of 100 training epochs. Data is divided into training, validation, and test sets in an 8:1:1 ratio. During training, after each epoch, comprehensive metrics such as accuracy, precision, recall, Matthews coefficient, and ROC-AUC are calculated for the validation set to dynamically evaluate model performance. To improve training stability and generalization ability, a dynamic learning rate scheduling mechanism is introduced: the learning rate is automatically halved when the validation set accuracy shows no improvement for 5 consecutive iterations; if there is no performance improvement for 15 consecutive iterations, an early stopping mechanism is triggered to terminate training. This mechanism effectively prevents overfitting and ensures smooth and efficient model convergence. When the validation set accuracy exceeds the historical best record, the system automatically saves the current parameter state as the optimal model for subsequent deployment and prediction.

[0090] 4. Model Performance Evaluation

[0091] After training, the model is systematically evaluated on both the validation and test sets. The model output is a six-dimensional probability vector (e.g., [0.06, 0.70, 0.12, 0.05, 0.04, 0.03]), which corresponds to the predicted probabilities of six categories of medicinal tastes: sour, bitter, sweet, pungent, salty, and bland. The category with the highest probability value is the model's prediction result.

[0092] Figure 2 A schematic diagram of the structure of the Chinese and Western medicine flavor prediction model based on ChemBERTa of this invention is given.

[0093] Example 2. Performance evaluation of the medicinal flavor prediction model

[0094] To comprehensively evaluate the performance of the ChemBERTa model based on the Transformer architecture on the task of predicting molecular medicinal flavors, this invention trained typical machine learning models, random forest and deep learning models (fully connected neural networks), as baseline models based on a constructed standardized six-flavor dataset, and compared and analyzed the prediction performance with that of the ChemBERTa model.

[0095] 1. As a comparison, the random forest model is built on the Scikit-learn framework, with input features being a 2048-bit Morgan fingerprint (radius=2) generated by RDKit. The model consists of 1000 decision trees, using the Gini index as the node splitting criterion. Finally, a majority voting mechanism is used to predict the six categories of medicinal flavors.

[0096] 2. Prediction of medicinal flavor using fully connected neural networks

[0097] The fully connected neural network employs a four-layer fully connected structure, including an input layer with a 2048-bit Morgan fingerprint (radius = 2), three hidden layers (256-dimensional, 128-dimensional, and 64-dimensional respectively, each using ReLU activation, batch normalization, and Dropout regularization with a dropout rate of 0.3), and an output layer (6-dimensional). The model uses the AdamW optimizer (learning rate 0.0005, weight decay coefficient 1×10⁻⁶). -4 The training was performed using the cross-entropy loss function with a batch size of 64. During training, the ReduceLROnPlateau learning rate scheduling strategy (decline factor of 0.5 and patience value of 5) and early stopping mechanism (patience value of 15) based on the validation set accuracy were adopted. The input data was preprocessed with standardization, and synthetic minority class oversampling was used on the training set to solve the class imbalance problem.

[0098] 3. Pre-trained model for predicting the flavors of Chinese and Western medicines based on deep learning and chemical language

[0099] The ChemBERTa model used in this invention is based on the RoBERTa architecture and is a Transformer model specifically optimized for SMILES representation in the chemical domain. The core structure of this model consists of six stacked Transformer encoder layers, each containing 12 self-attention heads. The hidden state dimension is 384, and the feedforward network dimension is 1536. The model vocabulary size is 520, and the maximum sequence processing length is 512 tokens. The model input is a normalized SMILES string, which is converted into a 384-dimensional vector representation through a learnable embedding layer, and positional encoding is added to preserve sequence order information. The model algorithm employs a masked language model pre-training strategy, pre-trained on SMILES sequences of 77 million PubChem compounds.

[0100] For the medicinal flavor classification task of this invention, a fully connected classification layer with an output dimension of 6 (384×6 weight matrix and 6-dimensional bias vector) was added on top of the pre-trained model. A learning rate of 5×10⁻⁶ was used during the fine-tuning phase. -4 AdamW optimizer (weight decay 1×10) -4 The model uses the cross-entropy loss function and is fine-tuned end-to-end over 100 training epochs. During training, the model employs an early stopping mechanism (patience value 15) and a learning rate reduction strategy (based on validation set accuracy, factor 0.5, patience value 5), ultimately achieving an accuracy of 0.862 and an ROC AUC of 0.974 on the test set.

[0101] 4. Indicator Evaluation

[0102] The results are shown in Table 1. The ChemBERTa-based Transformer model proposed in this invention exhibits optimal or near-optimal overall performance across multiple key evaluation metrics. Specifically:

[0103] In terms of overall accuracy, the Transformer model consistently outperformed fully connected neural networks (0.816, 0.837) and random forests (0.836, 0.800) on both the validation and test sets (0.854, 0.862), indicating that its predictions were the most reliable. Regarding accuracy and recall, the Transformer model achieved the highest values ​​on the test set (0.856, 0.825), achieving the best balance between accurate identification and broad coverage. In contrast, while the random forest had the highest precision on the validation set (0.874), its recall on the test set was significantly lower (0.721), indicating overfitting and insufficient generalization ability. The Transformer model achieved the highest values ​​on both the validation and test sets (0.817, 0.827), significantly outperforming the baseline model. This fully demonstrates that its classification results are highly reliable across all categories and are minimally affected by class imbalance. All models performed well in terms of ROC AUC values, but the Transformer model still ranked first with excellent performance of 0.975 on the validation set and 0.974 on the test set, demonstrating its strong class discrimination ability.

[0104] The reason why the ChemBERTa model based on the Transformer architecture significantly outperforms random forests and fully connected neural networks in the task of predicting medicinal flavors is the synergistic effect of its chemical language-level representation ability, structural invariant learning mechanism, and robust parameter optimization strategy. ChemBERTa uses SMILES sequences as input and captures long-range intramolecular dependencies through BPE subtotalization and multi-head self-attention mechanisms. This allows the model to simultaneously understand high-dimensional structural semantics such as functional group combinations, aromatic ring substitutions, and molecular topology, forming a globally context-aware molecular embedding. Random forests / fully connected neural networks, using Morgan fingerprints, only consider the presence / absence of local neighborhoods, making it difficult to express the equivalence of long-range relationships across bonds / rings and "different traversal orders of the same structure." Furthermore, random SMILES data augmentation introduces a multi-sequence perspective of equivalent structures during training, enabling the model to learn the structural invariance of "different symbol sequences - same structure - same semantics," fundamentally improving the model's robustness to input perturbations and its ability to distinguish between classes. In contrast, the fingerprints of random forests / fully connected neural networks are inherently insensitive to order but lack fine-grained contextual semantics, making it difficult to form clear decision boundaries for minority classes. At the optimization level, the AdamW optimizer and dynamic learning rate scheduling mechanism are introduced, enabling the model to achieve stable convergence while maintaining high expressiveness and avoiding overfitting. It is this series of designs that enable the model to achieve high consistency across classes in overall accuracy, demonstrate excellent balance and resistance to class imbalance in the Matthews coefficient, and exhibit the strongest class discrimination in ROC AUC, fully demonstrating the model's deep advantages in chemical semantic understanding and molecular representation.

[0105] In summary, although different models have varying performance on certain individual metrics, the ChemBERTa-based Transformer model proposed in this invention significantly outperforms the baseline machine learning and deep learning models compared to the comparison in most core metrics, especially in the comprehensive performance metrics of Matthews coefficient and stability. This fully demonstrates that the model possesses excellent comprehensive capabilities, robustness, and generalization ability in the multi-classification prediction task of molecular medicinal flavors.

[0106] Table 1. Performance comparison of Transformer with other baseline models in predicting medicinal flavors

[0107]

[0108] Example 3: Model Performance Evaluation Based on External Validation Sets of Traditional Chinese Medicine

[0109] To systematically evaluate the performance and applicability of the constructed medicinal flavor prediction model, this invention uses traditional Chinese medicine (TCM) as the validation object to conduct external model validation. Considering that TCM is typically composed of multiple chemical components, its overall flavor is largely a comprehensive expression of the taste characteristics of each component; and that different components contribute differently to the flavor, this invention achieves medicinal flavor prediction by assigning weights to the components and employing a weighted average strategy. Regarding weight settings, the weights of the main components are referenced from the relevant records of indicative components in the 2025 edition of the Chinese Pharmacopoeia, and the weight combination of the main and secondary components is optimized based on a grid search traversal: all possible weight allocation combinations are systematically traversed within the range of 0-1 with a step size of 0.1. For each weight ratio, the comprehensive performance index of the model on the validation set is calculated, and finally, the optimal weight configuration (i.e., a weight ratio of 1:9 for the main component and the secondary component) is selected as the standard parameter for the final prediction model, thereby improving the accuracy of medicinal flavor prediction.

[0110] The model performance evaluation employs a multi-class matching strategy based on the top three prediction results. A true positive result is considered if the actual flavor of the target herb appears in the top three predicted by the model (each herb in the 2025 edition of the Chinese Pharmacopoeia has a maximum of three flavors); a false negative result is considered if the actual flavor does not appear in the top three. Furthermore, if the model predicts a flavor that the herb does not actually possess, and that flavor appears in the top three, it is considered a false positive; if that flavor does not appear in the top three, it is considered a true negative. This evaluation strategy takes into account the characteristics of multiple flavor combinations recorded in the pharmacopoeia and can effectively measure the model's practical applicability.

[0111] The external validation results are shown in Table 2. The model exhibits excellent performance on both the validation and test sets. Table 3 shows the prediction results for the flavors of some representative traditional Chinese medicines. In terms of accuracy, the model achieves 0.929 and 0.930 on the validation and test sets, respectively, indicating extremely high overall accuracy in predicting the flavors of traditional Chinese medicines. In terms of precision, the model achieves 0.944 and 0.927 on the validation and test sets, respectively, indicating a high proportion of accurate predictions of the actual flavors. In terms of recall, the model achieves exceptionally high levels of 0.981 and 0.993 on the validation and test sets, respectively, demonstrating its excellent ability to capture and identify the true flavor characteristics of traditional Chinese medicines with an extremely low false negative rate. The ROC AUC value remains stable at around 0.795, indicating good classification and discrimination capabilities.

[0112] Table 2. Performance evaluation results of the model based on the external validation set of traditional Chinese medicine.

[0113]

[0114] Table 3. Medicinal Flavors of Representative Traditional Chinese Medicines Predicted by the Model

[0115]

[0116] Example 4: Prediction and Analysis of the Flavor of Western Medicine

[0117] To explore the "medicinal properties" of Western medicines, this study used an established medicinal taste prediction model to systematically predict and verify the medicinal taste attributes of some Western medicines reported in existing literature (the results are detailed in Table 4). The prediction results showed that: aspirin was predicted to be bitter (probability 0.999), consistent with the description of bitterness as the main medicinal taste in the literature's description of "pungent, bitter, and sour"; penicillin was predicted to be bitter (probability 1.000), completely matching the description of "bitter" in the literature; doxorubicin was predicted to be bitter (probability 0.999), consistent with the description of "pungent and bitter" in the literature; hydrochlorothiazide was predicted to be bitter (probability 0.993), consistent with the characteristic of bitterness as the main taste in the literature's description of "bitter and sweet"; irinotecan and gefitinib were both predicted to be bitter (probabilities 1.000 and 1.000, respectively), consistent with the literature. It is worth noting that cefadroxil was also predicted to be bitter (probability 0.987), further supporting the characteristic that most antibiotics have a bitter taste.

[0118] Table 4. Comparison of Western medicine flavor prediction results with literature.

[0119] .

Claims

1. A method for predicting the flavors of traditional Chinese and Western medicines based on deep learning and chemical language, characterized in that, Includes the following steps: S1: Introduce and initialize the ChemBERTa pre-trained chemical language model based on the RoBERTa architecture. The ChemBERTa pre-trained chemical language model was obtained by pre-training on the PubChem 77M molecular corpus and is used as the basic semantic encoder for the SMILES structure of chemical molecules. S2: Obtain the taste and structural data of drug molecules from the virtual taste database, molecular taste database, organic small molecule bioactivity database Pubchem, bitter taste database, pungent taste database, and plant molecular taste database. Perform deduplication and Pubchem ID matching, and merge the data to construct a drug taste dataset including six sub-datasets: sour, bitter, sweet, pungent, salty, and bland. Randomly divide the obtained dataset into training set, validation set, and test set. S3: After the final output layer of the initialized ChemBERTa base model obtained in S1, a linear fully connected classification layer is constructed and superimposed. The input dimension of the linear fully connected classification layer is consistent with the hidden layer dimension of the pre-trained model, and the output dimension is the number of medicinal flavor categories, thus obtaining the basic model for medicinal flavor prediction based on the RoBERTa architecture. In the linear fully connected classification layer in S3, a softmax activation function is set to output the medicinal flavor probability distribution value, a cross-entropy loss function is set to evaluate the accuracy of medicinal flavor prediction, and the AdamW optimizer algorithm is set to decouple weight decay and adaptive learning rate adjustment. The softmax activation function is: ， in, For the model to the first Linear output resembling medicinal flavor; For the input molecule to belong to the first Predicted probability of medicinal flavors The cross-entropy loss function is: ， in, For the first The actual label of the medicinal flavor; used to measure the difference between the classification prediction and the actual label; The AdamW optimizer has an adaptive learning rate of 5×10⁻⁶. -4 The weight decay rate is 1×10 -4 Simultaneously, the parameters of the added classification layer and all layers of the pre-trained model are updated, and the parameter update rules are as follows: ， in, for The parameters for the next iteration For learning rate, This is the weight decay coefficient. and These are the first-order and second-order moment estimates, respectively. Set to 1×10 -8 Used to avoid a denominator of zero; S4: Transfer learning is performed on the drug flavor dataset obtained in S2 and the basic drug flavor prediction model obtained in S3. The loss is calculated and the model parameters are fine-tuned through the backpropagation algorithm, so that the model is adapted from the general chemical language understanding ability to the specific drug flavor classification task. Its output changes from predicting the masked token to predicting the probability of the drug flavor category, thereby obtaining a Chinese and Western medicine flavor prediction model based on deep learning and chemical language. S5: The Chinese and Western medicine taste prediction model based on deep learning and chemical language obtained in S4 predicts the taste of the target drug molecule. For each input molecule, a six-dimensional probability vector is output, which corresponds to the predicted probability of six tastes: sour, bitter, sweet, pungent, salty, and bland. The final classification result is the taste category with the highest probability value.

2. The method for predicting the flavors of traditional Chinese and Western medicines based on deep learning and chemical language according to claim 1, characterized in that, The ChemBERTa pre-trained model for chemical language based on the RoBERTa architecture described in S1 is a deep bidirectional network with 12 attention heads and 6 layers of Transformer encoders stacked together, and a hidden state dimension of 384.

3. The method for predicting the flavors of traditional Chinese and Western medicines based on deep learning and chemical language according to claim 1, characterized in that, The drug flavor dataset described in S2 undergoes preprocessing and data augmentation before pretraining. The preprocessing includes normalizing SMILES using RDkit and mapping text labels to numerical IDs. The data augmentation involves randomly selecting starting atoms and traversal order using RDkit while maintaining chemical structural equivalence, generating multiple semantically equivalent but character-sequenced SMILES structures for each molecule. Depending on the scarcity of category samples, 3-10 random SMILES sequences are generated for each molecule. The generated samples and the original samples participate in model training together, with the category labels remaining unchanged. This significantly increases the amount of data for low-frequency drug flavor categories without introducing chemical noise.

4. The method for predicting the flavors of traditional Chinese and Western medicines based on deep learning and chemical language according to claim 3, characterized in that, The ratio of the training set, validation set, and test set in S2 is 8:1:

1.

5. The method for predicting the flavors of traditional Chinese and Western medicines based on deep learning and chemical language according to claim 1, characterized in that, In the transfer learning described in S4, 100 epochs are set. After each epoch, the model performance is evaluated using a validation set, and accuracy, precision, recall, Matthews coefficient, and ROC AUC are calculated. When the accuracy performance on the validation set exceeds the previous best record, the current model is saved. Early stopping and learning rate reduction strategies are adopted to prevent overfitting and optimize the training process, resulting in a pre-trained model for predicting the flavors of Chinese and Western medicines based on deep learning and chemical language.

6. The method for predicting the flavors of traditional Chinese and Western medicines based on deep learning and chemical language according to claim 5, characterized in that, The early stopping strategy is based on the accuracy of the validation set with a patience value of 5; the learning rate reduction strategy is based on the accuracy of the validation set with a reduction factor of 0.5 and a patience value of 5.

7. A system applying the method for predicting the flavors of traditional Chinese and Western medicines based on deep learning and chemical language as described in any one of claims 1-6, characterized in that, The system includes the following modules: (1) Drug Flavor Dataset Module, which is used to acquire the drug flavor dataset and update, enhance and divide it into sub-datasets; (2) Chinese and Western medicine flavor prediction module, the Chinese and Western medicine flavor prediction module is a Chinese and Western medicine flavor prediction model based on deep learning and chemical language obtained by modifying, transferring learning and fine-tuning the ChemBERTa pre-trained model based on the RoBERTa architecture described in S1 and S2-S4. (3) Drug molecule input module, which is used to input the SMILES structure of the drug molecule to be predicted and enter the Chinese and Western medicine flavor prediction module for prediction; (4) Flavor prediction result output module, which is used to output the predicted probability of the six flavors of the drug molecule (sour, bitter, sweet, pungent, salty, bland) and the flavor category with the highest probability value.

Images