ChemBERTa-FP anticancer drug prediction method based on combination of deep learning and chemical field

Abstract

The invention discloses a ChemBERTa-FP anti-cancer drug prediction method based on combination of deep learning and the chemical field. The ChemBERTa-FP anti-cancer drug prediction method comprises the following steps: step 1, acquiring data of a CI-60DTP database and data of a ChEMBL database; step 2, ChemBERTa and BERTbase are used as a basic pre-training model; a BERT base pre-trains a deep bidirectional language representation model, and an additional output layer is used for adjusting the pre-trained BERT base model, so that a model is created for a wide range of tasks under the condition that major architecture modification is not carried out; meanwhile, ChemBERTa is used as a basic pre-training model, RoBERTa is used as a basic model, pre-training is carried out on a PubChem 77M data set, and a specific chemical language SMILES is added in training; 3, the dimension represented by the model vector of ChemBERTa is 384, the number of heads is 12, and the number of layers of Transformer is 6. According to the invention, two deep learning models, namely ChemBERTa and BERT-base, are innovatively utilized in the field of anti-cancer molecule screening, which is the first application of the models.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical materials for promoting wound healing, and specifically relates to a ChemBERTa-FP anticancer drug prediction method based on the combination of deep learning and the field of chemistry. Background Art

[0002] In the process of drug molecule discovery and design, accurately predicting the biological activity and pharmacological properties of molecules has always been a complex and time-consuming challenge. Traditional methods mainly rely on experimental data and empirical formulas, which are not only costly but also inefficient. In recent years, computer-assisted screening technologies have shown great potential in this field. These technologies, including computer prediction models based on molecular descriptors / fingerprints and chemical language (SMILES), cover biological activity prediction, drug similarity prediction, and toxicity prediction, which can significantly reduce the over-reliance on time-consuming and labor-intensive experiments.

[0003] Among them, machine learning prediction models based on molecular descriptors / fingerprints to characterize chemical structures, including random forests (RF), support vector machines (SVM), extreme gradient boosting (XGBoost), K nearest neighbors (KNN), etc., have been widely used in a variety of prediction tasks such as biological activity, drug similarity and toxicity. Although these methods reduce the dependence on experiments, they are still insufficient in capturing the complex structural features and interactions of drug molecules, and the prediction accuracy needs to be improved.

[0004] Unlike descriptor / fingerprint-based methods, deep learning models based on natural language processing (NLP), such as BERT (Bidirectional Encoder Representations from Transformers), have been successfully applied to text and language processing. The ChemBERTa model draws on the BERT architecture, specifically for chemical language (SMILES), and is pre-trained on SMILES sequences of 77 million unlabeled compounds in the PubChem library to fully mine the substructure or fragment information of drug molecules and deeply capture the structural features and interaction relationships of complex molecules. By performing targeted fine-tuning on the pre-trained ChemBERTa model, competitive results have been shown in various molecular property prediction tasks.

[0005] However, the ChemBERTa algorithm has not yet been effectively applied to the prediction and screening of anticancer drugs, which is a potential area that has not been fully developed. Therefore, developing an efficient molecular prediction method that combines deep learning and chemical expertise is of great significance for accelerating the development of anticancer drugs. The method of the present invention can not only improve the accuracy and reliability of prediction, but also significantly shorten the cycle of drug discovery, providing strong impetus and support for the development of anticancer drugs. Summary of the invention

[0006] The purpose of the present invention is to provide an innovative anticancer drug prediction method, named ChemBERTa-FP, which is an improved ChemBERTa model that combines deep learning technology and molecular fingerprint features to predict the activity of anticancer drugs to improve the efficiency and accuracy of anticancer drug research and development. This method is particularly aimed at common problems in anticancer drug development, such as long research and development cycles and difficult analysis of complex molecular structures, and optimizes the accuracy of prediction through advanced algorithms. The present invention proposes a model framework that combines deep learning and chemical knowledge to predict the activity of anticancer drugs.

[0007] To achieve the above-mentioned purpose and other related purposes, the technical solution provided by the present invention is: a ChemBERTa-FP anticancer drug prediction method based on the combination of deep learning and chemistry, characterized in that it comprises the following steps:

[0008] Step 1: Obtain data from the CI-60DTP database and the ChEMBL database;

[0009] Dataset collection: Data from the NCI-60DTP database were used, based on the CDRUG dataset construction method, focusing on bioactive molecules associated with anticancer properties;

[0010] Data integration: Merged with the NCI-60 dataset constructed by DeepCancerMap to construct a dataset covering 9 types of tumors and 60 tumor cell lines, including 609,593 active records and 3,764,693 inactive records;

[0011] Step 2: ChemBERTa and BERT_base were used as base pre-trained models; BERT base pre-trained a deep bidirectional language representation model, and the pre-trained BERTbase model was adjusted with an additional output layer to create models for a wide range of tasks without major architectural modifications; at the same time, ChemBERTa was used as a base pre-trained model, which is based on RoBERTa as a base model and pre-trained on the PubChem 77M dataset, by adding a specific chemical language SMILES to the training;

[0012] Step 3: The dimension of the model vector representation of ChemBERTa is 384, the number of heads is 12, and the number of Transformer layers is 6; the fine-tuning process of ChemBERTa-FP classification is divided into the following steps:

[0013] 1) Data preparation: Preprocess the collected data and use ChemBERTa’s AutoTokenizer to convert the SMILES representation of chemical molecules into numerical Embeddings

[0014] 2) Load pre-trained model: Load the pre-trained ChemBERTa model, which has learned the contextual representation of the chemical language SMILES;

[0015] 3) Define the classification layer: Add a classification layer on top of the ChemBERTa model and three molecular fingerprint features: the classification layer of MACCS, PubChem and Pharmacophore ErG for chemical formula classification;

[0016] 4) Setting the loss function and optimizer: A composite loss function of Focal Loss and Cross Entropy Loss is used, and the Adam optimization algorithm is used to adjust the model parameters;

[0017] 5 Fine-tune the model: Use the prepared dataset to fine-tune the model. In this process, the parameters of the ChemBERTa-FP model are adjusted according to the objectives of the classification task through the back-propagation algorithm and gradient descent to make the model better suited to the classification task; and the final ChemBERTa-FP vector is fed into the linear classification layer for the binary classification task, which is used for anti-cancer screening and prediction of toxicity to normal cells;

[0018] 6) Evaluate the model: After fine-tuning, use the test set to evaluate the model and calculate the AUC metric to evaluate the model performance.

[0019] 7) Save and use the model: After each epoch, use the validation set to evaluate the model performance; when the performance on the validation set exceeds the previous best record, save the current model;

[0020] Step 4: In order to evaluate the performance of the developed model, the model performance evaluation indicators were used, which are specifically defined in the equation shown below; including: accuracy, recall, specificity, precision, and score;

[0021]

[0022] Among them, TN represents the number of true negatives, TP represents the number of true positives, FN represents the number of false negatives, FP represents the number of false positives; ACC represents accuracy, REC represents recall, SPE represents specificity, PRE represents precision (PRE) and F1 represents score.

[0023] The preferred technical solution is: the data in the ChEMBL database collects molecular data of 19 normal cell lines; a total of 6216 active records and 2312 inactive records; the processed data set of each of the above cell lines is randomly divided into three sub-data sets: training set, validation set and test set in a ratio of 8:1:1.

[0024] The preferred technical solution is: the NCI-60DTP database covers data sets of 9 types of tumors and 60 tumor cell lines, including 609,593 active records and 3,764,693 inactive records.

[0025] The preferred technical solution is: in order to more comprehensively evaluate the performance of the developed model and ensure its advancement and practicality in anti-cancer drug prediction; machine learning is supplemented, including: RF, SVM, KNN, NB and deep learning models; these models are all implemented using the Scikit-learn Python package; all models are trained on GPU RTX 4090 (24GB)*1, CPU12vCPU Intel(R)Xeon(R)Platinum 8352V CPU@2.10GHz, memory 90GB, hard disk 500GB.

[0026] The preferred technical solution is: in order to intuitively display the comprehensive performance of the classification model, ROC curve analysis is used; in the ROC curve analysis, the area value under the ROC curve is particularly emphasized as an important indicator for evaluating the classification ability of the model.

[0027] The preferred technical solution is: 0.5 is used as a threshold for the classification of anticancer drugs; when the probability value predicted by the model exceeds 0.5, the corresponding compound is considered to have potential anticancer effects.

[0028] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art:

[0029] 1. This invention innovatively uses two deep learning models, ChemBERTa and BERT-base, in the field of anticancer molecule screening. This is the first application of such models. We use six key performance indicators, including AUC, ACC, REC, SPE, PRE and F1 score, to comprehensively evaluate the model performance, ensuring the high efficiency and accuracy of the model in predicting anticancer molecules.

[0030] 2. Based on the ChemBERTa model, this invention integrates three molecular fingerprint features for the first time: MACCS, PubChem and Pharmacophore ErG, forming an innovative ChemBERTa-FP model. These molecular fingerprint features enrich the data input of the model. This integration not only improves the model's ability to predict the structure and chemical properties of anti-cancer molecules, but also significantly improves the accuracy of the prediction. In addition, even under standard computer configuration (GPU RTX 4060 (8GB) *1, CPU (AMDRyzen 9 7945HX with Radeon Graphics) 2.50GHz, memory 16GB), the model can complete the activity prediction of a single molecule in 60 different tumor cell lines within 10 seconds. This significant improvement in computing efficiency not only accelerates the screening process of anti-tumor drugs, but also improves its universality and practical value in biochemical research.

[0031] 3. A major innovation of this invention is to apply the ChemBERTa-FP model to the normal cell data set and develop a model specifically for identifying molecules that are toxic to normal cells. This progress is of great value in identifying anti-cancer molecules that are both effective and safe, and helps to screen out non-toxic and safer anti-tumor drugs, thereby accelerating the drug development process and clinical application. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is the framework diagram of the ChemBERTa pre-trained model.

[0033] Figure 2 This is the framework diagram of the ChemBERTa-FP model. DETAILED DESCRIPTION

[0034] The following is a description of the implementation of the present invention by means of specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification.

[0035] See also Figure 1-2. It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings in this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention, so they have no substantial technical significance. Any modification of the structure, change in the proportional relationship, or adjustment of the size should still fall within the scope of the technical content disclosed by the present invention without affecting the effects and purposes that can be achieved by the present invention. At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" quoted in this specification are only for the convenience of description, and are not used to limit the scope of the implementation of the present invention. Changes or adjustments in their relative relationships should also be regarded as the scope of the implementation of the present invention without substantially changing the technical content.

[0036] Unless otherwise specified, the reagents or materials described in the following examples are commercially available.

[0037] Example 1: Study on the impact of training data volume on ChemBERTa-FP model performance

[0038] In order to further clarify the impact of different numbers of ChemBERTa-FP models on model performance, the present invention uses a dataset updated in 2023 and merged with the 2016 NCI-60 database (containing 609,593 active records and 3,764,693 inactive records), and trains the ChemBERTa-FP model separately with the 2016 NCI-60 dataset (containing 591,597 active records and 2,537,387 inactive records), and evaluates the performance of the two models using six key performance indicators, including AUC, ACC, REC, SPE, PRE and F1 score.

[0039] The results are shown in Table 2. The performance of the ChemBERTa-FP model trained with the combined dataset (2023+2016) is higher than that of the 2016 NCI-60 dataset in terms of AUC, ACC, REC, SPE and F1 score. In particular, its AUC value can reach more than 90%, which is even better than the current advanced anti-cancer prediction model. It also shows that the data quality of the combined dataset is high. In addition, the larger the sample ratio, the better the predictive performance of the model. Based on this, the subsequent anti-tumor model establishment is based on the NCI-60 (2023+2016) dataset.

[0040] Table 2. Performance comparison of ChemBERTa-FP on different NCI-60 datasets (%)

[0041]

[0042] Example 2: Study on the influence of three fingerprint features on the performance of ChemBERTa model

[0043] Existing studies have found that the three fingerprint features (PubChem, MACCS and Pharmacophore ErG) are combined with a deep learning model (GNN), and compared with the unfused feature model and other traditional machine learning, the model after the fusion of features has excellent overall prediction performance of drug molecular properties (such as ADMET properties). Therefore, in order to further clarify the impact of the fusion of three fingerprint features on the predictive performance of ChemBERTa and BET-base models in anticancer drugs, the present invention is based on the NCI-60 data set, and ChemBERTa and BET-base are fused respectively through three molecular fingerprint features (MACCS, PubChem and Pharmacophore ErG), and ChemBERTa-FP and BERT-base-FP models are successfully constructed.

[0044] The model performance results are shown in Table 3. Compared with ChemBERTa, ChemBERTa-FP has slightly improved ACC, REC, and F1 scores, especially AUC scores. AUC is an important indicator because it does not depend on a specific classification threshold and can measure the overall effect of the model's ability to classify positive and negative samples. This shows that the fusion of fingerprint features enhances the overall performance of ChemBERT. Similarly, similar to ChemBERTa, BERT-base combined with fingerprint features (BERT-base-FP) also improves in accuracy and AUC scores, although this improvement is very small. These results show that both ChemBERTa and BERT-base have better predictive ability after combining molecular fingerprint features. In addition, compared with BERT-base-FP, although ChemBERTa-FP has slightly reduced ACC, PRE, and SPC, its REC, F1 scores, and especially AUC scores are significantly improved. Overall, ChemBERTa-FP model performance is the best in anticancer drug prediction ability.

[0045] Table 3. Performance comparison of ChemBERT and BERT-base before and after fusion of fingerprint features (%)

[0046]

[0047] Example 3: Comparison of the prediction performance of ChemBERTa-FP with typical machine learning and deep learning (DNN)

[0048] In order to comprehensively evaluate the performance of ChemBERTa-FP in anticancer drug prediction, the present invention trained typical machine learning models (RF, SVM, KNN, NB) and deep learning models (DNN) as baseline models based on the NCI-60 dataset, and compared the performance with the ChemBERTa-FP model.

[0049] The results are shown in Table 4. The ChemBERTa-FP model performs best in F1 score and AUC value, reaching 65.987% and 92.429% respectively, indicating that the overall performance of the model is optimal. Although the performance in ACC, REC, PRE and SPC is not optimal, these values ​​are much higher than most machine learning and deep learning models, further indicating that the ChemBERTa-FP model has a good comprehensive prediction ability for anticancer drugs.

[0050] Table 4. Performance comparison of ChemBERTa-FP model and other baseline models on NCI-60 cancer cell line dataset (%)

[0051]

[0052] Example 4: Model performance of ChemBERTa-FP constructed on normal cell line dataset

[0053] It is well known that anticancer drugs lack selective toxicity to normal human cells, resulting in serious side effects and limiting their clinical application. Therefore, predicting the cytotoxicity of compounds to normal cell lines is crucial in the early stages of anticancer drug discovery. Accordingly, we established prediction models for 19 normal cell lines, involving 6216 active records and 2312 inactive records. These models based on the ChemBERTa-FP method performed well for 19 normal cell lines (Table 5), with an average AUC value of 91.443% for the test set. Such a model can be used to predict whether anticancer drugs are toxic to normal cells. In general, the ChemBERTa-FP cell-based prediction model can be used not only to predict active molecules against various cancer cell lines, but also to detect the potential cytotoxicity of compounds to normal cell lines, which can be used for cell-based anticancer drug discovery.

[0054] Table 5. Performance evaluation of the ChemBERTa-FP model on 19 normal cell lines (%)

[0055]

[0056]

[0057] Example 5: Temsirolimus Case Study

[0058] Temsirolimus is an mTOR and serine / threonine kinase inhibitor, FDA approved for renal cell carcinoma (an aggressive kidney cancer). It is the first clinically effective mTOR inhibitor discovered, used to treat mantle cell lymphoma, and is also approved in Europe. Current studies have found that temsirolimus has an inhibitory effect on the proliferation of various tumor cells, such as lung cancer, prostate cancer, colon cancer, and breast cancer. The present invention uses the ChemBERTa-FP model constructed based on the NCI-60 data set to predict the anti-cancer activity of temsirolimus, and temsirolimus does not appear in the NCI-60 data set.

[0059] The prediction results are shown in Table 6. It is predicted that temsirolimus is effective against 9 tumor cell lines, including 100% prostate cancer cell lines, 100% leukemia cell lines, 100% central nervous system tumor cell lines, and 100% non-small cell lung cancer cell lines. The prediction probability of anticancer activity is high, reaching more than 80%. It is suggested that the prediction results of the ChemBERTa-FP model are consistent with the experimental results of researchers at this stage, indicating that the model can effectively predict the anticancer activity of drugs. In addition, the present invention further uses the ChemBERTa-FP model constructed based on 19 normal cell lines to predict the effect of temsirolimus on normal cells. The results show that temsirolimus has toxic effects on 68.42% of normal cell lines. The reason for the analysis may be that the mTOR pathway plays a key role in cell growth, proliferation and survival. As an mTOR inhibitor, the main mechanism of action of temsirolimus is to block the mTOR pathway, thereby inhibiting the proliferation and survival of cancer cells. However, this pathway is not only active in cancer cells, but also plays a role in normal cells. Therefore, in theory, temsirolimus may affect the growth and function of normal cells. At present, clinical trials have confirmed that temsirolimus has significant anti-cancer effects, but also has obvious toxic side effects. In summary, the ChemBERTa-FP model has important value in identifying effective and safe anti-cancer drugs.

[0060] Table 6. Prediction results of ChemBERTa-FP for temsirolimus based on the NCI-60 dataset (%)

[0061] Tumor type Activity prediction probability Effective proportion of cell lines Prostate cancer 95.814 100.000 leukemia 90.638 100.000 Central neurocytoma 82.728 100.000 Non-small cell lung cancer 80.544 100.000 Melanoma 78.284 88.890 Renal cell carcinoma 77.426 100.000 Breast cancer 76.401 100.000 Ovarian cancer 71.511 85.710 Colon cancer 62.772 71.430

[0062] Table 7. Prediction results of ChemBERTa-FP for temsirolimus based on 19 normal cell line datasets (%)

[0063]

[0064]

[0065] Embodiment 2:

[0066] An innovative anticancer drug prediction method, named ChemBERTa-FP, is an improved ChemBERTa model that combines deep learning technology and molecular fingerprint features to predict the activity of anticancer drugs to improve the efficiency and accuracy of anticancer drug research and development. This method is particularly targeted at common problems in anticancer drug development, such as long research and development cycles and difficult to analyze complex molecular structures, and optimizes the accuracy of predictions through advanced algorithms. The present invention proposes a model framework that combines deep learning and chemical knowledge to predict the activity of anticancer drugs. This is achieved by the following technical solutions:

[0067] 1. Dataset collection and processing

[0068] Dataset source: The datasets used in the present invention mainly come from the NCI-60DTP (https: / / wiki.nci.nih.gov / display / NCIDTPdata / NCI-60+Growth+Inhibition+Data) and ChEMBL (https: / / www.ebi.ac.uk / chembl / ) databases.

[0069] Construction of anti-cancer drug molecular data set:

[0070] Dataset collection: The data of the recent version of the NCI-60DTP database (as of July 2023) is used, based on the dataset construction method of LI et al.: CDRUG: a web server for predicting anticancer activity of chemical compounds, focusing on bioactive molecules associated with anticancer properties. Data integration: Merged with the NCI-60 dataset constructed by Wu et al. DeepCancerMap: Aversatiledeep learning platform for target-and cell-basedanticancer drug discovery; to construct a dataset covering 9 types of tumors and 60 tumor cell lines, including 609,593 active records and 3,764,693 inactive records.

[0071] Construction of normal cell-related molecular dataset:

[0072] Data source: Based on the molecular data of 19 normal cell lines collected in the ChEMBL database by Wu et al.

[0073] Data volume: A total of 6216 active records and 2312 inactive records.

[0074] The processed data sets of each cell line were randomly divided into three sub-data sets: training set, validation set and test set in a ratio of 8:1:1.

[0075] 2. Selection of BERT series pre-training models

[0076] This example uses two BERT models as the basic pre-trained models for this experiment, including ChemBERTa and BERT_base. BERT pre-trains a deep bidirectional language representation model, and the pre-trained BERT model can be fine-tuned with an additional output layer to create state-of-the-art models for a wide range of tasks without major architectural modifications. There are different versions of BERT pre-trained models, and we used BERT_base pre-trained by English Wikipedia and Book Corpus. At the same time, we also use ChemBERTa (https: / / huggingface.co / DeepChem / ChemBERTa-77M-MLM / tree / main) as the basic pre-trained model for this study ( Figure 1 ), which is based on RoBERTa as the base model and pre-trained on the PubChem 77M dataset (SMILES containing 77 million chemical substances). By adding the specific chemical language SMILES to the training, ChemBERTa can better understand and process chemical information. It shares a similar architecture with BERT, but shows stronger robustness in classification tasks, and provides a more advanced pre-trained model specifically for the prediction of small molecule drug properties. Therefore, by training the ChemBERTa model with labeled cancer-related datasets, improving and fine-tuning the architecture or adding more training data, and combining it with other advanced chemical informatics techniques, such as molecular fingerprint features, the model's predictive effect on the properties of anticancer drugs may be further improved.

[0077] 3. Construction of ChemBERTa-FP

[0078] The dimension of ChemBERTa's model vector representation is 384, the number of heads is 12, and the number of Transformer layers is 6 ( Figure 1 ). Fine-tuning process of ChemBERTa-FP classification ( Figure 2) can be divided into the following steps: 1) Data preparation: The data collected in this study are preprocessed, such as word segmentation, stop word removal, etc., and the AutoTokenizer of ChemBERTa (DeepChem / ChemBERTa-77M-MLM) developed by Chithrananda et al. is used to convert the SMILES representation of chemical molecules into numerical Embeddings, which is a high-dimensional numerical encoding that the model can handle. This process not only captures the structural characteristics of the molecules, but also retains the richness and complexity of chemical information. It is then input into the ChemBERTa model for deep learning training and prediction. 2) Loading pre-trained model: Load the pre-trained ChemBERTa model, which has learned the contextual representation of the chemical language SMILES. 3) Define the classification layer: Add a classification layer and a classification layer of three molecular fingerprint features (MACCS, PubChem, and Pharmacophore ErG) on top of the ChemBERTa model for chemical formula classification. These fingerprints, as additional features, contain various structural and chemical properties of molecules, enhancing the predictive ability of the model. 4) Setting loss function and optimizer: In order to effectively deal with the challenge of data imbalance and improve the model's recognition ability for minority categories, we adopted a composite loss function of Focal Loss combined with Cross Entropy Loss. In addition, we selected the Adam optimization algorithm to adjust the model parameters. 5) Fine-tuning model: Fine-tune the model using the prepared dataset. In this process, the parameters of the ChemBERTa-FP model are adjusted according to the objectives of the classification task, which is achieved through the back propagation algorithm and gradient descent to make the model better adapt to the classification task. And the final ChemBERTa-FP vector is fed into the linear classification layer for the binary classification task, which is used for anticancer screening and prediction of toxicity to normal cells. The fine-tuning process adapts the broad chemical understanding obtained during pre-training to the specific context of anticancer activity, generating a specialized model specifically for predicting the potential efficacy of molecular compounds as anticancer agents. At the same time, the parameter optimization conditions of the model are set as shown in Table 1. 6) Evaluate the model: After fine-tuning is completed, the model is evaluated using the test set, and the performance of the model is evaluated by calculating the AUC indicator. 7) Save and use the model: After each epoch, the model performance is evaluated using the validation set. When the performance on the validation set exceeds the previous best record, we save the current model. During the testing phase, we conduct a final evaluation of the model, focusing on its accuracy and reliability in predicting the activity of anticancer drugs for subsequent use.

[0079] Table 1. Parameters of different models

[0080]

[0081] 4. Baseline Model

[0082] In order to be able to more comprehensively evaluate the performance of our developed model and ensure its advancement and practicality in anticancer drug prediction. The present invention adds a series of classic machine learning (RF, SVM, KNN, NB) and deep learning models (DNN) as baseline models for comprehensive comparison. These models are all implemented using the Scikit-learn Python package (https: / / github.com / scikit-learn / scikit-learn, version: 0.24.1). All models are trained on GPU RTX4090 (24GB) * 1, CPU 12vCPU Intel (R) Xeon (R) Platinum 8352V CPU @ 2.10GHz, memory 90GB, hard disk 500GB.

[0083] 5. Model performance evaluation

[0084] In order to comprehensively evaluate the performance of the model we developed, the present invention adopts a series of model performance evaluation indicators, which are specifically defined as equations (1-5) shown below. These include: Accuracy (ACC), Recall (REC), Specificity (SPE), Precision (PRE) and F1 Score (F1). In addition, in order to intuitively demonstrate the comprehensive performance of the classification model, we also used the Receiver Operating Characteristic (ROC) curve analysis. In the ROC curve analysis, we particularly emphasized the area under the ROC curve (AUC) value as an important indicator for evaluating the classification ability of the model. In addition, the classification of anticancer drugs in the present invention uses 0.5 as the threshold. This means that when the probability value predicted by the model exceeds 0.5, the corresponding compound is considered to have potential anticancer effects. This setting is based on an in-depth analysis of the data characteristics, aiming to balance the sensitivity and specificity of the model, thereby ensuring that the selected candidate compounds have high reliability.

[0085]

[0086] Among them, TN represents the number of true negatives, TP represents the number of true positives, FN represents the number of false negatives, and FP represents the number of false positives.

[0087] The above description is only used to explain the preferred embodiments of the present invention, and is not intended to limit the present invention in any form. Therefore, any modifications or changes made to the present invention under the same inventive spirit should still be included in the scope of protection intended by the present invention.

Claims

1. A ChemBERTa-FP anticancer drug prediction method based on the combination of deep learning and chemistry, characterized by: The following steps are involved: Step 1: Obtain data from the CI-60DTP database and the ChEMBL database; Dataset collection: Data from the NCI-60DTP database were used, based on the CDRUG dataset construction method, focusing on bioactive molecules associated with anticancer properties; Data integration: Merged with the NCI-60 dataset constructed by DeepCancerMap to construct a dataset covering 9 types of tumors and 60 tumor cell lines, including 609,593 active records and 3,764,693 inactive records; Step 2: ChemBERTa and BERT_base were used as base pre-trained models; BERT base pre-trained a deep bidirectional language representation model, and the pre-trained BERTbase model was adjusted with an additional output layer to create models for a wide range of tasks without major architectural modifications; at the same time, ChemBERTa was used as a base pre-trained model, which is based on RoBERTa as a base model and pre-trained on the PubChem 77M dataset, by adding a specific chemical language SMILES to the training; Step 3: The dimension of the model vector representation of ChemBERTa is 384, the number of heads is 12, and the number of Transformer layers is 6; the fine-tuning process of ChemBERTa-FP classification is divided into the following steps: 1) Data preparation: Preprocess the collected data and use ChemBERTa’s AutoTokenizer to convert the SMILES representation of chemical molecules into numerical Embeddings 2) Load pre-trained model: Load the pre-trained ChemBERTa model, which has learned the contextual representation of the chemical language SMILES; 3) Define the classification layer: Add a classification layer on top of the ChemBERTa model and three molecular fingerprint features: the classification layer of MACCS, PubChem and Pharmacophore ErG for chemical formula classification; 4) Setting the loss function and optimizer: A composite loss function of Focal Loss and Cross Entropy Loss is used, and the Adam optimization algorithm is used to adjust the model parameters; 5 Fine-tune the model: Use the prepared dataset to fine-tune the model. In this process, the parameters of the ChemBERTa-FP model are adjusted according to the objectives of the classification task through the back-propagation algorithm and gradient descent to make the model better suited to the classification task; and the final ChemBERTa-FP vector is fed into the linear classification layer for the binary classification task, which is used for anti-cancer screening and prediction of toxicity to normal cells; 6) Evaluate the model: After fine-tuning, use the test set to evaluate the model and calculate the AUC metric to evaluate the model performance. 7) Save and use the model: After each epoch, use the validation set to evaluate the model performance; when the performance on the validation set exceeds the previous best record, save the current model; Step 4: In order to evaluate the performance of the developed model, the model performance evaluation indicators were used, which are specifically defined in the equation shown below; including: accuracy, recall, specificity, precision, and score; Among them, TN represents the number of true negatives, TP represents the number of true positives, FN represents the number of false negatives, FP represents the number of false positives; ACC represents accuracy, REC represents recall, SPE represents specificity, PRE represents precision, and F1 represents score.

2. The ChemBERTa-FP anticancer drug prediction method based on the combination of deep learning and chemistry according to claim 1, characterized in that: The data in the ChEMBL database collected molecular data from 19 normal cell lines; a total of 6216 active records and 2312 inactive records; the processed data sets of each of the above cell lines were randomly divided into three sub-data sets: training set, validation set and test set in a ratio of 8:1:

1.

3. The ChemBERTa-FP anticancer drug prediction method based on the combination of deep learning and chemistry according to claim 1, characterized in that: The NCI-60DTP database covers datasets of 9 types of tumors and 60 tumor cell lines, including 609,593 active records and 3,764,693 inactive records.

4. The ChemBERTa-FP anticancer drug prediction method based on the combination of deep learning and chemistry according to claim 1, characterized in that: In order to more comprehensively evaluate the performance of the developed model and ensure its advancement and practicality in anti-cancer drug prediction, machine learning was added, including RF, SVM, KNN, NB and deep learning models. These models were implemented using the Scikit-learn Python package. All models were trained on a GPU RTX 4090 (24GB)*1, CPU 12vCPUIntel(R)Xeon(R)Platinum 8352V CPU@2.10GHz, memory 90GB, and hard disk 500GB.

5. The ChemBERTa-FP anticancer drug prediction method based on the combination of deep learning and chemistry according to claim 1, characterized in that: In order to intuitively demonstrate the comprehensive performance of the classification model, ROC curve analysis was used; in the ROC curve analysis, special emphasis was placed on the area value under the ROC curve as an important indicator for evaluating the classification ability of the model.

6. The ChemBERTa-FP anticancer drug prediction method based on the combination of deep learning and chemistry according to claim 1, characterized in that: A threshold of 0.5 was used for the classification of anticancer drugs; when the probability value predicted by the model exceeded 0.5, the corresponding compound was considered to have potential anticancer effects.

Images