Protein and small molecule binding constant prediction method and storage medium

Abstract

The invention discloses a method for predicting binding constants of proteins and small molecules and a storage medium. The prediction method comprises the following steps: acquiring an amino acid sequence of a protein, an SMILES character string of a small molecule and a binding constant between the protein and the small molecule, inputting a pre-training data set formed by the amino acid sequence of the protein, the SMILES character string of the small molecule and the binding constant between the protein and the small molecule into different large language models, and screening to obtain a base large language model; a LoRA fine tuning base large language model is adopted; optimizing parameters in the generation task; obtaining a prediction model based on the fine-tuned base large language model and the optimized parameters; and inputting the amino acid sequence of the to-be-predicted protein and the SMILES character string of the small molecule into the prediction model, and outputting a binding constant. According to the method, biomolecules are converted into a chemical language mode, and inference can be carried out only by inputting characters of paired amino acid sequences and SMILES of small molecules at the same time. According to the invention, a decision tool can be provided for drug screening.

Description

Technical Field

[0001] The present invention relates to a method for predicting intermolecular interactions, and in particular to a method and storage medium for predicting protein-small molecule binding constants based on fine-tuning of a large language model. Background Art

[0002] The large language model is a huge neural network model trained based on deep learning technology, which has powerful language understanding and generation capabilities. The large language model has made breakthrough progress in various natural language processing tasks, such as question and answer, text generation, translation, summarization, etc., and has surpassed the human level in some tasks. Large-scale language models have made progress in various industries, and various general large language models have appeared. How to use large language models to solve problems in professional fields has become a new research hotspot, which involves pre-training and supervised fine-tuning of large language models. It is reported that GPT-3 needs to use 1,000 NVIDIAA100 graphics cards to train for one month to complete a training. Training a large language model from scratch has become less realistic. At present, a more economical approach is to fine-tune the large model so that it can be competent for tasks in a specific field.

[0003] ChemDFM, the first large language model for the field of chemistry, introduces a large amount of basic chemical knowledge into the large language model, conducts field pre-training and fine-tuning of the large model, and fully learns and masters the proprietary language and expression of chemical science. General large language models are usually based on general data that can be collected, and the chemical expertise included is limited. This leads to the general large language model's knowledge in relatively professional fields is also extremely limited, and it cannot accurately answer questions in more professional fields. The research team collected and sorted a large amount of public paper data, and finally obtained nearly 4 million papers on chemistry and related disciplines through screening. And a large amount of chemical textbooks and reference books and other data to provide basic knowledge. After pre-training and fine-tuning of instructions, the problems of lack of professional knowledge and difficulty in understanding professional language in the general large model during the specialization of large models were solved respectively. ChemDFM showed better generation results in questions and answers in chemical professional fields such as molecular property prediction, descriptor-based molecule generation, and reaction prediction and evaluation. Although ChemDFM can predict the structure and properties of chemical molecules based on chemical knowledge. However, the model does not specifically train the binding constants of proteins and small molecules, which are of greater concern in the field of drug design. Therefore, there is no corresponding biomedical model for drug screening. It is not possible to directly predict the binding constants from the description language of proteins and small molecules.

[0004] In the field of biochemistry, for proteins, proteins can be represented as amino acid sequences represented by 20 characters. This sequence information is unique to each protein. The structure and properties of the protein are determined by the sequence information. Recently, AlphaFold has achieved the direct prediction of the structure of the protein from the amino acid sequence information of the protein. The accuracy of the predicted structure has reached the atomic level, which is highly consistent with the experimentally resolved structure. For small molecules, small molecules can be represented as SMILES strings. SMILES strings played an important role in the early field of bioinformatics. For example, they can be used to quickly retrieve molecular databases through search strings. It is a commonly used chemical language. In addition, deep learning can be used to predict the properties of drug molecules from SMILES strings. For example, by establishing a quantitative structure-activity relationship between SMILES and structural properties, the prediction of molecular properties, toxicity and biological activity can be achieved (J.Chem.Inf.Model.2019,59,2,914). The multi-task BERT model can learn the pattern of features from the SMILES string, thereby realizing the prediction and interpretation of molecular properties (J.Chem.Inf.Model.2019,59,2,914). The traditional method of calculating the binding constant between proteins and small molecules depends on the three-dimensional structure of proteins and small molecules. However, not all proteins can be experimentally resolved. At present, it is still difficult to determine the structure of proteins by experimental methods. As of September 2024, the total number of structures in the Protein Data Bank (PDB) has exceeded 220,000, but compared with more than 200 million sequences in GenBank, the number of structures is still too small. Artificial intelligence is a key technology to solve the problem of directly predicting binding constants from sequences. Based on these studies, it can be found that the amino acid sequence of proteins and the SMILES of molecules already contain the required information. Therefore, the molecular structure of the complex composed of proteins and small molecules can be represented as a sequence in the form of a string and a special chemical language, SMILES, respectively, and this type of chemical language can convey the corresponding biochemical information. However, there is currently no large language model that can directly predict the binding constants between proteins and small molecules from chemical language.

[0005] The present invention uses a large language model to fine-tune the large language model on a locally constructed data set, migrates the large language model to the vertical field of drug design, and realizes a prediction of protein-ligand binding constants that is independent of the biological three-dimensional structure. Summary of the invention

[0006] Purpose of the invention: The purpose of the present invention is to provide a method for predicting the binding constant between a protein and a small molecule, solving the problem of how to directly predict the binding constant between a protein and a small molecule from the amino acid sequence of the protein and the SMILES of the small molecule. Another purpose of the present invention is to provide a computer-readable storage medium to solve the problem of how to execute the method for predicting the binding constant between a protein and a small molecule.

[0007] Technical solution: A method for predicting the binding constant between a protein and a small molecule according to the present invention comprises the following steps:

[0008] Obtain the amino acid sequence of the protein, the SMILES string of the small molecule, and the binding constant between the protein and the small molecule to form a dictionary of input data;

[0009] The pre-trained data set consisting of the amino acid sequence of the protein, the SMILES string of the small molecule and the binding constant between the protein and the small molecule is input into different large language models, and the base large language model is obtained by screening;

[0010] Low-order adapter LoRA fine-tuning base large language model using large language model;

[0011] Optimize parameters in the generated tasks;

[0012] Obtain a prediction model based on the fine-tuned base language model and optimized parameters;

[0013] The amino acid sequence of the protein to be predicted and the SMILES string of the small molecule are input into the prediction model, and the binding constant between the protein and the small molecule is output.

[0014] The present invention proposes to use a large language model to learn the chemical language of protein sequences and molecular SMILES strings, and to construct a data dictionary in the format of {(protein sequence, small molecule SMILES): binding constant}. The constructed dictionary data is used to fine-tune the parameters of the large language model, and the information conveyed by this special chemical language is mined, thereby realizing the question-and-answer binding constant prediction using the large language model. As is well known, the large language model is an artificial intelligence model that learns and infers language data. The present invention applies the large language model to the vertical field of drug screening for the first time, and fine-tunes the large language model in terms of parameters and input data through the language of chemical informatics, thereby achieving the purpose of predicting binding constants from chemical language.

[0015] Preferably, the step of obtaining the amino acid sequence of the protein, the SMILES string of the small molecule, and the binding constant between the protein and the small molecule comprises:

[0016] The method for obtaining the amino acid sequence of a protein is as follows: obtaining the protein ID number from the PDBbind data set, and downloading the fasta amino acid sequence file from the PDB database according to the ID number;

[0017] The method to obtain the SMILES string of a small molecule is: use Chem.MolFromMol2File(ligand) to convert the mol2 file of the small molecule into a SMILES string;

[0018] The binding constants between proteins and small molecules are collected from the PDBbind dataset.

[0019] Preferably, inputting the pre-trained data sets consisting of the amino acid sequence of the protein, the SMILES string of the small molecule and the binding constant between the protein and the small molecule into different large language models comprises:

[0020] Construct a dictionary in the format of {(protein amino acid sequence, small molecule SMILES string): combined constant}, convert the input format of the dictionary and input it into different large language models.

[0021] Preferably, the conversion function of the input format is:

[0022]

[0023] Preferably, the screening to obtain a base large language model includes:

[0024] The CASF dataset is used to test different large language models. The optimal large prediction model is selected as the base large language model based on evaluation indicators such as the number of valid molecules output, the proportion of digital results in the output results, and the distribution of output results in space.

[0025] Preferably, the large language models include LLaMA-3, Mistral and ChemDFM; the base large language model obtained after screening by the above-mentioned evaluation indicators is LLaMA-3.

[0026] Preferably, the low-order adapter LoRA fine-tuning base large language model using a large language model includes:

[0027] The pre-trained parameters are fixed and a bypass structure is added to the network;

[0028] Assume that the pre-trained weight is W 0 , the trainable parameter is ΔW; for the input x with this weight, the output is as follows:

[0029] h=W 0 x+ΔWx

[0030] Use the prompt word fine-tuning method to fine-tune the base large language model, and set a specified module when fine-tuning the model, which is a key module in the transformer layer;

[0031] The model fine-tuning method using HugginFace is based on the supervised fine-tuning trainer SFTTrainer.

[0032] The present invention uses LoRA, a low-order adapter of a large language model, to fine-tune the model. The LoRA method can reduce the amount of training parameters. We do not need to adjust all the parameters of the large language model, but only need to train a small number of parameters to adapt the model to downstream tasks.

[0033] Preferably, the parameters in the generation task include:

[0034] top_k: When sampling, only the top k words with the highest probability are considered;

[0035] top_p: When sampling, only the words whose probability accumulation reaches this value are considered;

[0036] repetition_penalty: Penalize repeated words to prevent duplicate content;

[0037] temperature: controls the "randomness" of the generated text; the higher the temperature, the more random the generated text.

[0038] After fine-tuning, the generation parameters are optimized. GenerationConfig is a class used to configure text generation tasks. Adjustment parameters can be used to control text generation.

[0039] Based on the above method, another aspect of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program implements the steps of any one of the methods of claims 1-8 when executed by a processor.

[0040] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0041] The present invention can use the fine-tuned large language model to directly predict the binding constants of proteins and ligand molecules from biochemical language. Its characteristics are that the biological molecules are converted into chemical language patterns, and a pair of biological molecule information (protein sequence, small molecule SMILES) needs to be input before inference can be made. The present invention is an attempt to expand the application of the large language model to the field of biomedicine, and can provide a decision-making tool for drug screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Flowchart for protein-small molecule binding constant prediction based on fine-tuning of large language models;

[0043] Figure 2 Schematic diagram of the large language model making predictions based on protein sequences and molecular SMILES;

[0044] Figure 3 Scatter plot of the predicted values ​​of the model without fine-tuning and the true values;

[0045] Figure 4 Schematic diagram of fine-tuning a large language model using LoRA;

[0046] Figure 5 Scoring of parameter optimization during model generation;

[0047] Figure 6 Scatter plot of the fine-tuned model prediction value and the true value, as well as a comparison chart of the Pearson coefficient. DETAILED DESCRIPTION

[0048] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0049] In order to be able to directly predict the binding constants of proteins and small molecules from the amino acid sequence of proteins and the SMILES of small molecules, the present invention proposes a technical solution for constructing and organizing biological data, using the biological data to fine-tune the base language model, and combining generation parameter optimization.

[0050] The flowchart of the technical method for predicting protein-small molecule binding constants based on fine-tuning of a large language model of the present invention is shown in Figure 1 . The present invention provides a method for predicting protein-small molecule binding constants based on fine-tuning of a large language model. It includes obtaining the ID number of the protein from the PDBbind database; obtaining the amino acid sequence of the protein from the PDB database according to the ID number; using RDKit to convert the sdf file of the small molecule into a SMILES string; making the protein sequence, small molecule SMILES and the binding constants of the protein and the small molecule into an input language model; making preliminary predictions on Llama-3 8T, Mistral-7B and ChemDFM respectively; comparing and selecting a relatively good large model as a base model for further pre-training and fine-tuning; using the data organized by the present invention to make an input dictionary of {(protein sequence, small molecule SMILES): binding constant}, using the dictionary to fine-tune the large model of Llama-3 8T, and using the LoRA method to fine-tune the model; reading the fine-tuned model to optimize the generation parameters; and using the optimized parameters to evaluate the model on the CASF data set. The specific steps are as follows:

[0051] S1. First, proteins and small molecules need to be expressed in chemical language.

[0052] The method of converting proteins into sequences is to obtain the protein ID number from the PDBbind dataset, which collects the three-dimensional structures of proteins and small molecules and the binding constants between the two. Since it is difficult to obtain structural information for regions with greater flexibility in the crystal structure, it is not possible to directly convert the PDB file into an amino acid sequence. It is necessary to download the fasta sequence file from the PDB database based on the PDB ID.

[0053] The method for converting a small molecule into a SMILES string is as follows: use RDKit to convert the sdf or mol2 format file of the small molecule into a SMILES string, and use Chem.MolFromMol2File(ligand) to convert the mol2 file of the small molecule into a SMILES string.

[0054] S2, the amino acid sequence of the protein, the SMILES of the small molecule and the binding constants in PDBbind are used to form a pre-training data set. The present invention requires training of the base model, so the data format is converted into a dictionary format to facilitate input into the fine-tuning process of the large model. Take one set of data as an example,

[0055]

[0056]

[0057] S3, converting the data set organized in S2 into the input format required by the large language model. The information required to be input in the present invention is the amino acid sequence of the protein, the SMILES of the small molecule and the binding constant pki. To this end, a dictionary in the format of {(Sequence, SMILES): pki} is constructed. When the dictionary is input into the large language model, the input format needs to be converted. The conversion function is:

[0058]

[0059] This converts the chemical language into a natural language format that can be input into an existing large language model. The input and output results are as follows: Figure 2 As shown. It can be seen that, unlike the previous question-and-answer model, the present invention requires the input of the amino acid sequence of the protein and the SMILES string of the small molecule at the same time to obtain the output result. If only one question is input, the returned result is empty, which is different from the general language model. Because the binding constant is determined by both the protein and the small molecule. The expected result of the present invention is that the amino acid sequence string of the protein and the SMILES of the small molecule can be input to predict the required binding constant.

[0060] S4. In order to obtain a language model with a good base, the present invention first screens the reported general language models. The present invention tests three commonly used large language models, LLaMA-3 8T, Mistral 7B and ChemDFM, and tests them on the collected CASF data set.

[0061] The LLaMA model is a large language model released by META, one version of which, the LLaMA-13B model, surpasses the GPT-3 model in natural language benchmarks. The 8B version of Llama 3 is suitable for deployment and development on consumer-grade GPUs.

[0062] Mistral 7B is a large model developed by French startup Mistral AI. In previous evaluations, Mistral7B has performed well in multiple benchmarks, especially in areas such as mathematical reasoning, coding, and common sense tasks.

[0063] The ChemDFM model is based on the classic open source large model LLaMa. A vast amount of chemical knowledge has been introduced into the model, which fully learns and masters the proprietary language and expressions of chemical science. It is now able to process the SMILES information of molecules and complete the prediction of structural properties.

[0064] Therefore, the present invention first performs a benchmark test on these three large language models, and then selects a better large language model as the base model according to the test results. The model was tested using the data in CASF. Since the large model has not been pre-trained and fine-tuned, all large language models cannot process biochemical information well. Since the large language model itself is used to process language information, some of the output results of the model are not numbers, but in the form of strings. The calculation results are shown in Table 1. The number of valid molecules that LLaMA-3 can output is 133, which is the largest among the three test models, and 46.7% of the output results are numbers. Although ChemDFM is a large language model in the field of chemistry, its test results cannot output a number from the prompt words of the sequence and SMILES every time.

[0065] Table 1 Prediction results of protein-ligand binding constants on the large language model without fine-tuning

[0066] Model Number / percentage of valid outputs Pearson coefficient RMSE LLaMA-3 133 / 46.7％ 0.02 6.1775 Mistral 76 / 26.7％ 0.00 6.5644 ChemDFM 37 / 13.0％ 0.04 6.6266

[0067] In addition, a scatter plot is also drawn between the predicted value and the true value. Figure 3 As shown, it can be seen that the results generated by LLaMA-3 are more distributed in space, while the binding constants generated by Mistral are relatively concentrated. Therefore, the present invention selects LLaMA-3 as the base model.

[0068] S5. Perform fine-tuning based on the base model.

[0069] The present invention selects LoRA, a low-order adapter of a large language model, to fine-tune the model. The LoRA method can reduce the amount of training parameters. We do not need to adjust all the parameters of the large language model. We only need to train a small number of parameters to adapt the model to downstream tasks. The schematic diagram of the method of fine-tuning the model by LoRA is shown in Figure 4 As shown in the figure, LoRA keeps the pre-trained parameters unchanged and adds a bypass structure in the network. Assume that the pre-trained weight is W 0 , the trainable parameter is ΔW. For the input x with this weight, the output is as follows:

[0070] h=W 0 x+ΔWx

[0071] The LLaMA-3 8B model was fine-tuned using the cue word fine-tuning method, and the specified modules were set during model fine-tuning.

[0072] target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj",]

[0073] Specifies the model modules to be fine-tuned. These modules are key modules in the transformer layer, such as the query, key, value mapping of the attention mechanism, and the upper and lower projection layers of the feedforward network.

[0074] The model fine-tuning method of HugginFace is based on the supervised fine-tuning trainer SFTTrainer, and the specific parameters are set

[0075]

[0076] S6. After fine-tuning, optimize the generation parameters. GenerationConfig is a class used to configure text generation tasks. Adjusting parameters can be used to control text generation. Optimizing parameter combinations can further improve the accuracy of generation results. The present invention mainly optimizes the following parameters:

[0077] top_k: When sampling, only the top k words with the highest probability are considered;

[0078] top_p: When sampling, only the words whose probability accumulation reaches this value are considered;

[0079] repetition_penalty: Penalize repeated words to prevent duplicate content;

[0080] temperature: controls the "randomness" of the generated text. The higher the temperature, the more random the generated text.

[0081] The scoring results during the optimization process are as follows Figure 5 As shown, we can see that the parameter combination of top_k=20, top_p=0.6, temperature=0.5, and repetition_penalty=1.05 can obtain better generation results.

[0082] S7, the above S5 step completes the supervised fine-tuning of the model, and the generation parameters are obtained from S6. Through the above steps, a large language model that can be inferred is obtained.

[0083] The present invention uses the obtained fine-tuning model and parameters to generate and predict the binding constants of proteins and ligand molecules. After selecting the corresponding protein and small molecule data in the CASF data and converting them into the amino acid sequence of the protein and the SMILES of the small molecule, the binding constant between the two can be output. The calculation results are as follows Figure 6 As shown in Figure 2, we can see that the generated data has a certain correlation with the true value. Figure 6 It can be clearly seen that the Pearson coefficient of the model output before and after fine-tuning has increased from 0.04 to 0.43. After fine-tuning, the Pearson coefficient of the generated predicted value and the true value has been significantly improved. In addition, 100% of the output results of all input sequences and SMILES are in digital form.

[0084] The present invention proposes a method for fine-tuning the binding constants of proteins and small molecules based on a large language model. The calculation results of the invention are significantly better than those of ChemDFM, a large language model in the field of chemistry. The present invention tests for the first time that the prediction of binding constants can be achieved from protein sequences and small molecule SMILES strings. It is the first attempt in the field of drug screening to use a large language model to predict the binding constants of proteins and small molecules. The process of the present invention will also provide a reference for technical solutions for the fine-tuning and application of large language models in other fields. Of course, the present invention also has certain limitations. Since the present invention uses data in PDBbind, the amount of data is only 14063 data, which is far from the data scale of natural language. Therefore, the generation result of the present invention does not achieve better results. By collecting more data, it is expected to improve the prediction accuracy of the method.

[0085] By inputting the biochemical language of SMILES of protein sequences and small molecules, the fine-tuned large language model can return the binding constant. The prediction of the binding constant measures the binding strength between the drug molecule and the protein, and is an indicator often used in drug screening. The present invention is expected to be used to quickly sort the lead compounds in the early stage of the drug screening process to determine whether to proceed to the next step of development. In summary, the present invention provides a fast and efficient decision-making tool for drug screening.

Claims

1. A method for predicting the binding constant between a protein and a small molecule, characterized in that: The steps include: Obtain the amino acid sequence of the protein, the SMILES string of the small molecule, and the binding constant between the protein and the small molecule to form a dictionary of input data; The pre-trained data set consisting of the amino acid sequence of the protein, the SMILES string of the small molecule and the binding constant between the protein and the small molecule is input into different large language models, and the base large language model is obtained by screening; Low-order adapter LoRA fine-tuning base large language model using large language model; Optimize parameters in the generated tasks; Obtain a prediction model based on the fine-tuned base language model and optimized parameters; The amino acid sequence of the protein to be predicted and the SMILES string of the small molecule are input into the prediction model, and the binding constant between the protein and the small molecule is output.

2. The method for predicting the binding constant between a protein and a small molecule according to claim 1, characterized in that: The method of obtaining the amino acid sequence of the protein, the SMILES string of the small molecule and the binding constant between the protein and the small molecule includes: The method for obtaining the amino acid sequence of a protein is as follows: obtaining the protein ID number from the PDBbind data set, and downloading the fasta amino acid sequence file from the PDB database according to the ID number; The method to obtain the SMILES string of a small molecule is: use Chem.MolFromMol2File(ligand) to convert the mol2 file of the small molecule into a SMILES string; The binding constants between proteins and small molecules are collected from the PDBbind dataset.

3. The method for predicting the binding constant between a protein and a small molecule according to claim 1, characterized in that: The pre-trained dataset consisting of the amino acid sequence of the protein, the SMILES string of the small molecule, and the binding constants between the protein and the small molecule is fed into different large language models including: Construct a dictionary in the format of {(protein amino acid sequence, small molecule SMILES string): combined constant}, convert the input format of the dictionary and input it into different large language models.

4. The method for predicting the binding constant between a protein and a small molecule according to claim 3, characterized in that: The conversion function of the input format is:

5. The method for predicting the binding constant between a protein and a small molecule according to claim 1, characterized in that: The screening to obtain the base large language model includes: The CASF dataset is used to test different large language models. The evaluation indicators include: the number of valid molecules output, the proportion of digital results in the output results, and the distribution of output results in space. The optimal large prediction model is selected as the base large language model based on the evaluation indicators.

6. The method for predicting the binding constant between a protein and a small molecule according to claim 5, characterized in that: The large language models include LLaMA-3, Mistral and ChemDFM; the base large language model obtained by screening according to the evaluation index is LLaMA-3.

7. The method for predicting the binding constant between a protein and a small molecule according to claim 1, characterized in that: The low-order adapter LoRA fine-tuning base large language model using a large language model includes: The pre-trained parameters are fixed and a bypass structure is added to the network; Assume that the pre-trained weight is W0 and the trainable parameter is ΔW; for the input x with this weight, the output is as follows: h=W0x+ΔWx Use the prompt word fine-tuning method to fine-tune the base large language model, and set a specified module when fine-tuning the model, which is a key module in the transformer layer; The model fine-tuning method using HugginFace is based on the supervised fine-tuning trainer SFTTrainer.

8. The method for predicting the binding constant between a protein and a small molecule according to claim 1, characterized in that: The parameters in the generation task include: top_k: When sampling, only the top k words with the highest probability are considered; top_p: When sampling, only the words whose probability accumulation reaches this value are considered; repetition_penalty: Penalize repeated words to prevent duplicate content; temperature: controls the "randomness" of the generated text; the higher the temperature, the more random the generated text.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images