A drug design method based on autoregressive model

Abstract

The invention discloses a drug design method based on an autoregressive model, and belongs to the technical field of drug design. The method comprises: applying a subword segmentation algorithm to biological text processing, training the protein and ligand information in a data set, obtaining a segmenter for the protein and the ligand, and constructing a segmenter for the autoregressive model on this basis; processing and converting the original data in the data set into a text form suitable for the autoregressive model, and then encoding it through the obtained segmenter to construct a training data set required for the autoregressive model; using the training data set to train the autoregressive model so that it can understand the SMILES representation of the ligand and learn the interaction mode between the protein and the ligand; using the trained autoregressive model to generate predicted ligand data, and post-processing through a chemical information tool to obtain candidate ligands with specific chemical structures; evaluating and optimizing the obtained candidate ligands to determine candidate molecules with the potential to become effective drugs.

Description

Technical Field

[0001] The present invention belongs to the technical field of drug design, and in particular relates to a drug design method based on an autoregressive model. Background Art

[0002] In recent years, drug discovery and development has made significant progress in computational chemistry and bioinformatics. Traditional drug design strategies include molecular docking, structure-activity relationship (QSAR), pharmacophores, unsupervised learning, deep learning, and generative adversarial networks (GANs).

[0003] Molecular docking is a technique used to simulate the interaction between small molecules and large molecules (usually proteins) and predict the optimal binding mode of small molecules in the active sites of proteins. This process includes multiple steps: first, the active site of the protein is identified, then the small molecule is placed in multiple possible positions in the active site, and possible binding modes are searched by rotating its chemical bonds and changing its conformation. The interaction energy is calculated for each possible binding posture, which usually involves factors such as van der Waals forces, hydrogen bonds, and electrostatic interactions. Ultimately, the most likely binding mode is determined based on the principle of minimum energy. It still has certain limitations:

[0004] Accuracy issue: Scoring functions commonly used in molecular docking attempt to predict the strength of intermolecular interactions. However, these scoring functions are usually based on simplified physical models and may not accurately reflect the complex biomolecular environment, such as dynamic water molecule effects and changes in ionic strength.

[0005] Computational complexity: The computational demands of molecular docking increase dramatically for molecules with multiple active sites or very large molecules. Furthermore, accounting for the flexibility of the ligand and target molecules further increases the computational complexity and required computational resources.

[0006] False positive and false negative problems: The docking process may produce high-scoring false positive results (apparently good binding) or miss truly valid binding poses (false negatives) due to inadequacies in the scoring function. This limits the reliability of docking for high-throughput screening.

[0007] Structure-activity relationship (QSAR) is a technique that uses statistical methods to establish relationships between molecular structural features and their biological activity. The core of this approach is to construct a mathematical model linking descriptors (such as molecular physicochemical properties, topological indices, and molecular graphs) with biological activity data. However, it still has certain limitations:

[0008] 1. QSAR models strongly rely on the quality and coverage of the input data. When the available compound dataset is unrepresentative or too small, the model's predictive performance may degrade significantly.

[0009] 2. Overfitting: When there are a large number of descriptors and limited training samples, QSAR models are prone to overfitting, that is, the model performs well on the training set but performs poorly on unseen data. This reduces the model's generalization ability.

[0010] The pharmacophore is based on the concept of identifying and defining key chemical features responsible for the biological activity of a molecule (such as hydrogen bond donors and acceptors, hydrophobic regions, ion docking receptors, etc.). These features are combined into a "virtual template" to screen or design new molecules that can interact with specific biological targets. The process of constructing a pharmacophore model usually involves identifying common features of known active molecules and using them as a basis to deduce the minimum required feature combination that can induce the desired biological response. Its limitations are as follows:

[0011] 1. This approach may be overly simplified and fail to consider the full range of molecular properties, such as flexibility and overall shape, which may lead to inaccurate activity predictions.

[0012] 2. Dynamic changes in target proteins (such as conformational changes) are often not considered in pharmacophore models. Due to the static nature of the models, they may not accurately predict the actual behavior of the molecule in vivo.

[0013] 3. The success of pharmacophore models depends heavily on the quality and relevance of the selected reference molecule set. If the reference molecules are insufficient or if there is significant variation in activity between molecules, the model may not accurately capture the true, universal pharmacophore characteristics.

[0014] Unsupervised learning involves learning patterns from data without explicit output labels, such as clustering compounds to discover new structures with similar biological activities. However, it still has certain limitations: it cannot directly predict specific biological activities or potencies, as its output is typically a new representation or classification of the data, rather than a direct activity prediction.

[0015] Early deep learning models primarily involved structures such as multilayer perceptrons (MLPs), convolutional neural networks (CNNs), and simple recurrent neural networks (RNNs). These models learned complex nonlinear relationships from large amounts of data to predict properties such as the biological activity of molecules or the toxicity of drugs. They automatically extracted features from data through stacked linear and nonlinear processing layers. However, they still had certain limitations:

[0016] 1. Lack of zero-shot learning: Zero-shot learning refers to a model's ability to make predictions about unseen categories without direct experience. Early deep learning models were often unable to make accurate predictions about new biological targets or compounds without training data. This is because they rely heavily on the data provided during training and lack the ability to generalize learned knowledge to novel situations.

[0017] 2. Lack of an attention mechanism: Attention mechanisms allow models to dynamically focus on the most relevant information when processing data. Early deep learning architectures (such as early CNNs and RNNs) lacked this mechanism, which limited their efficiency and accuracy when processing complex biological data with multiple key features. For example, in a drug molecule structure, certain specific parts may contribute significantly to its activity, but early models were unable to effectively identify and prioritize these key regions.

[0018] Generative adversarial networks (GANs) generate new molecular structures that are chemically plausible by training two networks: a generator and a discriminator. The generator attempts to create new compounds, while the discriminator evaluates the authenticity and effectiveness of these compounds. However, they still have certain limitations:

[0019] 1. The generated compounds may not be feasible in actual chemical synthesis or biological applications, because the structures generated by GANs may not take into account the synthetic feasibility or biological stability of the molecules.

[0020] 2. When training GANs, you may encounter mode collapse. This occurs when the generator begins to produce very similar or identical outputs, rather than diverse data. Mode collapse typically occurs when the generator finds a specific output strategy that "tricks" the discriminator and then repeats this strategy. In drug design, this means the generated compounds may be too unique and fail to cover a sufficiently broad range of chemical space, thus limiting the effectiveness of GANs in discovering new drug candidates.

[0021] In short, existing technologies still have shortcomings. Generally speaking, there are problems with low computational efficiency and poor design success rate, which leads to the fact that the research and development of new drugs still faces huge challenges and high costs. Summary of the Invention

[0022] In order to overcome the above-mentioned shortcomings of the prior art, the object of the present invention is to provide a drug design method based on an autoregressive model to improve the efficiency and success rate of drug design.

[0023] In order to achieve the above object, the technical solution adopted by the present invention is:

[0024] A drug design method based on an autoregressive model comprises the following steps:

[0025] Obtain a ligand dataset and a protein-ligand dataset, use a subword segmentation algorithm to train the protein and ligand information in the dataset, use the protein information to train to obtain a protein segmenter, and use the ligand information to train to obtain a ligand segmenter; and use the protein segmenter and the ligand segmenter to construct an autoregressive model segmenter;

[0026] Process and transform the raw data in the dataset into a text form suitable for the autoregressive model, and then encode it through the autoregressive model's word segmenter to construct the training dataset required for the autoregressive model;

[0027] The autoregressive model is trained using the constructed training data set so that it can understand and learn the interaction pattern between protein and ligand;

[0028] The trained autoregressive model is used to generate predicted ligand data, which are then post-processed using chemical information tools to obtain candidate ligands with specific chemical structures.

[0029] The obtained candidate ligands are evaluated and optimized, and chemical or bioinformatics tools are used for structural optimization and activity prediction to identify candidate molecules with the potential to become effective drugs.

[0030] In one embodiment, the ligand dataset is provided by a dataset such as ZINC, PubChem, ChEMBL, or DrugBank, and ligand information is extracted for learning the representation of ligand molecules, and the data is in SMILES format;

[0031] The protein-ligand dataset is provided by the datasets BindingDB, PDBbind, BioLIP or BindingMOAD. Protein information is extracted for learning the representation of protein amino acid sequences, and protein-ligand pair information is extracted for learning the correspondence between proteins and ligands. The data is in FASTA format and SMILES format.

[0032] In one embodiment, the objects used by the subword segmentation algorithm include: a textual representation of a ligand, a textual representation of a protein, and a textual representation composed of a protein and a ligand;

[0033] The subword segmentation algorithm breaks rare words into meaningful subwords without splitting common words into smaller subwords.

[0034] In one embodiment, the subword segmentation algorithm BPE algorithm is used to train the SMILES representation of the ligand in the jglaser / binding_ library to obtain a ligand segmenter; the segmenter contains the ligand vocab.json storage vocabulary and merges.txt storage merge operations, and the vocabulary size is 3560;

[0035] The protein vocabulary size is set to 50,000, and the subword segmentation algorithm BPE algorithm is used to train the protein sequences in the jglaser / binding_affinity library to obtain a protein tokenizer; the tokenizer contains the protein vocab.json storage vocabulary and merges.txt storage merge operations;

[0036] When constructing the tokenizer for the autoregressive model, we first perform a union operation on the ligand and protein vocabulary lists, delete the same words and the same words with different merge operations, and after processing the duplicate words, we add the initial 256 characters in the BPE algorithm as the vocabulary to fill the gaps caused by deleting the duplicate words. After processing, the vocabulary size of the tokenizer for the autoregressive model is 53080.

[0037] In one embodiment, the training dataset includes a ligand text dataset and a protein ligand text dataset, and is converted from the ligand dataset and the protein ligand dataset by the following steps:

[0038] Extract ligand information from the ligand dataset and convert it into ligand text data. Generate the ligand text dataset through the word segmenter of the autoregressive model to enable the model to understand the representation of the ligand.

[0039] Protein-ligand pairs are extracted from the protein-ligand dataset and converted into protein-ligand text data. The protein-ligand text dataset is generated through the word segmenter of the autoregressive model, so that the model can capture the potential characteristics and activity relationships of the chemical structure to generate new compounds with potential drug activity.

[0040] In one embodiment, when constructing the ligand text dataset, SMILES sequences are extracted from the zinc20 dataset, and each SMILES sequence is converted into ligand text data in the format of '<|startoftext|>'+' <l>'+SMILES sequence+'<|endoftext|>'; the ligand text data is encoded by the word segmenter of the autoregressive model to form a ligand text dataset;

[0041] When constructing the protein-ligand pair text dataset, protein-ligand pairs are extracted from the jglaser / binding_affinity dataset, and the SMILES representation of the protein and ligand of each protein-ligand pair is converted into protein-ligand pair text data in the format of '<|startoftext|>'+' '+Protein sequence in this pair+' <l>'+SMILES of the ligand in the pair+'<|endoftext|>'; Use the tokenizer of the autoregressive model to encode the protein-ligand pair text data.

[0042] In one embodiment, the training of the autoregressive model includes:

[0043] The autoregressive model GPT2 was selected as the base model, and the deep learning framework PyTorch was used for model training. The protein ligand text dataset sequence was input, and the target output corresponding to each input data element was the next element in the sequence.

[0044] In one embodiment, generating predicted ligand data using the trained autoregressive model comprises:

[0045] When predicting possible ligands for a specific protein, the amino acid sequence or FASTA format of the protein is input, and the SMILES format of the ligand predicted for the protein sequence is output;

[0046] If the protein sequence and part of the ligand are known, when predicting the rest of the ligand, based on the input protein amino acid sequence or FASTA format, the known part of the ligand (i.e. "prompt") is also input, and the predicted ligand SMILES format with a specific beginning is output.

[0047] In one embodiment, the autoregressive model is deployed locally or on the web, using the GPT-2 autoregressive model based on the HuggingFace library as its core;

[0048] The protein sequence is used as input and the GPT-2 autoregressive model is used to generate the corresponding drug molecule sequence. The GPT-2 autoregressive model outputs the SMILES format and is filtered by OpenBabel to eliminate unreasonable drug generation. It is then converted to the SDF format by OpenBabel.

[0049] In one embodiment, the chemical or biological information tools include OpenBabel and RDKit.

[0050] The above-mentioned drug design method based on the autoregressive model provided by the embodiment of the present invention has the following beneficial effects compared with the prior art:

[0051] The present invention utilizes a GPT-like model to perform autoregressive training on known interacting protein-ligand pairs, enabling the model to capture underlying chemical structure features and activity relationships, thereby generating new compounds with potential pharmaceutical activity. Specifically, a tokenizer is constructed in drug design based on autoregressive models to process and understand chemical and biological sequence data. This tokenizer is implemented using an advanced subword tokenizer algorithm, BPE. This algorithm automatically identifies and merges frequently occurring character pairs or character combinations by analyzing specific patterns in protein sequences and ligand chemical structures, thereby forming an efficient and compact vocabulary. By tailoring the tokenizer to the drug design process, the model's ability to process complex chemical and biological sequence data is significantly improved, thereby enhancing the accuracy of drug molecular structure prediction. Furthermore, by optimizing the size and structure of the vocabulary, the computational complexity of the model is reduced, improving the efficiency of the training and prediction processes.

[0052] Furthermore, because autoregressive generative methods exhibit greater flexibility in drug design strategies and rely on the model's own output as input for prediction, they can better capture the relationship between chemical structure and activity, thereby improving the quality of generated compounds and ensuring the success rate of drug design. Furthermore, because autoregressive generative methods are more stable during training than generative adversarial networks (GANs), they can be more easily optimized and adjusted, thereby improving the efficiency of drug design.

[0053] In addition, these new compounds may have unique chemical structures and mechanisms of action, which can more effectively explore chemical space than existing technologies and provide a wider range of candidate compounds for drug development. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 An overall flow chart of a drug design method based on an autoregressive model provided in an embodiment of the present invention;

[0055] Figure 2 The present invention provides a specific operational flow chart of a drug design method based on an autoregressive model. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0057] Proteins are among the most fundamental molecules in living systems, performing numerous vital functions within cells. Proteins are typically composed of one or more polypeptide chains, which interact with other molecules through forces such as hydrogen bonds, ionic bonds, and van der Waals forces. Small molecules that interact with proteins are called ligands. These interactions are highly specialized because they can cause conformational changes in proteins, thereby affecting their function. For example, drugs often exert their effects by binding to proteins, making the study of protein-ligand interactions crucial for drug design.

[0058] In order to improve the efficiency and success rate of drug design, an embodiment of the present invention provides a drug design method based on an autoregressive model, which includes the following steps:

[0059] Step 1: Apply the subword segmentation algorithm to biological text processing, train the protein and ligand information in the dataset, obtain protein and ligand segmenters, and build an autoregressive model segmenter based on this.

[0060] Step 2: Process and transform the original data in the dataset into a text form suitable for the autoregressive model, and then encode it through the word segmenter of the autoregressive model to construct the text training dataset required for the autoregressive model.

[0061] Step 3: Use the constructed training data set to train the autoregressive model so that it can understand and learn the interaction pattern between protein and ligand.

[0062] Step 4: Generate predicted ligand data using the trained autoregressive model and post-process them using chemical information tools to obtain candidate ligands with specific chemical structures; evaluate and optimize the obtained candidate ligands, and use chemical or bioinformatics tools for structure optimization and activity prediction to identify candidate molecules with the potential to become effective drugs.

[0063] For step 1 above:

[0064] This embodiment of the present invention proposes a novel method for constructing a tokenizer within the field of drug design, specifically within drug design methods based on autoregressive models. Taking into account the specificities of drug design, including the complexity of ligand and protein structures and the high accuracy requirements of the drug design process, this embodiment of the present invention constructs a specialized tokenizer to process and understand chemical and biological sequence data. This method is based on improving the accuracy and efficiency of autoregressive models in processing chemical structure and biological sequence data, while also optimizing the computational burden of the drug design process.

[0065] The present embodiment uses an advanced subword segmentation algorithm (e.g., Byte Pair Encoding (BPE) or WordPiece) to implement a word segmenter. This algorithm automatically identifies and merges frequently occurring character pairs or combinations by analyzing specific patterns in protein sequences and ligand chemical structures, thereby forming an efficient and compact vocabulary. Furthermore, the present embodiment includes a series of training steps, using protein and ligand data from large bioinformatics databases to train the word segmenter, ensuring that it accurately reflects the characteristics of biochemical data.

[0066] To implement a tokenizer, this embodiment of the present invention uses an implementation method based on the Hugging Face Tokenizers library. This approach allows for flexible customization and optimization of tokenization algorithms to adapt to diverse biochemical data characteristics. Specifically, using the library's API, researchers can customize the vocabulary based on the SMILES representation of ligands and the amino acid sequence of proteins, thereby constructing an autoregressive tokenizer optimized for drug design.

[0067] The embodiments of the present invention achieve significant advantages and benefits of a word segmenter: First, by tailoring a word segmenter for the drug design process, the model's ability to process complex chemical and biological sequence data is significantly improved, thereby increasing the accuracy of predicting drug molecular structures. Second, by optimizing the size and structure of the vocabulary, the method reduces the computational complexity of the model and improves the efficiency of the training and prediction processes. In addition, the embodiments of the present invention can also provide drug researchers with a more accurate and efficient tool to help them discover new drug candidates with potential value during the drug design and development process, accelerate the research and development cycle of new drugs, and reduce research and development costs.

[0068] For step 2 above:

[0069] This embodiment of the present invention proposes an efficient data processing method for converting raw data from pharmaceutical research into a format suitable for training autoregressive models. Given the advantages of autoregressive models in generative tasks, particularly in drug design, where they can predict the next possible compound structure based on given sequence data, this step aims to provide high-quality, efficient training data for these models. By optimizing the data processing process, this method aims to improve the efficiency of model training and the quality of generated compounds.

[0070] The embodiment of the present invention adopts a series of technical steps to process the raw data to form a text data training set, so that it is suitable for the training needs of the autoregressive model. First, the raw data (including the SMILES string of the ligand and the amino acid sequence of the protein) are preprocessed and converted into a unified text format. Then, the autoregressive model word segmenter constructed in step 1 is used to encode these text data to generate a training data set that can be directly used by the model. This process includes serializing chemical and biological information into a series of recognizable symbols or tags, which are used by the model to learn the structural characteristics and biological activity of the compound.

[0071] To create a text data training set, the present invention utilizes high-level programming languages ​​and data processing libraries (such as Python and its ecosystems, Pandas and NumPy), as well as specialized word segmentation tools (such as Hugging Face's Tokenizers library). The flexibility and power of these tools make extracting, converting, and encoding data from raw data in various sources and formats feasible and efficient.

[0072] By implementing precise and efficient data processing steps, the present invention provides multiple advantages and benefits when implementing text data training sets. First, it ensures the quality and consistency of training data, which is crucial for training high-performance autoregressive models. Second, through optimized data encoding, it reduces the time and resources required for model training, accelerating the pace of new drug discovery.

[0073] For step 3 above:

[0074] The present invention utilizes advanced autoregressive models to capture and understand the complex interaction patterns between proteins and ligands. In the field of drug design, accurately predicting protein-ligand interactions is crucial for identifying effective drug molecules. Traditional methods often rely on time-consuming experimental methods or computationally expensive simulations. The present invention trains autoregressive models to learn these interaction patterns directly from large-scale biochemical data, aiming to improve the efficiency and accuracy of drug candidate discovery.

[0075] This step details the process of training an autoregressive model. The key is to train the model using a specially preprocessed protein-ligand text dataset. The training goal is to enable the model to accurately predict the next element in a given sequence, thereby learning the interaction patterns between proteins and ligands. This process also involves randomly splitting the dataset into a training set and a validation set. The training set is used for model learning, while the validation set is used to evaluate the model's learning performance.

[0076] When selecting a model and training framework for model training, choose an autoregressive model architecture suitable for processing sequential data, such as the Transformer or GPT series models. In this embodiment of the present invention, GPT-2 was selected as the base framework for the autoregressive model due to its excellent performance in processing sequential data. The model was implemented and trained using the PyTorch deep learning library, which provides a flexible API and efficient computing power to support the rapid development and training of autoregressive models.

[0077] Training process and optimization strategy during model training. During the model training process, the embodiment of the present invention adopts a series of optimization strategies to improve training efficiency and model performance. First, an appropriate optimizer (such as Adam or SGD) is selected to minimize the loss function, which is crucial for fast convergence and avoiding overfitting. Secondly, the training process is further optimized by carefully configuring training parameters (such as learning rate, batch size, and training cycle) and implementing a strategy for dynamically adjusting the learning rate. The dataset segmentation strategy ensures the generalization ability of the model on various data. By learning on the training set and evaluating on the validation set, the training status of the model can be effectively monitored and adjusted.

[0078] To train the model, embodiments of the present invention can utilize various machine learning and deep learning frameworks, such as TensorFlow or PyTorch. These tools provide flexible and powerful APIs that support the design, training, and validation of autoregressive models. Furthermore, leveraging existing model libraries, such as the Hugging Face Transformers library, can further accelerate development and experimentation. These libraries provide pre-trained models and training techniques to help improve model performance.

[0079] The model training of the embodiment of the present invention demonstrates several key advantages:

[0080] (1) Efficient Learning and Improved Accuracy: By precisely training the autoregressive model, embodiments of the present invention enable a deep understanding of the complex interactions between proteins and ligands, significantly enhancing support for drug design. Leveraging deep learning capabilities, the model can accurately learn the patterns of these interactions, thereby improving the accuracy of drug candidate predictions.

[0081] (2) Optimizing Performance and Efficiency: The embodiments of the present invention utilize optimized training strategies and parameter adjustments to make the model training process more efficient and quickly achieve the desired performance level. Compared to traditional drug discovery methods, they can process large-scale datasets more quickly, accelerating the identification of potentially valuable drug candidates.

[0082] (3) Flexibility and Scalability: The choice of GPT-2 and PyTorch as implementation tools not only ensures strong model performance but also provides a high degree of flexibility and scalability for future research and applications. This choice of technical framework offers broad application prospects for drug design, enabling the exploration of new drug molecules and revealing new areas that may not be discovered by traditional methods.

[0083] For step 4 above:

[0084] The present invention utilizes a series of highly specialized ligand post-processing steps to identify promising candidate ligands from the extensive ligand data generated by an autoregressive model. These steps address the inefficiencies and low success rates inherent in traditional drug discovery. By combining computational modeling with cheminformatics tools, they enable the rapid and accurate identification and optimization of active ligand candidates. This approach is based on the principle that an automated and intelligent post-processing process can not only significantly accelerate the drug candidate screening process but also improve its accuracy and reliability.

[0085] Ligand generation in the examples of the present invention: using a trained autoregressive model, SMILES format data of the ligand is generated based on the amino acid sequence of a specific protein or known partial ligand information.

[0086] Ligand post-processing in the examples of the present invention: using chemical informatics tools to perform preliminary screening on the generated ligands, eliminating ligands with unreasonable chemical structures, retaining candidate ligands with reasonable structures, and converting them into SDF format.

[0087] Evaluation and optimization of candidate ligands in the examples of the present invention: 3D conformational generation of candidate ligands in SDF format is performed, and candidate molecules with the potential to become effective drugs are further screened through methods such as stereochemical analysis and synthetic feasibility assessment.

[0088] The present invention utilizes cheminformatics tools such as Open Babel and RDKit during the ligand generation and post-processing stages. These tools enable accurate chemical structure analysis and conversion of SMILES-formatted data, ensuring the rationality of the candidate ligand's chemical structure and the feasibility of the molecule. During the model training and ligand generation stages, deep learning frameworks such as PyTorch are employed to support complex autoregressive model structures and efficient model training.

[0089] Advantages and benefits of ligand generation and post-processing according to embodiments of the present invention:

[0090] (1) Efficiency improvement: Through automated ligand generation and intelligent post-processing processes, the efficiency of screening valuable candidates from a large number of chemical structures has been greatly improved.

[0091] (2) Enhanced accuracy: Combining advanced computational models and professional cheminformatics tools can more accurately identify ligands with reasonable chemical structures and potential for drug activity.

[0092] (3) Flexibility and scalability: It allows the prediction and screening process to be customized according to different drug development needs, ensuring the wide applicability and good scalability of the method.

[0093] (4) Comprehensive evaluation advantage: Through multi-dimensional structure optimization and activity prediction, the embodiments of the present invention not only consider the chemical rationality of the ligand, but also evaluate its biological activity and synthetic feasibility, providing a comprehensive scientific basis for drug development.

[0094] Combine Figure 1 and Figure 2 , the following steps are specifically described in detail in the embodiment of the present invention:

[0095] Step 1: Build a tokenizer for the autoregressive model.

[0096] Step 1.1: Obtain a ligand dataset and a protein ligand dataset, and use a subword segmentation algorithm to train the ligand information in the dataset to establish a ligand tokenizer, that is, to establish a labeled ligand vocabulary.

[0097] Step 1.2: Use the same subword segmentation algorithm as in step 1.1 to train the protein information in the dataset to obtain a protein segmenter, that is, to obtain a labeled protein vocabulary.

[0098] Step 1.3: Construct a tokenizer for the autoregressive model using the contents of the ligand tokenizer and the protein tokenizer. Specifically, combine the tokenizers obtained in steps 1.1 and 1.2 into a combined vocabulary, remove duplicate words from the combined vocabulary, and obtain the tokenizer for the autoregressive model. This is done by unioning the tokenized ligand and protein vocabulary and removing duplicate words to form the vocabulary for the autoregressive model.

[0099] In the present embodiment, ligand datasets are provided by ZINC, PubChem, ChEMBL, or DrugBank. Ligand information is extracted for learning the representation of ligand molecules. The data format is SMILES. Protein ligand datasets are provided by BindingDB, PDBbind, BioLIP, or BindingMOAD. Protein information is extracted for learning the representation of protein amino acid sequences. Protein-ligand pair information is also extracted for learning the correspondence between proteins and ligands. The data formats are FASTA and SMILES.

[0100] As a technical application in natural language processing tasks, subword segmentation algorithms typically include: BPE (BytePair Encoding), Byte-level BPE, WordPiece, Unigram, and SentencePiece. It can also include improvements based on commonly used subword segmentation algorithms, such as improvements to the BPE algorithm, which may include removing unused characters from the initial character set, or customizing the initial character set. The main idea of ​​the subword segmentation algorithm is to decompose rare words into meaningful subwords, while common words are not decomposed, that is, common words are not split into smaller subwords. The embodiment of the present invention applies it to the field of drug design for processing biological texts, for example, biological texts include SMILES format of ligands and text data of protein sequences. In the embodiment of the present invention, the objects used by the subword segmentation algorithm include: text representation of ligands, text representation of proteins, and text representation composed of proteins and ligands.

[0101] Step 2: Construct a text dataset (i.e., a training dataset). Convert the original data into a text form suitable for the autoregressive model, and then encode it through the autoregressive model's word segmenter to construct the training dataset required by the autoregressive model.

[0102] Step 2.1: Construct a ligand text dataset.

[0103] Extract ligand information from the ligand dataset and convert each sequence of ligand information into ligand text data. Then, use the subword segmentation algorithm in step 1, i.e., the tokenizer, to encode the ligand text data to form a ligand text dataset to help the model understand the ligand representation.

[0104] Step 2.2: Construct a protein-ligand pair text dataset.

[0105] Protein-ligand pairs are extracted from the protein-ligand dataset and converted into protein-ligand pair text data. The autoregressive model's tokenizer is then used to encode the protein-ligand pair text data to generate a protein-ligand text dataset. This helps the autoregressive model capture the potential features of chemical structures and activity relationships, enabling the generation of new compounds with potential drug activity.

[0106] Take the construction of protein-ligand pair text dataset as an example:

[0107] The protein PI3Kα interacts with its ligand BYL719, thus forming a protein-ligand pair: wherein the amino acid sequence of PI3Kα is as follows (also shown in SEQ NO.1):

[0108]

[0109] The SMILES representation of the ligand is as follows:

[0110] CC1=C(SC(=N1)NC(=O)N2CCCC2C(=O)N)C3=CC(=NC=C3)C(C)(C)C(F)(F)F

[0111] The conversion into protein-ligand pair text data is as follows:

[0112] '<|startoftext|>'+'< / l> '+Protein sequence in this pair+' <l>'+SMILES of the ligand in the pair+'<|endoftext|>'

[0113] So the result is:

[0114] <|startoftext|>< / l> <l>CC1=C(SC(=N1)NC(=O)N2CCCC2C(=O)N)C3=CC(=NC=C3)C(C)(C)C(F)(F)F<|endoftext|>

[0115] Then, the tokenizer of the autoregressive model constructed in step 1 is used to segment the protein-ligand pair text data. The results are as follows:

[0116] ['<|startoftext|>','<','P','>','MPP','RPS','S','GELWGIHLMPPRILVECLLPNGMIVTLECLREA','TLITIK H','ELFKEARK','YPLHQLL','QD','ES','S','YIFVSVTQEAEREEFFDETRRLCDLRLFQPFLKVIEPVGNREEKILNREI',

[0117] 'GFAIGM', 'PVCEFDMVKDPEVQDFFRRNILNVC','KEAVDLRDLNSPHSRAMYVY', 'PPN', 'VES', 'S','PELPKHIYNKLDKGQIIVVIW','VIVS','PNNDK','QKYTL','KIN','H','DCVPE','QVIA','EAIR','KKTRS','MLLS','S','EQL','KLCVLEYQ','GKYILKVCGCDEYFLEKY','PLSQYK','YI','RSC','I','ML','GRMPNLMLMAKESL','YSQL','PMDCFT','MPSYSRRISTATPYMN','GETS','TK','SLW','VIN','S','AL','RIKIL','CATYVN','VNIRDIDKIYVRTGIYHGGEPLCDNVNTQRVPCSNPRWNEWLNYDIYIPDLP RAARLCLSICS','VKGRK','GAKEE','HC','PLA','W','GNINLFDYTDTLVSGKMALNLWPVPHGLEDLLNPIGVTGSNPNKETPCLELEFDWFSSVVKFPDMSVIEEHAN', 'WSVS', 'REA', 'GF', 'S','YSHAGLSNRLARDNELRENDKE','QLKAIS','TRD','PLSEITEQEKDFLWSHRH','YCVTIPE','ILPK','LLLS','VK','WN','S','RDE',' VAQMYCL','VKDWPPIK','PEQA','ME','LLDC','NY','PDPM','VRGFA','VRCLEKYLTDDKLSQYLIQLVQVLKYEQYLDNLLVRFLLKKALTNQRIGHF FFWHLKSEMHNKTVSQRFGLLLESYCRACGMYLKHLNRQVEAMEKLINLT DILKQEKKDETQKVQMKFLVEQMRR',

[0118] 'PDFMDALQGFLSPLNPAHQLGNLRLEEC','RI','MS','SA','KR','PLWLNWENPDIMSELLFQNNEIIFKNGDDLRQDMLTLQIIRIMENIWQNQGL DLRMLPYGCLSIGDCV','GLIEVV','RN','S','HTIM','QIQC','KGGLKGAL','QF',

[0119] 'NSHTLHQWLKDKNKGEIYDAAIDLFTRSCAGYCVATFILGIGDRHNSNIMV KDDGQLFHIDFGHFLDHKKKKFGYKRERVPFVLTQDFLIVISK','GAQEC','TK','TREFERFQEMCYKAY','LAIRQHA','NLFI','NLF','S' ,'MML','GSGMPELQSFDDIAYIRKTLALDKTEQEALE','YFMK','QMNDAHHGGWTTKMDWIFHTIKQHALN','<','L','>','CC','1','=', 'C','(','SC','(=','N','1',')','NC','(=','O',')','N','2','CCCC','2','C','(=','O',')','N',')','C','3','= ','CC','(=','NC','=','C','3',')','C','(','C',')(','C',')','C','(','F',')(','F',')','F','<|endoftext|>']

[0120] In the above results, each word is enclosed by " and separated by commas.

[0121] The above vocabulary can be found by searching the vocabulary list of the tokenizer in step 1 to find the corresponding token ID for a specific vocabulary word, which is an integer. Each token ID has a one-to-one correspondence with a specific token in the dictionary. The encoding process is completed by converting each word in the given text into a corresponding token ID and arranging these token IDs in the order of the original text. The result is as follows:

[0122] [53082,28,48,30,1828,745,51,16584,16972,16930,16565,353,350,51,16579,15247,16445,16402,1740,633,51,16411,3591,15953,15991,2037,40,16293,15337,4385,15560,5786,51,44137,16545,16417,16 317,403,2555,41,320,16368,14807,16945,16631,6606,372,3301,2260,51,262,15797,16310,16530,15774,16462,616,765,55,16489,16268,665,318,51,16985,16988,1492,16394,16318,7433,1112,302,510, 51,1299,16378,16380,4520,359,29665,493,35003,16943,16949,16451,346,357,451,480,16490,11166,340,51,16236,16313,16603,490,16606,16953,372,16423,16323,2307,5124,51,3671,16437,14846,166 30,28,44,30,49778,17,29,35,8,49851,49779,46,17,9,49783,49779,47,9,46,18,49802,18,35,49779,47,9,46,9,35,19,29,49778,49779,49783,29,35,19,9,35,8,35,49793,35,9,35,8,38,49793,38,9,38,0]

[0123] Finally, after encoding, the computer can use it for calculation of integers, and then the autoregressive model can be trained on this basis.

[0124] Step 3: Model training, that is, using the constructed training data set to train the autoregressive model so that it can understand and learn the interaction pattern between protein and ligand.

[0125] When training the autoregressive model, use the protein ligand text dataset obtained in step 2. The sequence of the protein ligand text dataset is input into the autoregressive model, and the target output corresponding to each input data element is the next element in the sequence. During training, the dataset is randomly shuffled and divided into a training set and a validation set. The training set is used to train the model, and the validation set is used to evaluate the training results.

[0126] In one embodiment, this step can be further described as follows:

[0127] Step 3.1: Select the autoregressive model GPT2 as the base model and use a deep learning framework such as PyTorch for model training.

[0128] Step 3.2: Select an appropriate optimizer to minimize the loss function and configure appropriate training parameters and strategies. Split the dataset into training and validation sets, and use an appropriate batch size during training.

[0129] Step 4: Post-processing for generated ligands.

[0130] Step 4.1: Generate predicted ligand data using the trained autoregressive model. The output SMILES format data is processed using cheminformatics tools to eliminate unsuitable ligands and identify candidate ligands with specific chemical structures. These candidate ligands are then converted to SDF format. Common cheminformatics tools, such as Open Babel and RDKit, can be used to evaluate the generated predicted ligand data and confirm their plausibility.

[0131] When using the trained model for prediction, the prediction process depends on the specific application scenario. When predicting possible ligands for a specific protein, the protein's amino acid sequence or FASTA format is input, and the output is a SMILES format prediction of the ligand for the protein sequence. If the protein sequence and part of the ligand are known, when predicting the rest of the ligand, the known part of the ligand (i.e., "prompt") can be input at the same time as the protein's amino acid sequence or FASTA format, and the output is a SMILES format prediction of the ligand with a specific beginning.

[0132] Step 4.2: Evaluate and optimize the resulting candidate ligands, using chemical or bioinformatics tools for structural optimization and activity prediction to identify candidates with the potential to become effective drugs. Specifically, generate 3D conformations of the SDF files and assess the rationality of their drug structures based on stereochemistry and synthetic feasibility.

[0133] In one embodiment, the above-mentioned subword segmentation algorithm is a Byte Pair Encoding (BPE) segmentation algorithm, and its steps are as follows:

[0134] (1) Initialize the vocabulary. Consider all characters in the text as basic units and construct an initial vocabulary.

[0135] (2) Count the frequency of occurrence of character pairs. For a given text data, count the frequency of occurrence of each adjacent character pair.

[0136] (3) Merge high-frequency character pairs. Select the character pairs with the highest frequency and merge them into a new character. At the same time, update the vocabulary and text data, replacing the original character pairs with the new characters.

[0137] (4) Repeat steps 2-3 according to the preset number of merges or vocabulary size until the predetermined conditions are met.

[0138] In one embodiment, the dataset is from the zinc20 dataset and the jglaser / binding_affinity dataset.

[0139] The ligand tokenizer uses the BPE algorithm to train the SMILES representation of the ligands in the jglaser / binding_affinity library. The tokenizer contains the vocab.json storage vocabulary of the ligand and the merges.txt storage merge operations. The vocabulary size is 3560.

[0140] In one embodiment, the protein vocabulary size is set to 50,000, and the BPE algorithm is used to train the protein sequences in the jglaser / binding_affinity library to obtain a protein tokenizer, which contains the protein vocab.json storage vocabulary and merges.txt storage merge operations.

[0141] In one embodiment, a tokenizer for constructing an autoregressive model first performs a union operation on the ligand and protein vocabularies. The ligand and protein information is represented by SMILES. Since there are characters in the SMILES representation that overlap with the amino acid representation, it is necessary to process duplicate words when integrating the vocabularies. The vocab.json and merges.txt files of the ligand and protein tokenizers are checked separately, and the same words and the same words with different merge operations are deleted. After processing the duplicate words, the initial 256 characters in the BPE algorithm are added as words to make up for the gaps caused by deleting duplicate words. The word list size of the tokenizer after processing is 53080 (excluding special characters).

[0142] In one embodiment, when constructing the ligand text dataset, SMILES sequences are extracted from the zinc20 dataset, and each SMILES sequence is converted into ligand text data in the format of '<|startoftext|>'+' <l>'+SMILES sequence+'<|endoftext|>'. This text data is encoded using the autoregressive model's tokenizer to form the ligand text dataset. During training, 0.0001% of the Zinc20 dataset is randomly sampled in each epoch as the training set, and the last 0.00001% of the Zinc20 dataset is used as the validation set.

[0143] In one embodiment, when constructing a protein-ligand pair text dataset, protein-ligand pairs are extracted from the jglaser / binding_affinity dataset, and the SMILES representation of the protein and ligand of each protein-ligand pair is converted into protein-ligand pair text data in the format of '<|startoftext|>'+'< / l> < / l> '+Protein sequence in this pair+' <l>'+SMILES of the ligand in the pair+'<|endoftext|>', and then encode it using the tokenizer of the autoregressive model. When training and validating protein-ligand pair text data, all data were used, and the training and validation sets were divided into a ratio of 9:1.

[0144] In one embodiment, an autoregressive language model uses the GPT-2 model (GPT2LMHeadModel) based on the Hugging Face library as the core of the method to design drugs targeting specific proteins. The protein sequence is used as input, and the GPT2LMHeadModel model is used to generate the corresponding drug molecule sequence (such as SMILES representation). By training and optimizing the GPT2LMHeadModel model, candidate molecules with potential drug activity are generated, thereby improving the efficiency and success rate of drug design.

[0145] In one embodiment, model training uses the transformers and datasets library developed by the HuggingFace team. GPT2LMHeadModel from the transformers library is selected as the base model, and training is performed using PyTorch. The AdamW optimizer is used, with a learning rate of 5e-4, 100 warmup steps, and an epsilon value of 1e-8. The dataset is split into training and validation sets in a 9:1 ratio, and training is performed with a batch size of 8. The learning rate is adjusted using the linear warmup strategy (get_linear_schedule_with_warmup).

[0146] In one embodiment, when training the model, a loss function needs to be defined where y is a real sequence, is the sequence generated by the model, which can be expressed as the negative log-likelihood of the conditional probability. During the training process, the model parameters θ are optimized by gradient descent to minimize the loss function.

[0147] In one embodiment, the model output is in SMILES format and filtered through Open Babel to remove unreasonable drug generation. Open Babel then converts the generated drug files into SDF format. However, directly naming the generated drug files using the SMILES format can result in excessively long file names, which may be subject to file name length restrictions in the file system. Therefore, the converted SDF files are named using a SMILES hash value, such as the SHA1 hash algorithm. A mapping table is created in the output directory to store the correspondence between SMILES and hash values, ensuring that file names are not subject to length restrictions.

[0148] In one embodiment, the autoregressive model can be deployed locally, leveraging the advantages of local computing devices to achieve higher computational efficiency and more flexible scheduling strategies. It supports running on Windows and Linux operating systems. Users only need to follow the instructions in the README document or by running python drug_generator.py -h.

[0149] In one embodiment, the autoregressive model can be deployed on the web. For example, by deploying it on Google Colab, users who are not familiar with the installation process can use DrugGPT more easily, and cloud GPU resources can be fully utilized to significantly improve operating efficiency.

[0150] In summary, to address the low success rates, high R&D costs, and long development cycles of current drug design methods, the present invention first constructs a GPT-like model and trains it using a large number of known protein-ligand interaction pairs. This model is then used to generate potential small molecule compounds, which are then screened and optimized through methods such as molecular docking, QSAR, and pharmacophore analysis. This approach leverages advanced natural language processing technology to explore the chemical space relevant to drug design, discovering new molecules with potential binding to specific proteins. This provides a rapid and efficient new approach for generating drug candidate molecules.

[0151] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.< / l> < / l>

Claims

1. A drug design method based on an autoregressive model, characterized in that: The steps include: Obtain a ligand data set and a protein-ligand data set, use a subword segmentation algorithm to train the protein and ligand information in the data set, use the protein information to train to obtain a protein segmenter, and use the ligand information to train to obtain a ligand segmenter; and use the protein segmenter and the ligand segmenter to construct an autoregressive model segmenter; Process and transform the original data in the data set into a text form suitable for the autoregressive model, and then encode it through the word segmenter of the autoregressive model to construct the training data set required for the autoregressive model; The autoregressive model is trained using the constructed training data set so that it can understand and learn the interaction pattern between protein and ligand; The trained autoregressive model is used to generate predicted ligand data, which are then post-processed by chemical information tools to obtain candidate ligands with chemical structures. Evaluate and optimize the obtained candidate ligands, and use chemical or bioinformatics tools to perform structural optimization and activity prediction to identify candidate molecules that have the potential to become effective drugs; The subword segmentation algorithm BPE algorithm is used to train the SMILES representation of the ligand in the jglaser / binding_ library to obtain the ligand segmenter; the segmenter contains the vocab.json storage vocabulary of the ligand and the merges.txt storage merge operation, and the size of the vocabulary is 3560; The word list size of proteins is set to 50,000, and the subword segmentation algorithm BPE algorithm is used to train the protein sequences in the jglaser / binding_affinity library to obtain the protein word segmenter; the word segmenter contains the protein vocab.json storage word list and merges.txt storage merge operations; When constructing the tokenizer of the autoregressive model, the ligand and protein vocabulary lists are firstly unioned to delete the same words and the same words with different merging operations. After processing the repeated words, the initial 256 characters in the BPE algorithm are added as the vocabulary to make up for the vacancies caused by deleting the repeated words. After processing, the vocabulary size of the tokenizer of the autoregressive model is 53080.

2. The drug design method based on the autoregressive model according to claim 1, characterized in that: The ligand data set is provided by the data set ZINC, PubChem, ChEMBL or DrugBank, and the ligand information is extracted for learning the representation of ligand molecules, and the data format is SMILES format; The protein-ligand dataset is provided by the datasets BindingDB, PDBbind, BioLIP or BindingMOAD. Protein information is extracted for learning the representation of protein amino acid sequences, and protein-ligand pair information is extracted for learning the correspondence between proteins and ligands. The data is in FASTA format and SMILES format.

3. The drug design method based on the autoregressive model according to claim 1 or 2, characterized in that: The objects used by the subword segmentation algorithm include: a textual representation of a ligand, a textual representation of a protein, and a textual representation composed of a protein and a ligand; The subword segmentation algorithm breaks rare words into meaningful subwords without splitting common words into smaller subwords.

4. The drug design method based on the autoregressive model according to claim 1, characterized in that: The training data set includes a ligand text data set and a protein ligand text data set, which is obtained by transforming the ligand data set and the protein ligand data set, and the steps are as follows: Extract ligand information from the ligand dataset and convert it into ligand text data. Generate a ligand text dataset through the tokenizer of the autoregressive model so that the model can understand the representation of the ligand. Protein-ligand pairs are extracted from the protein-ligand dataset and converted into protein-ligand text data. The protein-ligand text dataset is generated through the word segmenter of the autoregressive model, so that the model can capture the potential characteristics and activity relationships of the chemical structure to generate new compounds with potential drug activity.

5. The drug design method based on the autoregressive model according to claim 4, characterized in that: When constructing the ligand text dataset, extract the SMILES sequence from the zinc20 dataset and convert each SMILES sequence into ligand text data in the format of '<|startoftext|>'+' <l> '+SMILES sequence+'<|endoftext|>'; Encode the ligand text data through the tokenizer of the autoregressive model to form a ligand text dataset;< / l> When constructing the protein-ligand pair text dataset, extract the protein-ligand pairs from the jglaser / binding_affinity dataset and convert the SMILES representation of the protein and ligand of each protein-ligand pair into protein-ligand pair text data in the format of '<|startoftext|>'+' '+Protein sequence in this pair+' <l>'+SMILES of the ligand in the pair+'<|endoftext|>'; The tokenizer of the autoregressive model is used to encode the protein-ligand pair text data. < / l> 6. The drug design method based on the autoregressive model according to claim 5, characterized in that: The training of the autoregressive model comprises: The autoregressive model GPT2 was selected as the basic model, and the deep learning framework PyTorch was used for model training. The protein ligand text dataset sequence was input into the autoregressive model, and the target output corresponding to each input data element was the next element in the sequence.

7. The drug design method based on the autoregressive model according to claim 6, characterized in that: The method of generating predicted ligand data using the trained autoregressive model comprises: When predicting possible ligands for a specific protein, the amino acid sequence or FASTA format of the protein is input, and the SMILES format of the ligand predicted for the protein sequence is output; If the protein sequence and part of the ligand are known, when predicting the rest of the ligand, based on the input protein amino acid sequence or FASTA format, the known part of the ligand is also input, and the predicted ligand with a specific beginning is output in SMILES format.

8. The drug design method based on the autoregressive model according to claim 6, characterized in that: The autoregressive model is deployed locally or on the web, using the GPT-2 autoregressive model based on the Hugging Face library as the core; The protein sequence is used as input and the GPT-2 autoregressive model is used to generate the corresponding drug molecule sequence. The GPT-2 autoregressive model outputs the SMILES format and is screened by Open Babel to eliminate unreasonable drug generation, and then converted to SDF format by OpenBabel.

9. The drug design method based on the autoregressive model according to claim 1, characterized in that: The chemical or biological information tools include Open Babel and RDKit.

Images