A drug-target binding affinity prediction method based on TransVAE

Abstract

The present invention relates to the field related to artificial intelligence and drug discovery. Specifically, a drug-target binding affinity (DTA) prediction method based on the combination of Transformer and Variational Autoencoder (VAE) is invented. It is used to solve two problems existing in most of the currently disclosed DTA prediction methods based on artificial intelligence: 1) Most methods only focus on the feature representation of drugs and targets, but ignore the impact of the sub-structure of drugs and targets on the DTA prediction results, that is, the feature acquisition is not fine-grained enough; 2) There is a lack of effective feature fusion methods, which leads to the problem of unsatisfactory DTA prediction effect. The present invention combines the advantages of the two models of Transformer and VAE, respectively encodes the features of drugs and targets, and then inputs them into the attention pooling module designed by the present invention for feature fusion, and finally inputs them into the prediction module for DTA prediction. The present invention can be applied to the prediction of drug-target binding affinity.

Description

[0001] Technical Field: The present invention relates to the field of artificial intelligence and drug discovery. Specifically, the present invention discloses a drug-target binding affinity (DTA) prediction method based on the combination of Transformer and Variational Autoencoder (VAE). Background Art

[0002] Artificial intelligence-based drug discovery has become a hot research area in recent years. The application of AI technology in drug discovery can significantly shorten drug discovery timelines and reduce repetitive workloads. Drug-target binding affinity prediction, in particular, can analyze the degree of drug-target interaction, facilitating the identification of optimal drug-target pairs (DTPs). However, AI-based drug-target binding affinity prediction is still in its early stages, with much research focused on predicting drug-target interactions while neglecting the prediction of drug-target binding affinity. While some institutions and researchers have investigated AI-based drug-target binding affinity prediction methods, existing approaches often neglect modeling specific characteristics of proteins and drugs. Therefore, the design of efficient drug-target binding affinity prediction methods is crucial.

[0003] 1. Professional terminology

[0004] (1) Deep Learning. Deep learning technology has achieved remarkable success in recent years, especially in natural language processing and image processing. This technology is developed from multi-layer neural networks. Its essence is to build a machine learning model with a large number of neural network hidden layers, and to train and learn more representative features through massive training data to increase classification accuracy. The main purpose of deep learning is "feature learning". Unlike traditional shallow learning, deep learning often has more hidden layers. Through the interaction of features between layers, the original feature representation is transformed into a new feature space, and then the training effect is optimized through the loss function and optimization function through the feature interaction information, thereby optimizing the model.

[0005] (2) Transformer. Transformer is a deep learning model based on the self-attention mechanism. Unlike traditional deep convolutional networks, the Transformer encoder consists of multi-head self-attention modules, each of which consists of multiple self-attention modules, and the layers are connected by residuals. The self-attention mechanism module is used to extract the tensor parameters of the input data, mainly including query (Q), key (K) and value (V). Among them, Q is used to interact with other key vectors (key vector) to obtain the weights of the vector (weights); K is used to interact with the query vector (query vector) to assist other vectors in outputting results; V is the result of summing the weights generated by Q and other K with its own weight. The self-attention mechanism can be expressed by the following formula:

[0006]

[0007] where d k are the dimensions of Q and K.

[0008] (3) Variational Autoencoder. Variational Autoencoder (VAE) is developed based on Autoencoder (AE). It upgrades the hidden layer of AE. VAE assumes the hidden layer after neural network encoding to be a standard Gaussian distribution, and then samples a feature from this distribution, and then uses this feature for decoding, transforming the encoding and decoding process of AE into a Bayesian probability model. VAE solves the problem of non-regularized latent space in AE and provides generative capabilities for the entire space. Unlike AE, VAE outputs the parameters of a predefined distribution in the vector space of each input, and then imposes constraints on this latent distribution, forcing it to become a normal distribution. This constraint ensures that the latent space is regularized. This makes its latent space generative.

[0009] 2. Related research progress

[0010] According to relevant literature at home and abroad, in recent years, good results related to drug-target binding affinity prediction have emerged one after another, such as DeepDTA (Ozturk et al. 2018), DeepAffinity (Karimi et al. 2019), GraphDTA (Nguyen et al. 2021), DeepDTAF (Wang et al. 2021), etc. Among them, DeepDTA uses two deep convolutional neural networks (CNNs) to encode drug and protein sequences respectively, which achieved good results at the time; DeepAffinity uses CNN to encode protein sequences and an unsupervised sequence-to-sequence model (Seq2Seq; Sutskever et al. 2014) to encode drugs, and then trains a regression model to predict drug-target binding affinity; GraphDTA uses graph neural networks (GNNs) to encode drugs and CNN to encode proteins, and then connects the encoded features to predict drug-target binding affinity; DeepDTAF uses the protein-binding pocket to directly process target binding, and then uses the dilated convolution method to process multi-scale DTP features, and then connects features at different scales to predict drug-target binding affinity.

[0011] Patent document (application number: CN202210418625.4) discloses a drug-target affinity prediction method based on representation learning and graph neural networks. The core idea of this document is to use CNN and GNN to extract features from the two-dimensional structure of drugs / proteins, and GNN to extract features from the three-dimensional structure of drugs / proteins. After fusing the extracted feature information, the drug-target affinity value is predicted through a fully connected layer. Patent document (application number: CN 202110028123.6) discloses a drug-target binding affinity prediction method based on graph convolutional neural networks. The method proposed in this document first uses a Bi-GRU network to extract contextual relationship features of the drug. Then, it uses graph convolutional neural networks (GCN), graph attention neural networks (GAT), graph isomorphism networks (GIN), and graph convolutional attention joint networks (GCN-GAT) to jointly model the drug molecules. The long short-term memory (LSTM) model is then used to extract the protein’s feature set. Finally, the drug-target binding affinity is predicted through a fully connected layer.

[0012] Problems with currently available methods for predicting drug-target binding affinity

[0013] The above methods are of certain reference value for understanding the progress of malware detection. However, these methods also have some problems, mainly including:

[0014] (1) Most of the currently available methods focus on the characterization of drug-target binding affinity pairs, but do not pay attention to the impact of the substructure of drugs and targets on the prediction results of drug-target binding affinity, that is, the feature acquisition is not fine-grained enough.

[0015] (2) Most deep learning-based drug-target binding affinity prediction methods simply concatenate the encoded features of drugs and proteins. This method does not take into account the impact of feature fusion on the prediction effect, making it difficult to further improve the effect of drug-target binding affinity prediction.

[0016] To address the above issues, the present invention discloses a drug-target binding affinity prediction method based on TransVAE. This method uses Transformer to model protein sequences, facilitating the memorization of long protein sequence position information, which is of positive significance for finding suitable targets. It also uses Variational Autoencoder (VAE) to model drug data. VAE effectively addresses the latent space generation capability of traditional autoencoders, making it easier to obtain key features that affect drug-target binding affinity prediction. Finally, an attention pooling module is designed to perform feature fusion, further improving the effectiveness of drug-target binding affinity prediction. Summary of the Invention

[0017] The purpose of the present invention is to solve the shortcomings of existing drug-target binding affinity prediction methods, such as insufficient granularity in feature acquisition and low accuracy in drug-target binding affinity prediction, and to invent a drug-target binding affinity prediction method based on TransVAE.

[0018] The technical solution adopted by the present invention to solve the above technical problems is: a drug-target binding affinity prediction method based on TransVAE, the method specifically comprising the following steps:

[0019] Step 1: Extract drug information, protein information, and drug-target binding affinity information from the database, where the affinity index mainly includes the dissociation constant K d , inhibition constant K i and half-maximal inhibitory concentration (IC) 50 wait;

[0020] Step 2: Data preprocessing: drug data uses SMILES, and target data uses protein sequence. After noise reduction, the drug feature matrix and protein feature matrix are obtained respectively.

[0021] Step 3: Concatenate the drug signature matrix and target signature matrix from step 2 to obtain drug-target pairs (DTPs). DTPs formed by drugs and targets with known interactions are positive, while the remaining DTPs are negative. Generally, the number of negative DTPs is much greater than the number of negative DTPs. In the present invention, an equal number of negative DTPs is randomly selected from the negative DTPs to form a dataset with the positive DTPs. A portion of the dataset is selected as a training set, and the remaining portion is selected as a test set.

[0022] Step 4: Construct a drug-target binding affinity prediction model based on TransVAE. The model is mainly divided into a data encoding module, a feature fusion module and a prediction module. The data encoding module is divided into two sub-modules: drug encoding and target encoding. The variational autoencoder (VAE) model is used to encode drug data, and the Transformer model is used to encode target data. The encoded features of the drug and target are then input into the attention pooling module for feature fusion to obtain the fused features. Finally, the fused features are input into the prediction module for drug-target binding affinity prediction.

[0023] Step 5: Set the hyperparameters of the TransVAE-based drug-target binding affinity prediction model, where the optimizer uses the adam function and its initial learning rate is set to 0.0005; the loss function mean squared error (MSE) is used to evaluate the difference between the predicted results and the actual solution. The model is trained using the N-fold cross-validation method, and the drug is input into the Transformer encoding module and the target data is input into the variational autoencoder module for encoding. The encoded features of the drug and target are then input into the attention pooling module for feature fusion. Finally, the fused features are input into the prediction module for DTA prediction. The model is trained for multiple iterations until it no longer converges to obtain the training results.

[0024] Step 6: Use the test set to test the model prediction effect. If the DTA prediction result is not ideal, continue to adjust the model's hyperparameters and repeat the training process in step 7 until the prediction module outputs a DTA test result that meets the accuracy requirements. Then stop training and obtain the trained DTA prediction model.

[0025] Step 7: Input any drug and target that meets the format requirements into the trained DTA prediction model. After model calculation, the DTA score of the input drug and target can be obtained.

[0026] The present invention has the following beneficial effects: It designs a TransVAE-based drug-target binding affinity prediction method. This method, leveraging the advantages of both Transformer and VAE, overcomes the issues of existing DTA prediction methods, such as insufficiently granular target feature acquisition and overly simplistic feature fusion, which results in low prediction accuracy. Experimental verification demonstrates that the prediction performance of this drug-target binding affinity prediction method on publicly available datasets is significantly improved compared to recently published drug-target binding affinity prediction models. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1: Diagram of the architecture of the TransVAE-based drug-target binding affinity prediction model designed by the present invention;

[0028] Figure 2 : Architecture diagram of the attention pooling module designed in this invention. DETAILED DESCRIPTION

[0029] In order to better understand the purpose, technical solutions and advantages of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and specific example embodiments. Those skilled in the art can easily understand the advantages and effects of the present invention from the contents disclosed in this specification, but the present invention is not limited in any form. It should be pointed out that for those of ordinary skill in the art, several changes and improvements can be made without departing from the idea of the present invention, and these all fall within the scope of protection of the present invention. Some embodiments of the specific examples of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments can be extended to all drug-target binding affinity prediction uses.

[0030] According to the present invention, a drug-target binding affinity prediction method based on TransVAE is provided. The main process of the present invention is shown in the attached Figure 1 , combined with specific examples, the main implementation steps include:

[0031] Step 1: Data acquisition. This example obtains data from two public drug-target binding affinity prediction datasets, Davis (Davis et al. 2011) and KIBA (Torng et al. 2020). The Davis dataset contains specific entries from the kinase protein family and related inhibitors, as well as their respective dissociation constants K. d (kinase dissociation constant) value, including 68 drugs and 442 targets, of which 30,056 drug-target pairs (DTPs). d The value ranges from 0.016 to 10,000. For ease of calculation, this example uses K d The value is converted into logarithmic value pK d As the value of drug-target binding affinity, it is calculated as: The KIBA dataset contains the combined bioactivity of kinase inhibitors from different sources, such as the dissociation constant K d , inhibition constant K i and half-maximal inhibitory concentration (IC) 50The dataset contains 246,088 drug-target pairs with KIBA scores ranging from 0 to 17.2. To ensure data consistency with existing algorithms, this example deletes drug-target pairs with KIBA scores below 10, ultimately obtaining 2,111 drugs and 229 targets, including 118,254 drug-target pairs (DTPs).

[0032] Step 2: Data preprocessing. Convert the drugs into SMILES format. This example uses the Pubchem online database (https: / / pubchem.ncbi.nlm.nih.gov / ) for conversion; the target sequence is converted using the UniProt protein database (https: / / www.uniprot.org / ); and then convert the drugs and targets into array format. To store the drug-target interaction in the form of a label, this example extracts the drug-target interaction features from all arrays and stores them in a label dictionary. In the label dictionary, the drug-target interaction feature is used as the primary key, and the value is an integer.

[0033] Step 3: Design the TransVAE model. Please refer to Figure 1 This embodiment combines the advantages of the Transformer, variational autoencoder, and attention pooling modules to design a TransVAE model for DTA prediction. The TransVAE model mainly includes a data encoding module, a feature fusion module, and a prediction module. The data encoding module is divided into two sub-modules: drug encoding and target encoding. The variational autoencoder (VAE) model is used to encode drug data, and the Transformer model is used to encode target data. The encoded drug and target features are then input into the attention pooling module for feature fusion to obtain fused features. Finally, the fused features are input into the prediction module for drug-target binding affinity prediction.

[0034] Step 4: Set the hyperparameters of the TransVAE model. In order to ensure the fairness of the model and avoid the occurrence of overfitting, this embodiment adopts a five-fold cross-validation training strategy, that is, each data set is randomly divided into 6 non-overlapping equal parts, 1 of which is used as a test case, and the other 5 parts are used to train the model in combination with the hyperparameters. The main hyperparameters used are the development environment, the number of layers of the Transformer encoder, the dropout rate value, the optimization function, the initial learning rate, the loss function, the number of iterations, the batch size, etc. Table 1 shows the main hyperparameters used in this embodiment and their corresponding values.

[0035] Table 1. Hyperparameters and corresponding values used in this example

[0036]

[0037] Step 5: Model training. The five training sets divided in step 4 are fed into the model for training. This example uses the micro-carving model with given hyperparameters for the two datasets mentioned above. Training is stopped when the model no longer converges, and the trained model is saved. The number of iterations for this implementation is 3000.

[0038] Step 6: Model testing, test the trained model using the test cases divided in step 4; the evaluation indicators used in this example test are mainly: Concordance Index (CI), Area Under the Receiver Operating Characteristic curve (AUROC) and Mean Squared Error (MSE); To verify the effectiveness of the model, this example selected recent advanced drug-target binding affinity prediction models for comparison, mainly KronRLS (Nascimento et al. 2016), DeepDTA (Ozturk et al. 2018), GANsDTA (Zhao et al. 2020), DeepCDA (Abbasi et al. 2020) and Co-VAE (Li et al. 2022). It should be pointed out that some models did not adopt the five-fold cross-validation training strategy in the original literature. To ensure fairness in model comparison, this example adopted the five-fold cross-validation training strategy for all models. Except for the parameter settings unique to the model itself, all other hyperparameters of all models remained consistent. The experimental performance data are shown in Table 2. Through experimental comparison, it is found that the performance of the method of the present invention is significantly better than the existing drug-target binding affinity prediction method (↑ represents the larger the better, ↓ represents the smaller the better).

[0039] Table 2. Comparison of experimental results

[0040]

[0041] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, including but not limited to implementing the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers in systems, devices, and modules designed based on the methods provided by the present invention, without affecting the essence of the present invention.

[0042] It should be understood that the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. The present invention is not limited to the processes and structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A drug-target binding affinity prediction method based on TransVAE, characterized in that: The following steps are involved: Step 1: Extract drug information, protein information, and drug-target binding affinity information from the database, where affinity information includes dissociation constant , inhibition constant and half-maximal inhibitory concentration ; Step 2: Data preprocessing: drug data is in SMILES format, and target data is in protein sequence format. After noise reduction, drug feature matrix and protein feature matrix are obtained respectively. Step 3: Concatenate the drug feature matrix and target feature matrix from step 2 to obtain drug-target pairs. DTPs formed by drugs and targets with known relationships are positive, and DTPs formed by drugs and targets without known relationships are negative. Randomly select the same number of negative DTPs as the number of positive DTPs from the negative DTPs to form a data set with the positive DTPs. Select a portion of the data set as the training set and the other portion as the test set. Step 4: Construct a TransVAE-based drug-target binding affinity prediction model, which is divided into a data encoding module, a feature fusion module, and a prediction module. The data encoding module is divided into two sub-modules: drug encoding and target encoding. The variational autoencoder model is used to encode drug data, and the Transformer model is used to encode target data. The encoded drug and target features are then input into the attention pooling module for feature fusion to obtain the fused features. Finally, the fused features are input into the prediction module for drug-target binding affinity prediction. Step 5: Set the hyperparameters of the TransVAE-based drug-target binding affinity prediction model. The optimizer uses the adam function, and its initial learning rate is set to 0.0005. The loss function, mean squared error, is used to evaluate the difference between the predicted results and the true solution. The model is trained using the N-fold cross-validation method. The drug data is input into the variational autoencoder module, and the target data is input into the Transformer module for encoding. The encoded drug and target features are then input into the attention pooling module for feature fusion. Finally, the fused features are input into the prediction module for DTA prediction. The model is trained for multiple iterations until it no longer converges, and the training results are obtained. Step 6: Use the test set to test the model's prediction performance. If the DTA prediction results are not ideal, continue adjusting the model's hyperparameters and repeat the training process in step 7 until the prediction module outputs a DTA test result that meets the accuracy requirements. Then, stop training and obtain a trained DTA prediction model. Step 7: Input any drug and target that meets the format requirements into the trained DTA prediction model. After model calculation, the DTA score of the input drug and target is obtained.

2. The drug-target binding affinity prediction method based on TransVAE according to claim 1, characterized in that: This method uses the attention pooling module to perform feature fusion. The attention pooling module first inputs the encoded drug and target features into the attention module, and the attention module then uses softmax The function is normalized and then input into the adaptive average pooling layer for weight learning. The weight values obtained after learning are matrix multiplied with the drug features and target features respectively to obtain the final fused features.

3. The drug-target binding affinity prediction method based on TransVAE according to claim 1, characterized in that: This method uses a prediction module to predict DTA scores. The module consists of three fully connected layers. The optimization function uses the adam function, and the loss function uses the mean squared error function to determine the gap between the predicted value and the true value.

Images