A drug target affinity prediction method, device and medium based on pseudo-label attention

Abstract

The present invention discloses a drug target affinity prediction method, device and medium based on pseudo-label attention, the method comprising: (1) extracting features and vectorizing the predicted protein and drug molecule pairs, and inputting them into a recurrent neural network model and a multi-layer perceptron model. (2) Using a pseudo-label attention model to generate vectors of multiple attention patterns for proteins and drug molecules respectively, and concatenating the two to obtain an interaction pattern vector sequence of protein-drug molecules. (3) Using a Transformer model to perform self-information aggregation and transmission on the interaction model vector sequence, and further performing nonlinear transformation on the output result through a feedforward neural network layer to obtain the final affinity prediction result of the protein and the drug molecule. Compared with general drug target affinity prediction methods, the present invention has stronger generalization ability and robustness, and the complexity is also linearly related to the length of the drug molecule or protein sequence, and has better scalability and length extrapolation.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and drug publication affinity prediction, and specifically relates to a drug target affinity prediction method based on pseudo-label attention. Background Art

[0002] Drug target affinity prediction based on deep learning has always been one of the core issues in the field of drug discovery. It aims to predict the affinity between proteins and drug molecules through deep learning models, taking the features of proteins and drug molecules as input. Existing drug target prediction models are generally divided into two categories. One is the double-tower model, which obtains vector representations of proteins and drug molecules respectively through independent protein feature extraction networks and drug feature extraction networks, and then concatenates the two and predicts affinity through a fully connected layer; the other is to directly concatenate protein sequences and drug molecule sequences at the sequence level, and send them to the Transformer network to learn the interaction between the two, and finally predict affinity through a fully connected layer. The former of these two methods lacks deep interaction at the protein residue level and the drug molecule atomic level, and it is often insufficient to interact only through the final fully connected layer at the global representation of protein and drug molecules; although the latter can fully deeply interact between the two at the residue level and the atomic level through the Transformer network, it often has huge time and space complexity overhead, because the complexity of the Transformer network is a quadratic function of the length of the input sequence. Some larger proteins contain tens of thousands of residues, which is extremely unfriendly to model training and machine overhead.

[0003] Combining the analysis of these two types of drug target prediction models, it is not difficult to find that the twin-tower model will design a powerful feature extraction network for proteins or drug molecules to extract as complete a global expression vector as possible, so as to retain the original residue or atomic level information as much as possible; while the Transformer model relies on multi-layer stacked self-attention layers and feedforward neural network layers for both feature expression and information interaction. The two have their own advantages, and of course there are many works that combine the two, but this combination still does not solve the huge time and space overhead problem of Transformer. Summary of the invention

[0004] The key problem to be solved by the present invention is to provide an efficient, low-complexity fine-grained information interaction scheme based on pseudo-label attention to address the problems of insufficient amino acid and atomic information interaction in existing drug target affinity prediction methods and excessive spatiotemporal overhead of the Transformer model.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0006] A drug target affinity prediction scheme based on pseudo-label attention, including:

[0007] (1) The input protein sequences and drug molecules are vectorized one by one amino acid, one atom, and one symbol in SMILES;

[0008] (2) Use BiLSTM and MLP to perform preliminary feature extraction on the vectorized protein and drug molecule data;

[0009] (3) Perform pseudo-label attention calculation on the extracted feature sequence, and calculate m attention modes for protein features and drug features respectively, where each attention mode corresponds to the characteristic mode of a certain type of protein or drug molecule. The m attention modes correspond to weighted summation of the entire protein or drug feature sequence in m weighted summation methods, and a total of m pseudo-label vectors of proteins and drug molecules can be obtained respectively;

[0010] (4) The m pseudo-label vectors of the protein and the m pseudo-label vectors of the drug molecule are concatenated and input into the Transformer model for self-interaction and information transfer to obtain the binding vector of the protein and the drug molecule, simulating the binding process of the protein and the small molecule;

[0011] (5) Use a feedforward neural network to combine the output of the Transformer model with the kmer features of the protein and the fingerprint features of the drug molecule to perform multi-layer nonlinear transformation and residual connection to achieve the final drug target affinity prediction;

[0012] Furthermore, the specific process of step (2) is as follows:

[0013] For the input x of protein sequence or drug molecule SMILES and atomic sequence at time t t and the historical information h of the previous t-1 moments t-1 , the input value at this moment can be calculated as:

[0014] h t =o t ⊙tanh(c t )

[0015] Among them t ,c t It's all about x t ,h t-1 The function is used to control the fusion of historical information and current input information to determine the information output at the current moment. LSTM operations are performed in two directions respectively. For the historical information in the positive and negative directions at time t and Splice to get the feature output at time t

[0016] Furthermore, the specific process of step (3) is as follows:

[0017] For the feature output of BiLSTM at time t A feed-forward neural network is constructed to calculate its score in m-class pseudo-label attention mode:

[0018]

[0019] FFN α represents a two-layer feedforward neural network layer with ReLU nonlinear activation function, represents the score of the feature output at time t in the m-class pseudo-label attention mode; for all α t Normalized on the first dimension, and the protein or drug molecule vector obtained by weighted summation in the j-th pseudo-label attention mode is calculated as:

[0020]

[0021] in It represents the score of the normalized output at time t under the jth pseudo label. A total of m such protein vectors or drug molecule vectors can be obtained respectively. The m vectors correspond to the attention patterns of protein sequences or drug molecules under m types of pseudo-labels. However, since these m types of attention patterns are not truly m types, but are obtained by the model through automatic clustering, we call them m pseudo-label attention vectors.

[0022] Furthermore, the specific process of step (4) is as follows:

[0023] The m pseudo-label vectors of proteins are concatenated with the m pseudo-label vectors of drug molecules to obtain a joint vector that combines protein features and molecular features:

[0024]

[0025] in m pseudo-label vectors representing proteins, Represents m pseudo-label vectors of drug molecules. The joint vector is subjected to self-interaction and information transfer based on the multi-head self-attention mechanism to simulate the interaction process between proteins and drug molecules:

[0026]

[0027] Where Q i ,K i ,Vi represents the three linear transformation layers of the i-th self-attention layer, d k Represents the number of neurons in each autonomous attention head and is transformed by a linear matrix W Z With bias b Z Integrate the expression of features. Finally, pass the feed-forward neural network layer FFN inter The final binding vector of the protein and drug molecule is obtained by connecting with the residual.

[0028] Furthermore, the specific process of step (5) is as follows:

[0029] First, we obtain the fingerprint vector of the drug molecule based on molecular fingerprint technology and protein amino acid statistical information kmer vector with protein And through two multi-layer perceptron networks MLP fgr ,MLP kmer Transform them to inter The same vector space, that is, the dimension is transformed to d h Then the binding vector of the protein and drug molecule is concatenated with the kmer vector and the fingerprint vector. For the first dimension 2m of the binding vector, we will use average pooling and maximum pooling techniques to aggregate, and finally we can concatenate a vector with a length of 4d. h The final unified representation vector of protein and drug molecules:

[0030] x uni =[pool max (x inter ),pool mean (x inter ),x fgr ,x kmer ]

[0031] where pool max ,pool mean Represents the maximum pooling and uniform pooling function, that is, for x inter The first dimension of is used to take the maximum or average value. Represents the final unified representation vector of protein and drug molecule. Finally, we use a multi-layer nonlinear transformation layer with residual connection The final predicted value of the affinity between the protein and the drug molecule is obtained.

[0032] Furthermore, we feed the prior data obtained from the experiment into the model for training, and use the mean square error between the actual affinity value and the predicted affinity value as the model's minimization objective, and optimize and learn all parameters of the entire model through the stochastic gradient descent algorithm.

[0033] Furthermore, h =256.

[0034] Furthermore, m=64.

[0035] Furthermore, FFN α , FFN inter , They both represent a two-layer neural network with 1024 and 256 neurons, and the output of the first layer is activated by the ReLU function.

[0036] Furthermore, MLP fgr ,MLP kmer Represents a three-layer neural network with 1024, 256, and 256 neurons, and the middle layers are activated by the ReLU function.

[0037] An electronic device comprises a memory and a processor, wherein a computer program is stored in the memory, and wherein when the computer program is executed by the processor, the processor implements any one of the above-mentioned drug target affinity prediction methods based on pseudo-label attention.

[0038] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, any of the above-mentioned drug target affinity prediction methods based on pseudo-label attention is implemented.

[0039] Beneficial Effects

[0040] The present invention invents a novel drug target affinity prediction method based on pseudo-label attention technology. The model fully obtains multiple aggregate representations at the residue level of proteins and the atomic level of drug molecules through the pseudo-label attention mechanism, and performs deep fine-grained information interaction and information transmission on these aggregate representations through subsequent Transformer layers. Since the number of these aggregate representations is a constant, the Transformer layer also has a constant level, or approximately equal to the linear level of complexity, and then the overall model realizes the fine-grained interaction between the residues of proteins and the atoms of drug molecules with extremely low complexity, and realizes high-precision affinity prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is the overall architecture diagram described in the embodiment of the present application;

[0042] Figure 2 It is a visualization of the theoretical space complexity analysis of the method described in the embodiment of the present application and the pure Transformer-like method; DETAILED DESCRIPTION

[0043] The following is a detailed description of an embodiment of the present invention. This embodiment is based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process to further explain the technical solution of the present invention.

[0044] This embodiment provides a drug target affinity prediction method based on pseudo-label attention, referring to Figure 1 As shown, the following steps are included:

[0045] 1. Model Input

[0046] Formally, for protein sequences, we use the embedding layer to encode amino acids as units, and we can get the following sequence vector:

[0047]

[0048] Among them l p represents the length of the protein sequence, d h Represents the dimension of the embedding.

[0049] For drug molecules, we use the SMILES sequence and the sequence of the concatenated atomic sequence, and the atomic order of the atomic sequence is consistent with that in SMILES. We also use the embedding layer for encoding, and we can get the following sequence vector:

[0050]

[0051] Among them l d represents the sum of the SMILES sequence length and the number of atoms, d h Represents the dimension of the embedding.

[0052] 2. Extraction of protein and drug molecule sequence features

[0053] For the extraction of protein sequence or drug molecule SMILES and atomic sequence features, we use a bidirectional LSTM network. t and the historical information h of the previous t-1 moments t-1 , the input value at this moment can be calculated as:

[0054] g t =tanh(W gx x t +W gh h t-1 +b g )

[0055] Where W gx , W gh 、b g are learnable weight coefficients and bias terms, xt The sequence embedding vector x representing the protein or drug molecule p ,x d The vector at the tth position. In addition, the input gate, forget gate, and output gate are calculated as follows:

[0056] i t =sigmoid(W ix x t +W ih h t-1 +b i )

[0057] f t =sigmoid(W fx x t +W fh h t-1 +b f )

[0058] o t =sigmoid(W ox x t +W oh h t-1 +b o )

[0059] Where W ix , W ih 、b i , W fx , W fh , b f , W ox , W oh , b o They are the learnable weight coefficients and bias terms of the input gate, forget gate, and output gate respectively;

[0060] Therefore, the input value at time t can be controlled based on the input gate and the forget gate to improve the performance on long-term dependency problems:

[0061] c t =f t ⊙c t-1 +i t ⊙g t

[0062] Where ⊙ represents the Hadamard product of the matrix;

[0063] Finally, the historical information h at the current time t is obtained through the output gate t :

[0064] h t =o t ⊙tanh(c t)

[0065] In addition, we performed LSTM operations in two directions (i.e., steps 1) to 3)) to calculate the historical information in both the positive and negative directions at time t. and Splice to get the feature output at time t

[0066] 3. Extraction of pseudo-label attention patterns for sequence features of proteins and drug molecules

[0067] First, for the feature output of BiLSTM at time t We construct a feed-forward neural network to calculate its score in m-class pseudo-label attention mode:

[0068]

[0069] FFN α represents a two-layer feedforward neural network layer with ReLU nonlinear activation function, represents the score of the feature output at time t in the m-type pseudo-label attention mode; let the score of the feature output at time t in the j-th pseudo-label attention mode be α t,j , take the scores of the feature outputs at all times in the jth pseudo-label attention mode and perform softmax normalization operation to obtain the importance of the feature outputs at each time in the jth pseudo-label attention mode:

[0070]

[0071] Thus, we can calculate the protein or drug molecule vector obtained by weighted summation under the j-th pseudo-label attention mode as:

[0072]

[0073] Finally, we can obtain m such protein vectors or drug molecule vectors respectively. The m vectors correspond to the attention patterns of protein sequences or drug molecules under m types of pseudo-labels. However, since these m types of attention patterns are not truly m types, but are obtained by the model through automatic clustering, we call them m pseudo-label attention vectors.

[0074] 4. Extraction of protein features combined with molecular features

[0075] The m pseudo-label vectors of proteins are concatenated with the m pseudo-label vectors of drug molecules to obtain a joint vector that combines protein features and molecular features:

[0076]

[0077] in m pseudo-label vectors representing proteins, m pseudo-label vectors representing drug molecules.

[0078] The joint vector is subjected to self-interaction and information transfer based on the multi-head self-attention mechanism to simulate the interaction process between protein and drug molecules:

[0079]

[0080] Z=[A 1 ,A 2 ,...]W Z +b Z

[0081] Where Q i ,K i ,V i represents the three linear transformation layers of the i-th self-attention layer, d k represents the number of neurons in each autonomous attention head, W Z ,b Z represents the learnable weight matrix and bias term, is the output of the multi-head self-attention layer.

[0082] Finally, the final binding vector of protein and drug molecule is obtained through the feedforward neural network layer with residual connection:

[0083] x inter =LayerNorm(v+Z)

[0084] x inter =LayerNorm(x inter +FFN inter (x inter ))

[0085] LayerNorm represents the layer normalization operation, FFN inter represents a two-layer feedforward neural network layer with ReLU nonlinear activation function, Represents the final binding vector of the protein and drug molecule.

[0086] 5. Combining protein kmer features and molecular fingerprint features to achieve the final affinity prediction

[0087] First, we obtain the fingerprint vector of the drug molecule based on molecular fingerprint technology and protein amino acid statistical information kmer vector with protein And transform them into x through a multi-layer perceptron network interThe same vector space:

[0088] x fgr =MLP fgr (v fgr )

[0089] x kmer =MLP kmer (v kmer )

[0090] Among them, MLP fgr ,MLP kmer Represent a two-layer and three-layer multilayer perceptron network with ReLU activation function, They represent the transformed protein kmer vector and the fingerprint vector of the drug molecule respectively.

[0091] The binding vector of the protein and drug molecule is concatenated with the kmer vector and the fingerprint vector. For the first dimension 2m of the binding vector, we will use average pooling and maximum pooling techniques to aggregate, and finally we can concatenate a vector with a length of 4d. h The final unified representation vector of protein and drug molecules:

[0092] x uni =[pool max (x inter ),pool mean (x inter ),x fgr ,x kmer ]

[0093] where pool max ,pool mean Represents the maximum pooling and uniform pooling function, that is, for x inter The first dimension of is used to take the maximum or average value. Represents the final unified representation vector of protein and drug molecules.

[0094] Finally, we obtain the final predicted affinity value between the protein and the drug molecule based on a multi-layer nonlinear transformation with residual connections:

[0095] x uni =x uni +FFN uni1 (BatchNorm(x uni ))

[0096] x uni =x uni +FFN uni2 (BatchNorm(x uni ))

[0097]

[0098] in represents a two-layer fully connected feedforward neural network with ReLU as the activation function, BatchNorm represents the batch normalization operation, denote the learnable weight matrix and parameters, respectively, to map the final result to a single predicted affinity value, is the final predicted affinity value.

[0099] VI. Experimental Verification

[0100] In order to verify the effectiveness of the present invention [hereinafter referred to as PseLabAttnDTA] in predicting drug target affinity and its performance superiority compared with other methods, this section evaluates the performance of PseLabAttnDTA through extensive experiments. We conducted affinity regression experiments on two classic affinity prediction data sets, Davis and Kiba, respectively, and used CI and MSE as evaluation indicators, where CI represents the monotonicity score evaluation after the predicted affinity is sorted according to the size relationship of the true affinity, and MSE represents the square error between the predicted affinity and the true affinity. We selected DeepDTA, GraphDTA, SGC_SAGE, DeepCDA and other twin tower models or sequence models based on graph neural networks, convolutional neural networks or Transforemr as experimental comparison baseline models, and in order to ensure the fairness of the experimental comparison, we all used the same data partitioning to obtain training sets, validation sets and test sets. The training set is used for model training and the best performing model on the validation set is saved during the training process. Finally, the results of the model on the test set are recorded and recorded in the following table.

[0101] Table 1 Performance comparison of PseLabAttnDTA and other baseline models

[0102]

[0103] From Table 1, it is not difficult to see that PseLabAttnDTA has achieved good performance on both Davis and Kiba datasets, which shows that PseLabAttnDTA has strong drug target affinity prediction ability and generalization, especially in the MSE index. It is better than the previous optimal method SGC_SAGE on both datasets, and the prediction error is reduced by 5.29% and 6.30% respectively. Since the regression task directly optimizes the MSE loss, this also fully demonstrates the superiority and robustness of PseLabAttnDTA in regression problems.

[0104] The above embodiments are preferred embodiments of the present application. Ordinary technicians in this field can also make various changes or improvements on this basis. Without departing from the patent concept of the present application, these changes or improvements should fall within the scope of protection required by the present application.

Claims

1. A drug target affinity prediction method based on pseudo-label attention, characterized in that: The following steps are included (1) The input protein sequences and drug molecules are vectorized one by one amino acid, one atom, and one symbol in SMILES; (2) Use BiLSTM and MLP to perform preliminary feature extraction on the vectorized protein and drug molecule data; (3) Pseudo-label attention calculation is performed on the extracted feature sequence. M attention modes are calculated for protein features and drug features respectively. Each attention mode corresponds to the characteristic mode of a certain type of protein or drug molecule. The m attention modes correspond to weighted summation of the entire protein or drug feature sequence in m weighted summation methods. A total of m pseudo-label vectors of proteins and drug molecules can be obtained respectively. The specific process is as follows: 1) For the feature output of BiLSTM at time t A feed-forward neural network is constructed to calculate its score in m-class pseudo-label attention mode: FFN α represents a two-layer feedforward neural network layer with ReLU nonlinear activation function, Represents the score of the feature output at time t in the m-type pseudo-label attention mode; 2) For all α t Normalized on the first dimension, and the protein or drug molecule vector obtained by weighted summation in the j-th pseudo-label attention mode is calculated as: in It represents the score of the normalized output at time t under the jth pseudo label. A total of m such protein vectors or drug molecule vectors can be obtained. The m vectors correspond to the attention patterns of protein sequences or drug molecules under m types of pseudo labels, which are called m pseudo label vectors; (4) The m pseudo-label vectors of the protein and the m pseudo-label vectors of the drug molecule are concatenated and input into the Transformer model for self-interaction and information transfer to obtain the binding vector of the protein and the drug molecule, simulating the binding process of the protein and the small molecule; (5) A feedforward neural network is used to combine the output of the Transformer model with the kmer features of the protein and the fingerprint features of the drug molecule to perform multi-layer nonlinear transformation and residual connection to achieve the final drug target affinity prediction.

2. The drug target affinity prediction method based on pseudo-label attention according to claim 1, characterized in that: The specific process of step (2) is as follows: (1) For the input x of protein sequence or drug molecule SMILES and atomic sequence at time t t and the historical information h of the previous t-1 moments t-1 , the input value at this moment can be calculated as: h t =o t ⊙tanh(c t ) Among them t ,c t It's all about x t ,h t-1 The function is used to control the fusion of historical information and current input information to determine the information output at the current moment; (2) Perform LSTM operations in two directions respectively, for the historical information in the positive and negative directions at time t and Splice to get the feature output at time t 3. The drug target affinity prediction method based on pseudo-label attention according to claim 1, characterized in that: The specific process of step (4) is as follows: (1) Concatenate the m pseudo-label vectors of proteins and the m pseudo-label vectors of drug molecules to obtain a joint vector that combines protein features and molecular features: in m pseudo-label vectors representing proteins, m pseudo-label vectors representing drug molecules; (2) The joint vector is subjected to self-interaction and information transfer based on a multi-head self-attention mechanism to simulate the interaction process between proteins and drug molecules: Where Q i ,K i ,V i represents the three linear transformation layers of the i-th self-attention layer, d k Represents the number of neurons in each autonomous attention head and is transformed by a linear matrix W Z With bias b Z Expression of integrated features; (3) Finally, through the feed-forward neural network layer FFN inter Connect with the residual to get the final protein-drug molecule binding vector, FFN inter They both represent a two-layer neural network with 1024 and 256 neurons, and the output of the first layer is activated by the ReLU function.

4. The drug target affinity prediction method based on pseudo-label attention according to claim 1, characterized in that: The specific process of step (5) is as follows: (1) First, the fingerprint vector of the drug molecule is obtained based on the molecular fingerprint technology and the amino acid statistical information of the protein. kmer vector with protein And through two multi-layer perceptron networks MLP fgr ,MLP kmer Transform them to inter The same vector space, that is, the dimension is transformed to d h ; (2) The binding vector of the protein and drug molecule is concatenated with the kmer vector and the fingerprint vector. For the first dimension 2m of the binding vector, average pooling and maximum pooling techniques are used for aggregation, and finally a 4d long vector is concatenated. h The final unified representation vector of protein and drug molecules: x uni =[pool max (x inter ),pool mean (x inter ),x fgr ,x kmer ] where pool max ,pool mean Represents the maximum pooling and uniform pooling function, that is, for x inter The first dimension of is used to take the maximum or average value. The unified representation vector representing the final protein and drug molecule; (3) Finally, based on a multi-layer nonlinear transformation layer with residual connections The final predicted affinity value between the protein and the drug molecule is obtained. They both represent a two-layer neural network with 1024 and 256 neurons, and the output of the first layer is activated by the ReLU function.

5. According to the drug target affinity prediction method based on pseudo-label attention in claim 1, the prior data obtained from the experiment is fed into the model for training, and the mean square error between the true affinity value and the predicted affinity value is used as the minimization target of the model, and all parameters of the entire model are optimized and learned by the stochastic gradient descent algorithm.

6. The drug target affinity prediction method based on pseudo-label attention according to any one of claims 1, 3, and 4, characterized in that d h ＝256。 7. The drug target affinity prediction method based on pseudo-label attention according to claim 1, characterized in that m＝64。 8. The drug target affinity prediction method based on pseudo-label attention according to claim 4, characterized in that MLP fgr ,MLP kmer Represents a three-layer neural network with 1024, 256, and 256 neurons, and the middle layers are activated by the ReLU function.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor is caused to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images