Drug target effect prediction method based on sequence feature fusion and attention mechanism

Abstract

The invention belongs to the technical field of artificial intelligence assisted drug design, and particularly relates to a drug target effect prediction method based on sequence feature fusion and an attention mechanism, comprising the following steps: extracting fragments; constructing a model: constructing a convolutional feature learning module and a multi-head interactive feature learning module, namely, performing fragment feature learning on an input sequence fragment by using a convolutional neural network module, inputting the obtained features into a multi-head interactive feature learning module containing a drug interactive attention block and a target interactive attention block for learning interactive features, and finally obtaining an interactive vector; and finally, predicting the interaction between the drug and the target. According to the method, noise input can be reduced, interaction information in the sequence can be effectively utilized, the prediction accuracy is improved, in actual drug development, the prediction speed of drug target interaction can be increased by using the method, and the method plays a great role in screening drugs, reducing side effects of the drugs and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence-assisted drug design, and specifically relates to a drug target action prediction method based on sequence feature fusion and attention mechanism. Background Art

[0002] Drugs achieve therapeutic effects by interacting with multiple targets in the body, among which targets are an important type of target whose function can be enhanced or inhibited by drugs to achieve phenotypic effects for clinical treatment purposes. Usually, researchers conduct a lot of biological and chemical studies in the laboratory to identify DTI in order to screen candidate drugs and even discover the hidden functions of targets. However, due to the large amount of molecular and target data, and the amount of data is still growing exponentially, screening one by one will consume a lot of time and resources, resulting in inefficiency. Therefore, it is necessary to develop efficient and accurate computational methods to predict DTI.

[0003] Deep learning (DL) and machine learning (ML) have achieved great success in natural language processing and medical imaging, and they also play a huge role in the field of bioinformatics. In terms of using traditional machine learning to predict DTI, similarity / distance, support vector machine (SVM), etc. are several commonly used methods. Since most traditional machine learning methods require manual selection and construction of features, have poor adaptability, and their accuracy needs to be improved, deep learning methods have been used by more and more researchers in recent years.

[0004] Drug-target interaction prediction methods based on deep learning usually regard DTI tasks as binary classification tasks, and are divided into two categories according to the form of their input data: sequence-based methods and graph-based methods. However, there are some common problems with current deep learning methods: the graph data used by graph-based methods will lead to a surge in computational complexity and high model complexity, which will limit the application of prediction models in practice; sequence-based methods can reduce computational costs, but they usually directly use complete sequences for prediction, which will introduce a lot of noise and have a greater impact on the prediction results. In addition, these methods ignore the interactions between sequence fragments and only focus on the overall structure of drugs and targets instead of focusing on substructures related to prediction. The interactive information in the sequence may be wasted. In addition, some current fragment segmentation methods are not suitable because they do not fully consider the impact of fragments on the entire sequence. Such fragment segmentation methods will reduce the accuracy of prediction. Therefore, it is necessary to provide a new method to solve the above problems. Summary of the invention

[0005] In response to the above technical problems, the present invention provides a drug-target action prediction method based on sequence feature fusion and attention mechanism, aiming to effectively solve the problems of low efficiency, ignoring the interaction between sequence fragments and introducing noise in existing methods, and realize accurate prediction of drug-target interaction.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] The drug target action prediction method based on sequence feature fusion and attention mechanism includes the following steps:

[0008] S1. Extract fragments: construct a sequence segmentation method to process the input drug sequence and target sequence to obtain various fragment types;

[0009] S2. Model construction: Construct a convolutional feature learning module and a multi-head interaction feature learning module: Use the convolutional neural network module to learn the fragment features of the input sequence fragments, and then input the obtained features into the multi-head interaction feature learning module containing the drug interaction attention block and the target interaction attention block to learn the interaction features, and finally obtain the interaction vector;

[0010] S3. Predict interactions: Use a prediction module to predict drug-target interactions based on the interaction vectors obtained above, and use multiple indicators to evaluate the prediction results.

[0011] The sequence segmentation method constructed in S1 is: for the SMILES sequence of the drug, the content in the brackets (i.e., the side chain of the drug) is used as the drug fragment; for the amino acid sequence of the target, it is cut using the 3-gram method, and the cutting method is to take every three connected amino acids as a fragment, thereby obtaining a variety of target fragments, and finally the obtained fragments are used to construct a drug fragment dictionary and a target fragment dictionary respectively.

[0012] The method for obtaining the interaction vector in S2 is: inputting the two segment dictionaries obtained in S1 into the convolutional feature learning module respectively to perform segment feature learning, and then inputting the obtained vector containing the segment features into the multi-head interaction feature learning module to obtain the final interaction vector.

[0013] The convolutional feature learning module in S2 contains four convolutional layers, each of which is composed of a Conv1d convolutional layer, a BatchNorm1d normalization layer, and an ELU activation function in sequence, and generates a vector representing the features of the entire segment by applying the convolution operation to each segment in the segment dictionary.

[0014] The method of inputting the obtained features in S2 into the multi-head interactive feature learning module for learning interactive features is: constructing a new multi-head interactive feature learning module based on the multi-head attention mechanism and the cross-attention mechanism, including a drug interactive attention block and a target interactive attention block; in the drug interactive attention block, the self-attention of the target is calculated according to the target feature vector to obtain the important features of the target, and then the original target feature vector is residually connected with the target feature vector after using the self-attention mechanism to obtain a new target feature vector; then the original drug feature vector and the new target feature vector are cross-attention operated: Q is obtained according to the drug feature vector, and K and V are obtained according to the new target feature vector; the number of heads of the multi-head attention is set to 8, and finally an interactive feature vector of the drug and the target is obtained by performing 4 cross-attention operations, and another interactive feature vector will be obtained by exchanging the direction of the drug feature and the target feature. The two interactive feature vectors are connected in one dimension to obtain the final interactive vector.

[0015] The prediction module in S3 is composed of 4 multi-layer perceptron layers, each of which is composed of a fully connected layer, a ReLU activation function and a Dropout layer in sequence. The interaction vector is used as input data, and the output obtained by passing through the 4 multi-layer perceptron layers is passed through a fully connected layer to obtain the final prediction value.

[0016] The prediction method in S3 is evaluated using precision, recall and area under the ROC curve AUC.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] The present invention first uses a sequence processing method to process a one-dimensional drug sequence and a target sequence to obtain a variety of sequence fragment types; then the sequence fragments are input into a convolutional feature learning module for feature learning, and then a multi-head cross attention mechanism is used to learn the interaction between the drug and the target, and finally input into a prediction module for prediction. The present invention can reduce noise input, effectively utilize the interactive information in the sequence, and improve the prediction accuracy. In actual drug development, the prediction speed of drug-target interaction can be accelerated by using the present invention, which has a great effect in screening drugs, reducing drug side effects, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0020] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0021] Figure 1 It is a schematic diagram of the flow chart of the prediction method of the present invention;

[0022] Figure 2 It is a schematic diagram of the prediction model structure of the present invention;

[0023] Figure 3 Schematic diagram of the execution process of the convolutional feature learning module in the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. These descriptions are only to further illustrate the features and advantages of the present invention, rather than to limit the claims of the present invention. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0025] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0026] In this embodiment, Figure 1 As shown in FIG. 1 , the drug target action prediction method based on sequence feature fusion and attention mechanism includes the following steps:

[0027] Step 1: Extract fragments: Construct a sequence segmentation method to process the input drug sequence and target sequence to obtain various fragment types.

[0028] In step 1, the sequence segmentation method includes: for the SMILES sequence of the drug, the content in the brackets (i.e., the drug side chain) is used as the drug fragment; for the amino acid sequence of the target, the 3-gram method is used to cut it (i.e., every three connected amino acids are used as a fragment), thereby obtaining a variety of target fragments, and finally the obtained fragments are used to construct a drug fragment dictionary and a target fragment dictionary respectively.

[0029] Step 2: Model construction: Figure 2As shown, a convolutional feature learning module and a multi-head interaction feature learning module are constructed: a convolutional neural network module is used to learn the fragment features of the input sequence fragments, and then the obtained features are input into a multi-head interaction feature learning module including a drug interaction attention block and a target interaction attention block for learning the interaction features, and finally an interaction vector is obtained.

[0030] In step 2, the two segment dictionaries obtained in step 1 are respectively input into the convolutional feature learning module to perform segment feature learning, and the obtained vector containing the segment features is input into the multi-head interactive feature learning module to obtain the final interactive vector. Step 2 includes the following steps 2-1 and 2-2:

[0031] Step 2-1: Convolutional feature learning module: The convolutional feature learning module contains four convolutional layers, each of which consists of a Conv1d convolution layer, a BatchNorm1d normalization layer, and an ELU activation function in sequence. By applying the convolution operation to each fragment in the fragment dictionary, a vector representing the features of the entire fragment is generated.

[0032] Step 2-2: Multi-head interactive feature learning module: The present invention constructs a new multi-head interactive feature learning module based on the multi-head attention mechanism and the cross-attention mechanism, including a drug interactive attention block and a target interactive attention block; in the drug interactive attention block, the target self-attention is calculated according to the target feature vector obtained in step 2-1 to obtain the important features of the target, and then the original target feature vector is residually connected with the target feature vector after using the self-attention mechanism to obtain a new target feature vector, and then the original drug feature vector obtained in step 2-1 is cross-attended with the new target feature vector: Q is obtained according to the drug feature vector, and K and V are obtained according to the new target feature vector; the number of heads of the multi-head attention is set to 8, and finally an interactive feature vector of the drug and the target is obtained by performing 4 cross-attention operations, and another interactive feature vector will be obtained by exchanging the direction of the drug feature and the target feature. The two interactive feature vectors are connected in one dimension to obtain the final interactive vector.

[0033] Step 3: Predict interactions: Figure 3 As shown, a prediction module is used to predict the drug-target interaction based on the interaction vector obtained above, and multiple indicators are used to evaluate the prediction results. Preferably, the prediction method of this embodiment is evaluated using precision, recall and area under the ROC curve AUC.

[0034] In step 3, the prediction module is composed of 4 multi-layer perceptron layers, each of which is composed of a fully connected layer, a ReLU activation function and a Dropout layer in sequence. The interaction vector is used as input data, and the output obtained by passing through 4 multi-layer perceptron layers is passed through a fully connected layer to obtain the final prediction value.

[0035] The loss function used in this embodiment is the cross entropy loss, and the formula is:

[0036]

[0037] The method of this embodiment is implemented using Pytorch and trained using the Adam optimizer. After multiple experiments to adjust the parameters, this embodiment sets the learning rate to 0.001, the batch size to 32, and each experiment runs 100 cycles, allowing early stopping when the effect is good.

[0038] The drug-target interaction prediction method based on sequence feature fusion and multi-head cross-attention mechanism proposed in this embodiment has AUC values ​​of 0.988, 0.992, and 0.911 on three public data sets commonly used for drug-target interaction prediction, respectively, proving that the drug-target interaction prediction method based on sequence feature fusion and multi-head cross-attention mechanism proposed in this embodiment has good prediction ability.

[0039] Only the preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention, and various changes should be included in the protection scope of the present invention.

Claims

1. A drug target action prediction method based on sequence feature fusion and attention mechanism, characterized in that: The following steps are involved: S1. Extract fragments: construct a sequence segmentation method to process the input drug sequence and target sequence to obtain various fragment types; S2. Model construction: Construct a convolutional feature learning module and a multi-head interaction feature learning module: First, use the convolutional neural network module to learn the fragment features of the input sequence fragments, and then input the obtained features into the multi-head interaction feature learning module containing the drug interaction attention block and the target interaction attention block to learn the interaction features, and finally obtain the interaction vector; S3. Predict interactions: Use a prediction module to predict drug-target interactions based on the interaction vectors obtained above, and use multiple indicators to evaluate the prediction results.

2. The drug target action prediction method based on sequence feature fusion and attention mechanism according to claim 1, characterized in that: The sequence segmentation method constructed in S1 is: for the SMILES sequence of the drug, the content in the brackets (i.e., the side chain of the drug) is used as the drug fragment; for the amino acid sequence of the target, it is cut using the 3-gram method, and the cutting method is to take every three connected amino acids as a fragment, thereby obtaining a variety of target fragments, and finally the obtained fragments are used to construct a drug fragment dictionary and a target fragment dictionary respectively.

3. The method for drug target action prediction based on sequence feature fusion and attention mechanism according to claim 1, characterized in that: The method for obtaining the interaction vector in S2 is: inputting the two segment dictionaries obtained in S1 into the convolutional feature learning module respectively to perform segment feature learning, and then inputting the obtained vector containing the segment features into the multi-head interaction feature learning module to obtain the final interaction vector.

4. The method for drug target action prediction based on sequence feature fusion and attention mechanism according to claim 1, characterized in that: The convolutional feature learning module in S2 contains four convolutional layers, each of which is composed of a Conv1d convolutional layer, a BatchNorm1d normalization layer, and an ELU activation function in sequence, and generates a vector representing the features of the entire segment by applying the convolution operation to each segment in the segment dictionary.

5. The method for drug target action prediction based on sequence feature fusion and attention mechanism according to claim 1, characterized in that: The method of inputting the obtained features in S2 into the multi-head interactive feature learning module for learning interactive features is: constructing a new multi-head interactive feature learning module based on the multi-head attention mechanism and the cross-attention mechanism, including a drug interactive attention block and a target interactive attention block; in the drug interactive attention block, the self-attention of the target is calculated according to the target feature vector to obtain the important features of the target, and then the original target feature vector is residually connected with the target feature vector after using the self-attention mechanism to obtain a new target feature vector; then the original drug feature vector and the new target feature vector are cross-attention operated: Q is obtained according to the drug feature vector, and K and V are obtained according to the new target feature vector; the number of heads of the multi-head attention is set to 8, and finally an interactive feature vector of the drug and the target is obtained by performing 4 cross-attention operations, and another interactive feature vector will be obtained by exchanging the direction of the drug feature and the target feature. The two interactive feature vectors are connected in one dimension to obtain the final interactive vector.

6. The method for drug target action prediction based on sequence feature fusion and attention mechanism according to claim 1, characterized in that: The prediction module in S3 is composed of 4 multi-layer perceptron layers, each of which is composed of a fully connected layer, a ReLU activation function and a Dropout layer in sequence. The interaction vector is used as input data, and the output obtained by passing through the 4 multi-layer perceptron layers is passed through a fully connected layer to obtain the final prediction value.

7. The method for drug target action prediction based on sequence feature fusion and attention mechanism according to claim 1, characterized in that: The prediction method in S3 is evaluated using precision, recall and area under the ROC curve AUC.

Images