Drug-target interaction prediction method based on joint network attention

Abstract

The application belongs to the field of bioinformatics and relates to a drug and target interaction prediction method based on joint network attention. First, multi-task self-supervised feature learning is performed on the drug molecular graph and protein sequence to extract the context information of drug substructures and protein residues, thereby obtaining high-precision representation features. Second, a machine learning model is combined to realize the interaction between drugs and targets. Finally, the trained model realizes the interaction prediction between drugs and targets. The application can maintain excellent prediction performance and strong generalization ability under limited labeled data conditions. Experimental results show that the method has significant advantages in drug discovery, target screening and candidate drug mechanism identification, and can provide efficient and reliable tool support for new drug research and development.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of bioinformatics, and relates to a drug and target interaction prediction method based on joint network attention, which comprises technologies such as deep learning, LSTM, CNN and attention mechanism. BACKGROUND

[0002] Drug-target interaction prediction is an important part of drug research and development, and can support drug screening, target validation and candidate compound optimization. Although traditional experimental methods are accurate, they are time-consuming and costly, and are difficult to meet the needs of large-scale drug discovery. Existing computational methods such as molecular docking rely on high-quality three-dimensional structures, have high computational complexity and limited applicability; traditional machine learning relies on artificial features and is difficult to capture complex nonlinear relationships and has insufficient generalization ability; although deep learning has the advantage of automatic feature extraction, it still has bottlenecks in large-scale data processing and complex interaction mode modeling. Therefore, the application proposes a drug-target interaction prediction method based on joint network attention, which processes drug and target features by combining multiple networks, captures key interactions by combining kernel functions and attention mechanisms, improves prediction accuracy and efficiency, and enhances the generalization ability of the model, providing an efficient and reliable computational tool for drug screening and new drug design. SUMMARY

[0003] The application proposes a drug-target interaction prediction method based on joint network attention, which aims to improve the accuracy of drug-target interaction prediction, improve drug screening efficiency, and provide auxiliary support for new drug research and development. This method comprehensively utilizes drug molecular graph modeling, protein sequence embedding representation, self-supervised pre-training feature learning, joint network feature mapping and multi-head attention fusion, etc. It can fully explore the complex nonlinear relationship and multi-level interaction features between drugs and targets, thereby realizing efficient and stable interaction prediction. The specific technical process is as follows:

[0004] Step 1, first model the drug data, based on the linear symbolic representation sequence of the chemical structure, input the data in the form of a molecular graph, use the extended connectivity fingerprint algorithm to extract molecular substructures, and decompose a complete molecule into vocabulary units, each substructure unit is a token, and the entire molecule is regarded as a long sequence sentence. Then, convert the substructure into a vector embedding to form an n x d matrix, where n is the number of substructures and d is the feature dimension, and capture the context relationship between substructures through a Transformer encoder.

[0005] Step 2, the target data is modeled, and the original sequence is standardized, which does not rely on three-dimensional structure. Then, a large-scale pre-trained protein language model is used for sequence representation learning. The model learns the contextual dependency between residues through the mask language modeling task, captures the potential structural and functional features in the sequence under unsupervised conditions, and maps each amino acid residue to a high-dimensional embedding vector. The self-attention mechanism is used to effectively model the relationship between distant residues. Further, the contact relationship between residues can be inferred according to the attention weight, and the residue contact graph is constructed. The three-dimensional spatial information implied by the model is combined to enhance the description of the protein folding state and potential binding site. Finally, the sequence-level context embedding representation is output, which not only retains local residue features but also contains global structure information, which can be used as a key input for subsequent drug-target interaction prediction.

[0006] Step 3, the drug molecule data in step 1 and the target protein data in step 2 are pre-trained using a self-supervised feature learning method. For drugs, multi-task learning such as substructure mask prediction, molecular descriptor prediction, and functional group prediction is used. For proteins, residue-level context features are learned using mask language modeling.

[0007] Step 4, the pre-trained features of drugs and proteins in step 3 are input into a joint network attention module composed of LSTM layers, CNN layers, and SAM (self-attention mechanism) layers. The LSTM layer is used to capture the long-term dependencies of the sequence and obtain the global semantics between molecular substructures and protein residues. The CNN layer performs convolution operations on the LSTM output to extract local patterns and residual features. The SAM layer is composed of a self-attention mechanism and a multi-layer perceptron. The encoding features of drugs and proteins are input into the joint network, and the attention mechanism is introduced to highlight the key interaction sites. The attention weight can automatically identify the most important molecular substructure or protein residue for prediction, thereby enhancing the discriminability of the model. Based on the fusion representation, the attention weighted features are mapped to the standardized probability space through the fully connected prediction layer, and the prediction results of drug-target interaction are output, including the existence of interaction and the binding strength score.

[0008] Step 5, the binding strength score calculated in step 4 is used as the final drug-target interaction prediction result through a binary classification output layer using binary classification cross-entropy loss. The pre-trained drug and protein features obtained through self-supervised pre-training are input into the downstream prediction network, and the model is tested under three experimental settings: conventional planning, new drug prediction, and new target prediction. The prediction results are compared with the true labeled values.

[0009] A drug-target interaction prediction method based on joint network attention, the implementation process of step 1 is as follows: in the drug data preprocessing stage, first, the original molecular data is structured modeling. Take the linear symbolic expression sequence of chemical structure as input, and convert it into a graph structure through a molecular analysis tool, where atoms are mapped to nodes of the graph, and chemical bonds are mapped to edges between nodes. Each central atom and its directly connected atoms and chemical bonds are collectively regarded as a basic structural unit, thereby obtaining a two-dimensional molecular graph representation that can reflect the topological relationship of the molecule. On this basis, further decompose the molecular graph using the extended connectivity fingerprint algorithm, divide the whole molecule into several substructure units, and define each substructure as a basic symbolic unit. The entire molecule is regarded as a long sequence arranged in order according to the chemical bond relationship of these basic symbolic units. Then, the above substructure is symbolized into a vector form, and a matrix representation containing molecular substructure embedding is constructed. On the basis of this representation, the sequence modeling Transformer encoder is used to model the context relationship of the molecular sequence, thereby capturing the semantic dependency and combination rules between substructures.

[0010] A drug-target interaction prediction method based on joint network attention, the implementation process of step 2 is as follows: in the target data processing stage, take the amino acid primary sequence of the protein as the input source, without relying on experimentally determined three-dimensional structure data, introduce a large-scale pre-trained protein language model to learn the representation of the amino acid sequence. This kind of model learns the context dependency relationship between residues in an unsupervised condition through the mask language modeling task, thereby automatically capturing the structural features and functional features hidden in the protein sequence. In this way, each amino acid residue will be mapped to a high-dimensional embedding vector, which contains not only the properties of the residue itself, but also the context information in the global sequence environment. Further, the long-range dependency relationship between residues is modeled by using self-attention mechanism, and different weights are assigned to the importance of different residues to identify the residue pairs that are far apart in the sequence but have potential interaction in the spatial structure. Finally, the sequence context embedding representation containing residue-level features is output.

[0011] The method for predicting the association between drugs and targets based on joint network attention comprises the following steps: 3. The implementation process is as follows: the features of the drug molecules and the target proteins are pre-trained using self-supervised learning, and the robust and general representation features are automatically extracted from a large-scale unlabeled drug and protein database. In the drug molecule end, the molecule is first converted into a substructure sequence and vectorized embedded. In the self-supervised learning task, three prediction modes are set for different structures of the molecule respectively: (1) substructure mask prediction: part of the substructure token is randomly masked, and the model is required to restore the masked content according to the context, so as to learn the combination semantics between substructures; (2) molecular descriptor prediction: based on the overall molecular representation, the physicochemical properties calculated by the chemical information tool, such as molecular weight and polar surface area, are predicted, so as to strengthen the global representation ability of the model; (3) functional group prediction: through multi-label classification, it is judged whether there is a specific functional group in the molecule, such as hydroxyl, carboxyl and aromatic ring, so as to enhance the recognition ability of the model to local chemical fragments. The three types of tasks are jointly optimized, so that the model can learn both the local functional features of the molecule and the overall molecular semantics. In the protein end, the amino acid sequence is first standardized, and then input into the pre-trained protein language model. The mask language modeling strategy is adopted: part of the residues are randomly masked and the true class is predicted, so that the model learns the context dependence relationship between residues. At the same time, the self-attention mechanism is used to model the global sequence, and the potential interaction information of the long-distance residues is automatically mined. Further, the residue contact relationship can be inferred according to the attention weight, the residue contact graph is constructed, and the spatial information hidden in the model is combined, so as to enhance the representation ability of the protein folding state and the binding site. Finally, the output protein embedding vector contains both local residue information and global structure features, which can be used as high-quality input for drug-target interaction prediction.

[0012] A drug-target interaction prediction method based on joint network attention, the implementation process of step 4 is as follows: after the feature modeling of the drug and the target is completed, the representation obtained by pre-training of the present application is input into the joint network attention module. The module is composed of a long short-term memory network, a convolutional neural network and a self-attention mechanism. Among them, the long short-term memory network layer is used to depict the long-term dependence in the sequence, so as to obtain the global semantic information between the molecular substructure and the protein residue; the convolutional neural network layer performs local convolution operation on the output of the long short-term memory network layer, further extracts local mode and residual feature; the self-attention mechanism layer combines the self-attention mechanism and the multilayer perception, which can automatically highlight the key interaction sites. Through the module, the features of the drug and the protein are effectively fused on the multi-scale level, and the most important molecular fragments or residues for prediction are highlighted under the action of the attention mechanism. Finally, the fused representation is mapped to the standardized probability space through the full connection prediction layer, and the prediction result of the drug-target interaction is output, including whether the interaction exists and the quantitative score of the binding strength.

[0013] A drug-target interaction prediction method based on joint network attention, the implementation process of step 5 is as follows:

[0014] The model further maps the drug-target interaction representation calculated in the previous link to the binding strength score. The score is processed by a logical normalization function to convert it into a probability value in the interval [0,1], and the interaction prediction result is given by the binary classification output layer. During the training process, the present application adopts binary cross entropy as the optimization target, so that the model can maximize the probability difference between positive and negative samples. In order to verify the effectiveness of the method, the drug and protein features obtained by self-supervised pre-training are input into the downstream prediction network, and fine-tuning and testing are carried out on the existing labeled data set. In the experimental section, three scenes of regular data division, new drug prediction and new target prediction are set respectively, and the prediction results are compared with the real labeled values. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 It is a flow chart of a drug-target interaction prediction method based on joint network attention.

[0016] Figure 2 It is a flow chart of modeling drug data.

[0017] Figure 3 It is a flow chart of modeling target data.

[0018] Figure 4 It is a flow chart of self-supervised pre-training feature learning of drug and target data.

[0019] Figure 5 It is a joint network attention module structure. DETAILED DESCRIPTION

[0020] The application is described in detail below in conjunction with the accompanying drawings and examples.

[0021] The application proposes a drug-target interaction prediction method based on joint network attention, and the specific process is as shown in Figure 1 : modeling drug data, modeling target data, self-supervised pre-training feature learning of drug and target data, joint network attention training and drug-target interaction prediction five processes, and the specific steps are as follows:

[0022] Step 1, the modeling process of drug data is as shown in Figure 2 : first, based on the linear symbolic expression sequence of chemical structure, the molecule is parsed into a graph structure form input, wherein the node represents the atom in the molecule, and the edge represents the chemical bond between the atoms. On this basis, a substructure segmentation algorithm is used to decompose the complete molecule into several substructure units, each unit is regarded as a basic token, and the whole molecule is converted into a sequence representation composed of tokens. In order to further improve the preprocessing accuracy, an atom grouping based strategy is introduced in the substructure segmentation link, that is, taking the atoms on the ring as the center, extending to its edge, and taking the directly connected chemical bond and the group on the bond as a whole group for division. For example, for ethanol molecule ( ), first, take the methyl carbon ( ) as the center, divide the three hydrogen atoms connected to it and a carbon-carbon bond into a group, that is, form group; then take the methylene carbon ( ) as the center, take the two hydrogen atoms connected to it, a hydroxyl ( ) and a carbon-carbon bond with as a group, that is, form group; different carbon atoms connected structure units form different groups, so as to ensure that the molecule can retain local functional group information when it is disassembled, and avoid excessive fragmentation of the fragment. In this way, all molecular data are uniformly grouped and preprocessed to obtain a more chemically reasonable substructure unit set. Then, embed these tokens into d dimensional vector space to construct the embedding matrix of molecular sequence: ; wherein n represents the number of substructures, d represents the feature dimension. Then, the substructure is converted into a vector embedding to form a matrix of n × d , and finally the context relationship between substructures is captured by the Transformer encoder.

[0023] Step 2, the preprocessing process of target data is as shown inFigure 3 As shown: First, the target protein data is taken as the input source, and the original sequence is standardized, including removing illegal characters, uniform coding format, and segmenting or truncating long sequences, to ensure the consistency and integrity of the input, without relying on experimental determination of the three-dimensional structure; Then the standardized sequence is input into a large-scale pre-trained protein language model, which learns the context dependence between residues in an unsupervised manner through the mask language modeling task, that is, by randomly masking part of the residue symbols and requiring the model to predict their true categories according to the context, to capture the potential semantic and structural features; On this basis, each amino acid residue is mapped to a high-dimensional embedding vector, which not only contains the physicochemical properties of the residue itself, but also integrates the context features in the global sequence environment, forming an embedding matrix that can fully describe the sequence-level feature distribution; Further, the self-attention mechanism is introduced to model the long-range dependence of residues within the sequence, and different weights are assigned according to the importance between residues, so as to identify residue pairs that are far apart in the sequence but may have potential interactions in the spatial structure; Finally, the sequence context embedding representation containing residue-level features is output, which contains both local and global feature information and can be used as a key input for subsequent drug and target interaction prediction.

[0024] Step 3, self-supervised pre-training feature learning for drug and target data, as shown in the flowchart Figure 4 To capture the context dependence between substructures, a BERT-based Transformer encoder is used to model the sequence through multi-head self-attention mechanism, and the attention calculation formula is: On this basis, a parallel multi-task self-supervised training framework is designed to jointly optimize three types of tasks: first, substructure mask prediction, randomly mask part of the tokens, and use the context to restore the true sequence, the loss function is cross-entropy: , where is the size of the dictionary, is the number of true distributions, is the predicted probability, is the total number of masked positions in the current batch; second, molecular descriptor prediction, use global representation to predict physicochemical properties such as molecular weight and polar surface area, the loss function is mean square error: , where is the number of molecular descriptors, i.e. the regression target dimension, is the true value of the th descriptor, is the model predicted value of the three, functional group prediction, to identify whether there are functional groups such as hydroxyl, carboxyl, etc. in the molecule through multi-label classification, and the loss function is binary cross entropy: wherein is the total number of functional group categories, is the functional group index, is the true label of the th functional group, is the predicted value of the model for the probability of the existence of the th functional group; and the final training target is the weighted sum of the loss of the above three tasks: ; wherein and are task weights, both of which are in the range of [0, 1], and the specific values of the two are determined according to the performance of the model on the validation set. When and are respectively taken as 0.4-0.6, the model can obtain stable and relatively optimal prediction performance on different data sets. Through the multi-task learning framework, the model can capture both local functional fragment features and global molecular semantic information of the molecule, and obtain a robust and generalizable molecular representation, providing high-quality input for subsequent drug-target interaction prediction.

[0025] Step 4, the flow of the joint network attention module structure is shown in Figure 5 : After completing the self-supervised feature learning of the drug and the target, the obtained sequence features are input into the joint network attention module, which includes a long short-term memory network layer, a convolutional neural network layer, and a self-attention mechanism layer in turn.

[0026] First, the input sequence of the drug or target is represented as: ; wherein, denotes the sequence length, denotes the embedding dimension. The input sequence is sent to the LSTM network to model the long-range dependencies in the sequence, and the output representation is obtained: ; wherein, is the hidden state at time , and is the memory cell. This process can capture the global semantic relationship between molecular substructures or protein residues.

[0027] Next, the sequence features output by the LSTM are input into the CNN layer to extract local neighborhood patterns. The convolution operation formula is: ; wherein, denotes one-dimensional convolution operation, is the convolution kernel parameter, is the bias, is the local feature matrix. CNN layers can enhance the local interaction features of adjacent fragments and suppress noise. Subsequently, the output results enter the self-attention mechanism layer. First, the feature vector of each time step is The attention score is calculated: The score is normalized to weight by the SoftMax function: The global representation vector of the sequence is obtained by weighted summation: Wherein, represents the importance of the th substructure or residue, is the fixed dimension vector representation after fusion. Finally, the drug representation vector and the target representation vector are input into the fusion network. Vector splicing and interaction operations can be used: Wherein, represents the element-wise product, represents the drug vector, represents the target vector represents the element-wise absolute difference. The fusion vector is mapped to the probability space by the fully connected layer and the hyperbolic tangent activation function: Wherein, represents the predicted probability value of the interaction between the drug and the target, which can be used for binary classification of whether the interaction exists, or as a scoring output of the binding affinity strength.

[0028] Step 5, in the prediction and verification phase, first input the fusion feature vector into the fully connected layer, and calculate the binding strength score by the logistic normalization function, map it to the probability value in the interval [0,1], as the prediction result of the existence of the interaction between the drug and the target; In the training process, the binary cross-entropy is used as the optimization target, so that the model can effectively distinguish between positive and negative samples. Subsequently, the drug and protein embeddings obtained by self-supervised learning pre-training are input into the downstream prediction network, and supervised fine-tuning is performed on the labeled data set, and the model is evaluated in three experimental scenarios of common drug prediction, new drug prediction and new target prediction. Finally, the difference between the predicted value and the true label is calculated, and the AUC index is used for performance comparison.

[0029] The drug-target interaction prediction method based on the joint network attention is fully verified on the existing annotated data set BIOSNAP. After 50 rounds of training, the method proposed in the application achieves an AUPR score of 0.942 on the test set, which is significantly better than the existing representative method: the AUPR of the ConPlex model under the same conditions is 0.912, and the AUPR of the MolTrans model is 0.891. The method proposed in the application improves the prediction accuracy by about 3% and 5.1% respectively. The experimental results fully show that the method proposed in the application has better overall performance than the prior art in the drug-target interaction prediction task, has strong practical value and promotion potential, especially under the cold start condition, which proves the effectiveness and practical value of the method in drug screening and new target discovery.

[0030] The above is a further detailed description of the application in combination with specific preferred embodiments, and the specific implementation of the application cannot be limited to these descriptions. For ordinary skilled persons in the art to which the application belongs, without departing from the concept of the application, a number of simple deductions or substitutions can be made, which should be considered as falling within the protection scope of the application.

Claims

1. A method for predicting drug-target interaction based on joint network attention, characterized in that, The method comprises five processes of modeling drug data, modeling target data, self-supervised pre-training feature learning of drug and target data, joint network attention training and drug-target interaction prediction, and the specific steps are as follows: Step 1, the original molecular data is structured and modeled, which is converted into a graph structure by a molecular analysis tool, and a substructure segmentation algorithm is used to decompose the whole molecule, taking the atom on the ring as the center, extending to the edge, and taking the directly connected chemical bond and the group on the bond as a whole group to divide and disassemble into several substructure fragments; Step 2, the target protein data is taken as the input source, and the original sequence is first standardized, including removing illegal characters, unifying the coding format, and segmenting or truncating long sequences; then the standardized sequence is input into a large-scale pre-trained protein language model to form an embedding matrix that can completely describe the sequence level feature distribution; on this basis, each amino acid residue is mapped to a high-dimensional embedding vector, and the long-range dependence of the residues in the sequence is modeled by using the self-attention mechanism, and different weights are assigned according to the importance between the residues, and finally the sequence context embedding representation containing the residue level features is output; Step 3, the drug molecule data in step 1 and the target protein data in step 2 are pre-trained feature learning in a self-supervised manner, and the representation features of drugs and proteins are learned on a large amount of unlabeled data; wherein, for drug data, multi-task learning of substructure mask prediction, molecular descriptor prediction and functional group prediction is adopted; for protein data, residue-level context features are obtained by using mask language modeling to learn, and pre-training features of molecules and proteins are obtained; Step 4, input the pre-trained features of the drug and protein obtained in step 3 into the joint network attention module, which sequentially includes a long short-term memory network layer, a convolutional neural network layer, and a self-attention mechanism layer, and outputs a drug vector and a target vector; then, the drug vector and the target vector are fused by vector splicing and interaction operation, including calculating the element-wise product and the element-wise absolute difference, thereby obtaining a fusion vector ; wherein, represents a splicing operation, represents an element-wise product, represents a drug vector, represents a target vector, represents an element-wise absolute difference, represents an element-wise product of a drug vector and a target vector; the fusion vector is mapped to a probability space by a fully connected layer and a hyperbolic logistic transformation activation function: ; wherein, represents a predicted probability value of the interaction between the drug and the target, which can be used for a binary classification task of whether the interaction exists, or as a scoring output of the binding affinity strength; Step 5, using binary classification cross-entropy loss, through a binary classification output layer, as the final drug-target interaction prediction result.

2. The drug-target interaction association prediction method based on joint network attention according to claim 1, characterized in that Taking the atom on the ring as the center, the directly connected chemical bond and the group on the bond are divided as a whole group, and the implementation process is as follows: By converting it into a graph structure through molecular analysis tools, atoms are mapped as nodes of the graph, and chemical bonds are mapped as edges between nodes. The complete molecule is decomposed into several substructure units using a substructure segmentation algorithm. Each central atom and its directly connected atoms and chemical bonds are considered as a basic structure unit. Each unit is considered as a basic token, and the overall molecule is converted into a sequence representation composed of tokens. For the ethanol molecule ( ), first, the methyl carbon ( ) is taken as the center, and the three hydrogen atoms connected to it and the carbon-carbon bond are collectively divided into a group, i.e., the formation of the group; then, the methylene carbon ( ) is taken as the center, and the two hydrogen atoms connected to it, the hydroxyl group ( ), and the carbon-carbon bond of are collectively considered as a group, i.e., the formation of the group; and the structure units connected by different carbon atoms form different groups.

3. The drug-target interaction association prediction method based on joint network attention according to claim 2, characterized in that The implementation process of setting up a multi-task learning framework for different molecular structures is as follows: First, substructure masking prediction, randomly masking part of the token, and recovering the true sequence using the context, with the loss function being cross-entropy: ;in V For dictionary size, For the true distribution number, To predict probabilities, This represents the total number of masked positions in the current batch. The first part represents the set of token indices to be masked; the second part is molecular descriptor prediction, which uses a global representation to predict physicochemical properties, such as molecular weight and polar surface area, with the loss function being the mean squared error. ;in It refers to the number of molecular descriptors, i.e., the dimension of the regression target. For the first The actual value of each descriptor The model predicts the first The third aspect is functional group prediction, which identifies the presence of hydroxyl and carboxyl functional groups in molecules through multi-label classification, with the loss function being binary cross-entropy. ;in This represents the total number of functional group categories. For functional group indexing, For the first The true label of a functional group For the model to the first The predicted probability of the presence of each functional group; the final training objective is the weighted sum of the losses from the above three tasks: ;in and This represents the task weight.

Images