A molecular-protein reaction prediction classification method based on improved Transformer network

Abstract

The application discloses a kind of molecular-protein reaction prediction classification methods based on the improvement of Transformer network: (1) respectively to molecular and protein feature coding, sequence data is converted into machine recognizable feature vector dataset, wherein molecule uses adjacency matrix representation;(2) the dataset is divided into training set, verification set, test set;(3) training set is input to the improved Transformer network and is trained, and the feature vector of data is learned using bidirectional attention mechanism.(4) molecular feature vector and protein feature vector after training are spliced into Transformer network, input to full connection layer, and classification result is obtained.The molecular-protein reaction prediction classification method of the improved Transformer network proposed in the application can effectively improve the molecular-protein reaction prediction classification accuracy.

Description

TECHNICAL FIELD

[0001] The application relates to a molecular-protein reaction prediction classification method, in particular to a molecular-protein reaction prediction classification method considering complex interactions between atoms and amino acids, and belongs to the technical field of machine learning. BACKGROUND

[0002] Prediction of interactions between molecules and proteins is an important step for modern medicine to develop new drugs. With the development of deep learning in recent years, prediction classification methods for interactions between ligands and target proteins have been developed, and the prediction classification results are more and more efficient and accurate. However, these methods often only consider the simple molecular interactions between molecules and proteins, and ignore the complex interactions between atoms and amino acids.

[0003] At present, there is still a lack of a molecular-protein reaction prediction classification method based on machine learning. SUMMARY

[0004] In view of the above technical problems, especially how to incorporate more important feature information into the network, the application provides a molecular-protein reaction prediction classification method based on an improved Transformer network. Compared with the most advanced baseline, the model of the application realizes significant performance improvement.

[0005] To solve the above technical problems, the application adopts the following technical solutions:

[0006] A molecular-protein reaction prediction classification method based on an improved Transformer network, the steps are as follows:

[0007] Step 1: data preprocessing is performed on the imported data set, and feature representation is performed on the molecular and protein sequences in the data set respectively; the protein sequence is split into overlapping n-gram amino acid sequences, and a one-dimensional protein vector is obtained by using the bidirectional encoder of the Bert model to learn embedding vectors from the text; the data set includes molecular node features, an adjacency matrix, a one-dimensional protein vector, and a label existing in the original data sample;

[0008] Step 2: the encoded data set is divided into a training set, a validation set and a test set;

[0009] Step 3: the training set is input into the Transformer network for training, and the attention mechanism is used to learn the feature vectors of the molecular data and the protein data respectively;

[0010] Step 4: The molecular reaction vector and the protein reaction vector after training of the Transformer network are weighted and calculated, then spliced, input into a fully connected layer, and the obtained training result is back propagated to obtain a trained model after multiple iterations;

[0011] Step 5: The model is used to test the test sample to obtain a test result.

[0012] Further, the steps of step 1 are as follows:

[0013] The single data sample in the data set pool consists of a molecular sequence, a protein sequence and a label indicating whether there is a reaction; the molecular sequence in the data is represented by relying on the RDKit python package to generate a two-dimensional molecular graph, and based on the two-dimensional molecular graph, an adjacency matrix corresponding to the graph is generated n is the number of nodes;

[0014] The local breadth-first search of each node of the molecular graph with a specified radius r is performed to obtain a subgraph composed of adjacent nodes and edges of the node; based on the information of these subgraphs, the feature representation of each node is updated, including atomic type, charge, multiplicity and mass, etc., so as to obtain the initial molecular node features of the data sample

[0015] For the protein sequence, since there are only 20 types of amino acids, in order to increase the diversity, based on the n-gram language model at that time, the words in the protein sequence are defined as n-gram amino acids, and the given protein sequence is split into overlapping amino acid sequences, for example, the protein sequence MSTSSLR…TNPFLL is split into MST, STS, TSS, SSL, …, TNP, NPF, PFL, FLL. For a given protein sequence S = s1s2s3…s k , it is split into the following sequences:

[0016] S' = [s1; s2; s3], [s2; s3; s4], [s3; s4; s5], …, [s k-2 ; s k-1 ; s k ]

[0017] Where k is the length of the protein sequence;

[0018] For the split sequence, the bidirectional encoder of the Bert model is used to learn the embedding vector from the text composed of the above k-2 protein words, and [s k ; s k+1 ; s k+2 ] is represented as c k , and the corresponding protein sequence is represented as:

[0019] Bert encoder (S′)=C={c1,c2,…c k-2}

[0020] Finally, a data sample is preprocessed to obtain molecular node features F, adjacency matrix A, one-dimensional protein vector C, and label y existing in the original data sample;

[0021] Then repeat the above process for the remaining data samples to obtain the preprocessed data set:

[0022] data={[F1,A1,C1,y1],[F2,A2,C2,y2],…,[F m ,A m ,C m ,y m ]}

[0023] Where m is the number of data in the original data set.

[0024] Further, in step 1, the label y in the original data sample is 0 or 1, representing two categories of whether it can react, 0 for no reaction and 1 for reaction.

[0025] Further, the step 3 includes the following steps:

[0026] The initial molecular node feature F in the training set is obtained by embedding layer n is the number of nodes, and d is the embedding dimension; then F′ and the adjacency matrix A are input into the GNN layer for feature learning, and finally the hidden feature vector of all nodes of the molecule is obtained

[0027] The one-dimensional protein vector C in the training set is encoded by the embedding layer to obtain Input into the gated convolutional network with Conv1D and gated linear unit for feature learning to obtain the feature representation of the protein The specific operation is as follows:

[0028]

[0029] Where L h represents the output of the hth layer in the gated convolutional network, is the input of the hth layer, is the learning parameter, h is the number of hidden layers, d h is the dimension of the hth layer hidden feature vector, σ is the sigmoid function, is the matrix product;

[0030] The obtained molecular feature representation f and the feature representation L of the protein are taken as inputs of a Transformer decoder, and the Transformer decoder includes a multi-head self-attention layer;

[0031] The reaction information of the molecule and the protein is extracted through the self-attention layer respectively, and the specific operation is as follows:

[0032] f′ h-1 =LayerNorm h (f h-1 +attention(f h-1 ,L h-1 ,L h-1 ))

[0033] L′ h-1 =LayerNorm h (L h-1 +attention(L h-1 ,f h-1 ,f h-1 ))

[0034] f h =LayerNorm h (f′ h-1 +feedforward(f′ h-1 ))

[0035] L h =LayerNorm h (L′ h-1 +feedforward(L′ h-1 ))

[0036] Where LayerNorm h is the h-th normalization layer, feedforward is the Feed-Forward layer of the Transformer network, f′ h-1 is the (h-1)-th molecular feature vector with reaction feature weight, f h is the h-th molecular feature vector input, L′ h-1 is the (h-1)-th protein feature vector with reaction feature weight, L h is the h-th protein feature vector input;

[0037] After the reaction features are extracted by the decoder, the molecular reaction sequence [x1, x2,..., x n ] and the protein reaction sequence [l1, l2,..., l k ] are obtained respectively.

[0038] Further, in step 4, the self-attention layer includes 3 inputs, key K, value V, and query Q, and is calculated as follows:

[0039]

[0040] where d k is a scaling factor depending on the number of layers.

[0041] Further, step 4 includes the following steps:

[0042] The L2 norm of the molecular reaction vector and the protein reaction vector output by the Transformer decoder is calculated, as follows:

[0043]

[0044]

[0045] where x i is the i-th node vector, x′ i is the L2 norm of the node vector, l j is the j-th word vector, and l′ j is the L2 norm of the word vector.

[0046] The weight of the vector can be calculated by the softmax function, as follows:

[0047]

[0048]

[0049] The final reaction vector is obtained by weighted calculation of the molecular reaction vector and the protein reaction vector:

[0050]

[0051]

[0052] where y c is the molecular reaction vector, and y p is the protein reaction vector.

[0053] The obtained reaction vector is spliced and then input to a fully connected layer, and finally the probability of the reaction between the molecule and the protein is returned The binary cross-entropy is used as the loss function to train the model.

[0054] The optimized network model is obtained by training, and in the iteration process, the network model with the highest validation accuracy is saved.

[0055] Furthermore, in step 4, the concatenation of the reaction vectors is performed using the following formula:

[0056] y cat =cat(y c ,y p ).

[0057] Furthermore, in step 4, the formula for using binary cross-entropy as the loss function to train the model is as follows:

[0058]

[0059] Where y is the real label vector encoded by one-hot encoding, such as [0,1]. This is the network's output vector.

[0060] Furthermore, in step 5, the test set is input into the saved network model, and the obtained classification results are subjected to softmax normalization calculation to map the results to the [0,1] interval.

[0061]

[0062] Where class represents the label category, 0 / 1 represents no response / response, and P(class) represents the probability that the prediction result is of that class.

[0063] Compare P(0) and P(1) to obtain the predicted label, and then compare it with the true label of the test set.

[0064] The beneficial effects of this invention are as follows:

[0065] The method provided by this invention can extract more accurate feature information for training and learning, which can improve the accuracy of target task classification, making the classification more accurate and robust. Attached Figure Description

[0066] Figure 1 This is a flowchart illustrating a specific implementation of the present invention; Detailed Implementation

[0067] The present invention will be further described below with reference to the embodiments shown in the accompanying drawings. The scope of the present invention is not limited thereto.

[0068] Example

[0069] like Figure 1 As shown, this invention provides a molecular-protein reaction prediction and classification method based on an improved Transformer network, with the following steps:

[0070] Step 1: Perform data preprocessing on the imported dataset, and perform feature representation on the molecular protein sequences in the dataset;

[0071] The implementation process is as follows:

[0072] For a single data sample, it is composed of molecular sequences, protein sequences and labels indicating whether there is a reaction or not.

[0073] The molecular sequence in the data is represented by relying on the RDKit python package to generate a two-dimensional molecular graph. Based on the two-dimensional molecular graph, an adjacency matrix corresponding to the graph can be generated n is the number of nodes. Perform local breadth-first search on each node of the molecular graph with a specified radius r to obtain a subgraph composed of adjacent nodes and edges of the node. Based on the information of these subgraphs, the feature representation of each node can be updated, such as atom type, charge, multiplicity, mass, etc., so as to obtain the initial molecular node features of the data sample

[0074] For protein sequences, since there are only 20 types of amino acids, in order to increase diversity, based on the n-gram language model at that time, the words in the protein sequence are defined as n-gram amino acids. For a given protein sequence, it is split into overlapping n-gram amino acid sequences, for example, the protein sequence MSTSSLR…TNPFLL is split into MST, STS, TSS, SSL, …, TNP, NPF, PFL, FLL. Therefore, for a given protein sequence S = s1s2s3…s k , it is split into the following sequence:

[0075] S' = [s1; s2; s3], [s2; s3; s4], [s3; s4; s5], …, [s k-2 ; s k-1 ; s k ]

[0076] Where n is the length of the protein sequence. For the split sequence, the bidirectional encoder of the Bert model is used to learn embedding vectors from the text composed of these n-2 protein words, such as [s k ; s k+1 ; s k+2 ] is represented as c k , and the corresponding protein sequence is represented as:

[0077] Bert encoder (S') = C = {c1, c2, … c k-2}

[0078] Finally, a data sample is preprocessed to obtain molecular node features F, adjacency matrix A, one-dimensional protein vector C, and label y existing in the original data sample. The label y is 0 or 1, representing two categories of whether to react, 0 being unable to react and 1 being able to react.

[0079] Then, the above process is repeated for the remaining data samples to obtain the preprocessed data set:

[0080] data = {[F1, A1, C1, y1}, [F2, A2, C2, y2}, …, [Fm, Am, Cm, ym]} m m m m

[0081] Where m is the number of the original data set.

[0082] Step 2: The encoded data set is divided into a training set, a validation set and a test set;

[0083] The specific implementation process is as follows:

[0084] The data preprocessed through step 1 is randomly divided into a training set, a validation set and a test set in a ratio of 8:1:1.

[0085]

[0086]

[0087]

[0088] Where m train , m valid , m test respectively represent the number of data in the training set, the validation set and the test set.

[0089] Step 3: The training set is input into the Transformer network for training, and the attention mechanism is used to learn the feature vectors of the molecular data and the protein data respectively;

[0090] The specific implementation process is as follows:

[0091] The initial molecular node features F in the training set are obtained through an embedding layer to obtain n is the number of nodes, and d is the embedding dimension. Then, F′ and the adjacency matrix A are input into the GNN layer for feature learning, and finally the hidden feature vector of all nodes of the molecule is obtained

[0092] The one-dimensional protein vector C in the training set is encoded through an embedding layer to obtain ​​​​The input is fed into a gated convolutional network with Conv1D and Gated Linear Units for feature learning, obtaining the feature representation of the protein The specific operation is as follows:

[0093]

[0094] Wherein L h represents the output of the hth layer in the gated convolutional network, is the input of the hth layer, is a learning parameter, h is the number of hidden layers, d h is the dimension of the hth layer hidden feature vector, and σ is the sigmoid function, is the matrix product.

[0095] The obtained molecular feature representation f and the feature representation L of the protein are taken as the input of the Transformer decoder, and the key calculation of the decoder is the multi-head self-attention layer, which is composed of several scaled dot-attention layers and is used to extract the interaction information between the encoder and the decoder. The self-attention layer contains 3 inputs, key K, value V and query Q, and is calculated in the following manner:

[0096]

[0097] Wherein d k is a scaling factor depending on the number of layers. The reaction information of the molecule and the protein is extracted through the self-attention layer, and the specific operation is as follows:

[0098] f′ h-1 =LayerNorm h (f h-1 +attention(f h-1 , L h-1 , L h-1 ))

[0099] L′ h-1 =LayerNorm h (L h-1 +attention(L h-1 , f h-1 , f h-1 ))

[0100] f h =LayerNorm h (f′ h-1 +feedforward(f′ h-1 ))

[0101] L h =LayerNorm h (L′h-1 +feedforward(L′ h-1 ))

[0102] LayerNorm h f′ is the h-th normalization layer, feedforward is the feed-forward layer of the Transformer network, and f′ is the h-th normalization layer. h-1 f is the molecular eigenvector of the (h-1)th layer with reaction feature weights. h As the molecular feature vector input for the h-th layer, L′ h-1 Let L be the protein feature vector of the (h-1)th layer with response feature weights. h The input is the protein feature vector of layer h. After the decoder extracts the reaction features, the molecular reaction sequences [x1, x2, ..., x] are obtained. n ] and protein reaction sequences [l1, l2, ..., l k ].

[0103] Step 4: Perform weighted calculations on the molecular response vector and protein response vector after training the Transformer network, then concatenate them and input them into the fully connected layer. Perform backpropagation on the obtained training results and iterate multiple times to obtain the trained model.

[0104] The specific implementation process is as follows:

[0105] The L2 norm of the molecular reaction vector and protein reaction vector output by the Transformer decoder are calculated separately, as shown in the following formula:

[0106]

[0107]

[0108] Where x i Let x′ be the vector of the i-th node. i Let l be the Euclidean norm of the node vector. j Let l′ be the vector of the j-th word. j Let be the Euclidean norm of the word vector.

[0109] The weights of the vectors can be calculated using the softmax function, as shown below:

[0110]

[0111]

[0112] The final reaction vector is obtained by weighting the molecular reaction vector and the protein reaction vector separately:

[0113]

[0114]

[0115] where y c is the molecular reaction vector, y p is the protein reaction vector;

[0116] The obtained reaction vectors are spliced:

[0117] y cat = cat(y c , y p )

[0118] and input to the fully connected layer, and finally return the results of the molecular and protein reaction The model is trained using binary cross-entropy as the loss function:

[0119]

[0120] where y is the true label vector via one-hot encoding, such as [0, 1], is the output vector of the network.

[0121] The optimized network model is obtained by training, and in the iteration process, the network model with the highest validation accuracy is saved.

[0122] Step 5: Input the test set into the saved network model, perform softmax normalization calculation on the obtained classification results, and map the results to the [0, 1] interval.

[0123]

[0124] where class represents the label category, 0 / 1 represents no / has reaction, and P(class) represents the probability that the prediction result is of this class,

[0125] Compare P(0) and P(1) to obtain the predicted label, and then compare it with the true label of the test set.

[0126] This embodiment selects two data sets of Human and C.elegans for effect verification. The Human data set contains 3369 highly reliable reactive data and 3359 non-reactive data, involving 1052 compound molecular sequences and 852 protein sequences. The C.elegans data set contains 4000 highly reliable reactive data and 3786 non-reactive data, involving 1434 compound molecular sequences and 2504 protein sequences. This embodiment uses AUC (area under the curve), precision and recall as evaluation indexes. The area under the curve represents the area under the ROC curve, and the larger the area, the better the classification performance. The precision is the proportion of correct predictions among the predicted positive samples. The recall is the probability of predicting positive samples among the actual positive samples. The algorithms compared in this example include transformerCPI and GNN-CPI. The transformerCPI is a model using multi-layer Transformer, and the GNN-CPI is a model based on recurrent neural network to extract protein sequence features. As can be seen from the results in Table 1, the present example has better classification performance.

[0127] Table 1 Comparison of calculation results of each algorithm

[0128]

[0129] The above describes only the preferred specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications, equivalent replacements and improvements made by any person skilled in the art within the technical scope disclosed by the present application shall be included in the scope of protection of the present application.

Claims

1. A method for predicting classification of molecular-protein reaction based on improved Transformer network, characterized in that, The steps are as follows: Step 1: data preprocessing is performed on the imported data set, and the molecular and protein sequences in the data set are respectively represented by features; the protein sequence is split into overlapping n-gram amino acid sequences, and a one-dimensional protein vector is obtained by using the bidirectional encoder of the Bert model to learn embedding vectors from the text; the preprocessed data set includes molecular node features, adjacency matrix, one-dimensional protein vector, and labels existing in the original data sample; In step 1, the molecular sequences in the data are represented by feature, two-dimensional molecular graphs are generated, and based on the two-dimensional molecular graphs, the adjacency matrix corresponding to the graphs is generated ; local breadth-first search is performed on each node of the molecular graph with a specified radius r to obtain a subgraph composed of adjacent nodes and edges of the node; based on the information of the subgraphs, the feature representation of each node is updated, including atomic type, charge, multiplicity and mass, so as to obtain the initial molecular node features of the data sample ; For protein sequences, based on the n-gram language model at that time, the words in the protein sequence are defined as n-gram amino acids, and for a given protein sequence S = s1s2s3…s k , it is split into the following sequence: Where k is the length of the protein sequence; For the split sequence, the bidirectional encoder of the Bert model learns embedding vectors from the text composed of the above k-2 protein words, and the embedding vectors are represented as are represented as The corresponding protein sequence is represented as: Finally, a data sample is preprocessed to obtain molecular node features , an adjacency matrix , a one-dimensional protein vector C, and a label y existing in the original data sample; Then repeat the above process for the remaining data samples to obtain the preprocessed data set: Where m is the number of data in the original data set; Step 2: the preprocessed data set is divided into training set, validation set and test set; Step 3: input the training set into the Transformer network for training, and use the attention mechanism to learn the feature vectors of molecular data and protein data respectively; Step 4: the molecular reaction vector and the protein reaction vector trained by the Transformer network are weighted and calculated, then spliced, input into the full connection layer, and the training result is back propagated, and the trained model is obtained after multiple iterations; Step 5: test the test sample by using the model to obtain the test result.

2. The method of claim 1, wherein the molecular-protein reaction prediction classification is based on a plurality of molecular-protein reaction prediction classification models. In step 2, the data is randomly divided into training set, validation set and test set in the ratio of 8:1:

1.

3. The method of claim 1, wherein the molecular-protein reaction prediction classification is based on a plurality of molecular-protein reaction prediction classification models. Step 3 includes the following steps: The initial molecular node feature F in the training set is obtained through an embedding layer to obtain an initial feature embedding representation , n is the number of nodes, is the embedding dimension; then, the initial feature embedding representation and the adjacency matrix A are input into a GNN layer for feature learning, and finally, the hidden feature vector of all nodes of the molecule is obtained ; The one-dimensional protein vector C in the training set is encoded by the embedding layer to obtain , and is input into a gated convolutional network with Conv1D and a gated linear unit for feature learning to obtain the feature representation of the protein , and the specific operation is as follows: wherein denotes the output of the h-th layer in the gated convolutional network, is the input of the h-th layer, , is a learning parameter, h is the number of hidden layers, is the dimension of the h-th layer hidden feature vector, is a sigmoid function, is a matrix multiplication; The obtained molecular feature representation and the feature representation of the protein As an input of the Transformer decoder, the Transformer decoder comprises a multi-head self-attention layer; The reaction information of molecules and proteins is extracted through self-attention layers, and the specific operation is as follows: wherein is the h-th normalized layer, is the Feed-Forward layer of the Transformer network, is the h-1-th molecular feature vector with reaction feature weights, is the h-th molecular feature vector input, is the h-1-th protein feature vector with reaction feature weights, is the h-th protein feature vector input; After the decoder extracts the reaction features, the molecular reaction sequence and the protein reaction sequence are obtained respectively.

4. The method of claim 3, wherein the molecular-protein reaction prediction classification is determined by a machine learning algorithm. In step 4, the self-attention layer includes 3 inputs, key K, value V, and query Q, and is calculated as follows: wherein is a layer number dependent scaling factor.

5. The method of claim 1, wherein the molecular-protein reaction prediction classification is selected from the group consisting of: Step 4 includes the following steps: ​ The L2 norm of the molecular reaction vector and the protein reaction vector output by the Transformer decoder is calculated, and the formula is as follows: wherein is the i-th node vector, is the L2 norm of the node vector, is the j-th word vector, is the L2 norm of the word vector; The weight of the vector can be calculated by the softmax function, as shown in the following formula: The final reaction vector is obtained by weighting the molecular reaction vector and the protein reaction vector respectively: wherein, is a molecular reaction vector, is a protein reaction vector; The resulting reaction vector is concatenated and fed into a fully connected layer, which returns the probability of the molecule reacting with the protein The model is trained using binary cross-entropy as the loss function; The optimized network model is obtained by training, and in the iteration process, the network model with the highest validation accuracy is saved.

6. The molecular-protein reaction prediction and classification method according to claim 5, characterized in that: In step 4, the splicing of the reaction vector is performed by the following formula:

7. The molecular-protein reaction prediction and classification method according to claim 5, characterized in that: In step 4, the binary cross entropy is used as the loss function to train the model, and the formula is as follows: where y is the true label vector via one-hot encoding, is the output vector of the network.

8. The method of claim 1, wherein: In step 5, the test set is input into the saved network model, and the classification results obtained are normalized The results are mapped to the interval [0, 1] by normalization calculation. where class represents the label class, 0 / 1 represents no / has response, represents the probability that the prediction result is this class, Comparing and the predicted labels to the true labels of the test set.

Images