A deep learning-based method for predicting compound-protein interactions

Abstract

The present invention discloses a compound-protein interaction prediction method based on deep learning, which includes the following steps: step A, using a CNN module to learn local features of compound images and then using a Transformer encoder to learn semantic relationships of the learned features; step B, using a k-gram method to divide protein sequences and then using several Transformer encoders to learn semantic relationships; step C, further learning the learned features of compounds and proteins, and step D, obtaining prediction results through a fully connected layer. The present invention proposes a method with high prediction accuracy that can effectively capture the features of compound molecular images.

Description

Technical Field

[0001] The present invention relates to the technical field of drug design and image processing, and in particular to a method for predicting compound-protein interaction by combining CNN and Transformer to learn compound image features. Background Art

[0002] Drug discovery is a field of bioinformatics that aims to discover new molecular structures with desired pharmacological properties, involving a wide range of scientific disciplines including biology, chemistry and pharmacology. New drug molecules interact with target proteins to provide therapeutic benefits to patients.

[0003] In recent years, in the context of computer-aided drug design, people have begun to have great interest in developing automated machine learning techniques, thereby discovering a large number of credible, diverse and novel candidate molecules in the vast and unstructured molecular space. Accurately predicting the interaction between compounds and proteins can not only reduce the occurrence of some diseases, but also reduce the cost of drug development, which is crucial to both patients and society. Summary of the invention

[0004] The purpose of this invention is to propose a method for predicting compound-protein interactions by combining CNN and Transformer to learn compound image features based on the information of compound molecular images. This method can significantly improve the accuracy of compound-protein interaction prediction.

[0005] The specific technical solution for achieving the purpose of the present invention is:

[0006] A compound-protein interaction prediction method based on deep learning, the method comprising the following steps:

[0007] 1) Obtaining a characteristic graph of a compound molecule, specifically including:

[0008] 1.1) Generate compound molecular images using RDkit software based on the SMILES sequence of the compound molecule;

[0009] 1.2) Input the generated compound molecule image into the CNN module to learn features, where the image size is H×W×3, H and W represent the length and width of the image respectively, and 3 represents the color channel of the image; use the CNN module to learn the local features of the compound molecule image, and the CNN module is composed of: convolution layer, BatchNormalization layer, LeakyReLU function activation layer, convolution layer, BatchNormalization layer, LeakyReLU function activation layer and maximum pooling layer; the size of the feature map after CNN learning is m and n represent the length and width of the molecular feature map of the compound after CNN learning, respectively, and c represents the channel of the feature map;

[0010] 1.3) According to the obtained feature map X CNN , flattened according to its channel dimension as the input token of Transformer, the new feature map size is Where m·n is the product of the length and width of the compound molecular feature map, and c is the channel of the feature map;

[0011] 1.4) The new feature map X token As the input of the Transformer encoder, the semantic relationship in the feature map is then learned through several Transformer encoders. Each Transformer encoder is composed as follows:

[0012] 1.4.1) The LayerNormalization layer processes data of different lengths;

[0013] 1.4.2) The multi-head attention layer converts the input vector into three different vectors, all of which have dimension d: query vector q, key vector k, and value vector v, and compresses the vectors into corresponding matrices Q, K, and V. The calculation process is: The score of the query matrix Q and the key matrix K is calculated as: S = Q·K T , the normalized gradient stability score is: Use the softmax function to convert the score into probability: P = softmax(S n ), and obtain the weighted value matrix: Attention = V P; the formula for the whole process is: Where softmax represents the softmax activation function, Q and V represent the corresponding matrices compressed by the query vector q and the value vector v, and K T represents the transpose of the corresponding matrix of the key vector compression, d k represents the current input dimension; then, in order to solve the problem that self-attention is insensitive to position information, the position encoding of the same dimension is added to the original input embedding, and its formula is: pos represents the position of the word in the sentence, i represents the current dimension of the position encoding, and d is the dimension of the input vector; the total attention is composed of multiple head attentions: MultiHead(Q′, K′, V′) = concat(head 1 , ..., head h )W O , concat represents the concatenation operation, W O is the learnable feature transformation matrix, head iRepresents the Attention value of each attention head, and h represents a total of h attention heads;

[0014] 1.4.3) Then pass through the multi-layer perceptron layer to obtain the final semantic information feature map; the process of the multi-layer perceptron layer is: fully connected layer, GELU activation layer, DropPath layer, fully connected layer, DropPath layer;

[0015] 1.5) Through the learning representation of several Transformer encoders, the final feature map X of the compound molecule is obtained C ;

[0016] 2) Obtaining a feature map of the protein sequence, specifically including:

[0017] 2.1) The protein sequence is in FASTA format, where the sequence is composed of amino acids and each amino acid is represented by a single letter; the protein sequence is divided into words using the k-gram method, and the length of each word is k;

[0018] 2.2) Create a dictionary for the divided words, sort them in ascending order according to the first word that appears, use the dictionary's serial number to replace the original protein sequence's representative word, and then embed the serial number represented by each word;

[0019] 2.3) Add the position information of the original word in the protein sequence, and pass it through several Transformer encoder modules to learn the semantic information between each word in the protein sequence;

[0020] 2.4) After learning through several Transformer encoder modules, the final feature map X of the protein sequence is obtained P ;

[0021] 3) Characteristic graph X of compound molecules C and the characteristic graph X of the protein sequence P Conduct further learning and predict the final results, including:

[0022] 3.1) The feature map X C and the feature map X P Shallow learning of features through multi-layer perceptrons;

[0023] 3.2) The two feature maps are superimposed and a CNN module is used for deep feature learning. The CNN module design is as follows: convolution layer, BatchNormalization layer, LeakyReLU function activation layer, maximum pooling layer, convolution layer, BatchNormalization layer, LeakyReLU function activation layer, maximum pooling layer.

[0024] 3.3) Flatten the feature map and pass it through a fully connected layer to obtain the final prediction result, which is expressed as 0 or 1, where 0 represents no interaction between the compound and the protein, and 1 represents interaction.

[0025] The technical concept of the present invention is as follows: first, the compound molecule picture contains more molecular information. Its atomic information and spatial structure are well presented in the picture. By using CNN to learn local features of the compound molecule picture, the molecular information contained in each pixel can be effectively captured. Then, through the Transformer's learning ability for global features, several Transformer encoders are constructed to learn the semantic relationship of the feature graph. Secondly, for the insufficient representation defect of only 20 types of protein amino acids, through k-gram word segmentation, the combined amino acid types can be expanded to more types, which is conducive to the representation learning of proteins. Finally, according to the feature graphs of compounds and proteins, a multi-layer perceptron and CNN module are constructed to learn the feature graphs again, which effectively improves the prediction ability of the method.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows: first, feature learning is performed on compound molecular images. CNN and Transformer are used to comprehensively extract local and global features of the images, effectively capturing the information in the compound molecular images. Then the k-gram method effectively solves the problem of insufficient types of amino acid letter representations, and the creation of a Transformer encoder can effectively extract the semantic relationship between amino acid words in protein sequences. Finally, the features are effectively improved through multi-layer perceptron and CNN multi-level learning. Compared with other existing methods, this technology can significantly improve the training speed and prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the present invention;

[0028] Figure 2 yes Figure 1 Flowchart of CNN module in step 1;

[0029] Figure 3 This is a schematic diagram of word segmentation through k-gram. DETAILED DESCRIPTION

[0030] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0031] Example 1

[0032] See also Figures 1 to 3, a compound-protein interaction prediction method based on deep learning, the method comprising the following steps:

[0033] 1) Obtaining a characteristic graph of a compound molecule, specifically including:

[0034] 1.1) Generate compound molecular images using RDkit software based on the SMILES sequence of the compound molecule;

[0035] 1.2) Input the generated compound molecule image into the CNN module to learn features, where the image size is 256×256×3, 256 represents the length and width of the image, and 3 represents the color channel of the image; use the CNN module to learn the local features of the compound image. The CNN module consists of: convolution layer, BatchNormalization layer, LeakyReLU function activation layer, convolution layer, BatchNormalization layer, LeakyReLU function activation layer and maximum pooling layer; the size of the feature map after CNN is 16 represents the length and width of the molecular feature map of the compound after CNN learning, and 256 represents the channel of the feature map;

[0036] 1.3) According to the obtained feature map X CNN , flattened according to its channel dimension as the input token of Transformer, the new feature map size is Where 256 is the product of the length and width of the compound molecular feature map, and 256 is the channel of the feature map;

[0037] 1.4) The new feature map X token As the input of the Transformer encoder, the semantic relationship in the feature map is then learned through several Transformer encoder modules. The components of each Transformer encoder are as follows:

[0038] 1.4.1) The LayerNormalization layer processes data of different lengths;

[0039] 1.4.2) The multi-head attention layer converts the input vector into three different vectors, all of which have a dimension of 256: query vector q, key vector k, and value vector v. The vectors are compressed into corresponding matrices Q, K, and V. The calculation process is as follows: The score of the query matrix Q and the key matrix K is calculated as: S = Q·K T , the normalized gradient stability score is: Use the softmax function to convert the score into probability: P = softmax(S n ), and obtain the weighted value matrix: Attention = V P; the formula for the whole process is: Where softmax represents the softmax activation function, Q and V represent the corresponding matrices compressed by the query vector q and the value vector v, and K T represents the transpose of the corresponding matrix of the key vector compression, d k represents the current input dimension; then, in order to solve the problem that self-attention is insensitive to position information, the position encoding of the same dimension is added to the original input embedding, and its formula is: pos represents the position of the word in the sentence, i represents the current dimension of the position encoding, and d is the dimension of the input vector; the total attention is composed of multiple head attentions: MultiHead(Q′, K′, V′) = concat(head 1 , ..., head h )W O , concat represents the concatenation operation, W O is the learnable feature transformation matrix, head i Represents the Attention value of each attention head, and h represents a total of h attention heads;

[0040] 1.4.3) Then pass through the multi-layer perceptron layer to obtain the final semantic information feature map; the process of the multi-layer perceptron layer is: fully connected layer, GELU activation layer, DropPath layer, fully connected layer, DropPath layer;

[0041] 1.5) Through the learning representation of 4 Transformer encoders, the final feature map of the compound molecule is obtained The size of the feature map is 256 by 256;

[0042] 2) Obtaining a feature map of the protein sequence, specifically including:

[0043] 2.1) The protein sequence is in FASTA format, where the sequence is composed of amino acids and each amino acid is represented by a single letter; the protein sequence is divided into words using the k-gram method, and the length of each word is 3;

[0044] 2.2) Create a dictionary for the divided words, sort them in ascending order according to the first word that appears, use the dictionary's serial number to replace the original protein sequence's representative word, and then embed the serial number represented by each word;

[0045] 2.3) Add the position information of the original word in the protein sequence, and learn the semantic information between each word in the protein sequence through several Transformer encoder modules;

[0046] 2.4) The final feature map of the protein sequence is obtained after learning through 4 Transformer encoder modules The size of the feature map is 256 by 256;

[0047] 3) Characteristic graph X of compound molecules C and the characteristic graph X of the protein sequence P Conduct further learning and predict the final results, including:

[0048] 3.1) The feature map X C and the feature map X P Shallow learning of features through multi-layer perceptrons;

[0049] 3.2) Overlay the two feature maps to get the final feature representation map The feature map size is 256 times 256, and the number of channels is 2. A CNN module is used for deep feature learning. The CNN module design is: convolution layer, BatchNormalization layer, LeakyReLU function activation layer, maximum pooling layer, convolution layer, BatchNormalization layer, LeakyReLU function activation layer, maximum pooling layer.

[0050] 3.3) Flatten the feature map and pass it through a fully connected layer to obtain the final prediction result, which is expressed as 0 or 1, where 0 represents no interaction between the compound and the protein, and 1 represents interaction.

[0051] Example 2

[0052] This example conducts experiments on three data sets as examples. Experiments are conducted on three data sets of compound-protein interactions (Human, Celegans, Davis). Data set division: The data set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The method predicts the results by the following steps:

[0053] 1) Obtaining a characteristic graph of a compound molecule, specifically including:

[0054] 1.1) Generate compound molecular images using RDkit software based on the SMILES sequence of the compound molecule;

[0055] 1.2) Input the generated compound molecule image into the CNN module to learn features, where the image size is H×W×3, H and W represent the length and width of the image respectively, and 3 represents the color channel of the image; use the CNN module to learn the local features of the compound image, and the CNN module consists of: convolution layer, BatchNormalization layer, LeakyReLU function activation layer, convolution layer, BatchNormalization layer, LeakyReLU function activation layer and maximum pooling layer; the size of the feature map after CNN is m and n represent the length and width of the molecular feature map of the compound after CNN learning, respectively, and c represents the channel of the feature map;

[0056] 1.3) According to the obtained feature map X CNN , flattened according to its channel dimension as the input token of Transformer, the new feature map size is Where m·n is the product of the length and width of the compound molecular feature map, and c is the channel of the feature map;

[0057] 1.4) The new feature map X token As the input of the Transformer encoder, the semantic relationship in the feature map is then learned through several Transformer encoder modules. The components of each Transformer encoder are as follows:

[0058] 1.4.1) The LayerNormalization layer processes data of different lengths;

[0059] 1.4.2) The multi-head attention layer converts the input vector into three different vectors, all of which have dimension d: query vector q, key vector k, and value vector v, and compresses the vectors into corresponding matrices Q, K, and V. The calculation process is: The score of the query matrix Q and the key matrix K is calculated as: S = Q·K T , the normalized gradient stability score is: Use the softmax function to convert the score into probability: P = softmax(S n ), and obtain the weighted value matrix: Attention = V P; the formula for the whole process is: Where softmax represents the softmax activation function, Q and V represent the corresponding matrices compressed by the query vector q and the value vector v, and K T represents the transpose of the corresponding matrix of the key vector compression, d k represents the current input dimension; then, in order to solve the problem that self-attention is insensitive to position information, the position encoding of the same dimension is added to the original input embedding, and its formula is: pos represents the position of the word in the sentence, i represents the current dimension of the position encoding, and d is the dimension of the input vector; the total attention is composed of multiple head attentions: MultiHead(Q′, K′, V′) = concat(head 1 , ..., head h )W O , concat represents the concatenation operation, W O is the learnable feature transformation matrix, head i Represents the Attention value of each attention head, and h represents a total of h attention heads;

[0060] 1.4.3) Then pass through the multi-layer perceptron layer to obtain the final semantic information feature map; the process of the multi-layer perceptron layer is: fully connected layer, GELU activation layer, DropPath layer, fully connected layer, DropPath layer;

[0061] 1.5) Through the learning representation of several Transformer encoders, the final feature map X of the compound molecule is obtained C ;

[0062] 2) Obtaining a feature map of the protein sequence, specifically including:

[0063] 2.1) The protein sequence is in FASTA format, where the sequence is composed of amino acids and each amino acid is represented by a single letter; the protein sequence is divided into words using the k-gram method, and the length of each word is k;

[0064] 2.2) Create a dictionary for the divided words, sort them in ascending order according to the first word that appears, use the dictionary's serial number to replace the original protein sequence's representative word, and then embed the serial number represented by each word;

[0065] 2.3) Add the position information of the original word in the protein sequence, and pass it through several Transformer encoder modules to learn the semantic information between each word in the protein sequence;

[0066] 2.4) After learning through several Transformer encoder modules, the final feature map X of the protein sequence is obtained P ;

[0067] 3) Characteristic graph X of compound molecules C and the characteristic graph X of the protein sequence P Conduct further learning and predict the final results, including:

[0068] 3.1) The feature map X C and the feature map XP Shallow learning of features through multi-layer perceptrons;

[0069] 3.2) The two feature maps are superimposed and a CNN module is used for deep feature learning. The CNN module design is as follows: convolution layer, BatchNormalization layer, LeakyReLU function activation layer, maximum pooling layer, convolution layer, BatchNormalization layer, LeakyReLU function activation layer, maximum pooling layer.

[0070] 3.3) Flatten the feature map and pass it through a fully connected layer to obtain the final prediction result, which is expressed as 0 or 1, where 0 represents no interaction between the compound and the protein, and 1 represents interaction.

[0071] Table 1

[0072]

[0073] Table 2

[0074]

[0075]

[0076] Table 3

[0077]

[0078] According to the above steps and methods, a deep learning model CAT-CPI created by the present invention is experimentally compared with other machine learning methods and deep learning methods on the above data sets. Tables 1 and 2 are the results of comparing CAT-CPI with other machine learning methods on the Human and Celegans data sets respectively. Table 3 is the experimental results of comparing CAT-CPI with other deep learning models on the Davis data set. It can be seen that the models implemented by the present invention have achieved the best results.

[0079] The above description is the prediction effect obtained by the present invention using the Human, Celegans and Davis data sets as examples, and does not limit the scope of implementation of the present invention. Various modifications and improvements made thereto without departing from the scope involved in the basic content of the present invention should not be excluded from the scope of protection of the present invention.

Claims

1. A compound-protein interaction prediction method based on deep learning, characterized in that: The method comprises the following specific steps: 1) Obtaining a characteristic graph of a compound molecule, specifically including: 1.1) Generate compound molecular images using RDkit software based on the SMILES sequence of the compound molecule; 1.2) Input the generated compound molecule image into the CNN module to learn features, where the image size is H×W×3, H and W represent the length and width of the image respectively, and 3 represents the color channel of the image; use the CNN module to learn the local features of the compound molecule image, and the CNN module is composed of: convolution layer, BatchNormalization layer, LeakyReLU function activation layer, convolution layer, BatchNormalization layer, LeakyReLU function activation layer and maximum pooling layer; the size of the feature map after CNN learning is m and n represent the length and width of the molecular feature map of the compound after CNN learning, respectively, and c represents the channel of the feature map; 1.3) According to the obtained feature map X CNN , flattened according to its channel dimension as the input token of Transformer, the new feature map size is Where m·n is the product of the length and width of the compound molecular feature map, and c is the channel of the feature map; 1.4) The new feature map X token As the input of the Transformer encoder, the semantic relationship in the feature map is then learned through several Transformer encoders. Each Transformer encoder is composed as follows: 1.4.1) The LayerNormalization layer processes data of different lengths; 1.4.2) The multi-head attention layer converts the input vector into three different vectors, all of which have dimension d: query vector q, key vector k, and value vector v, and compresses the vectors into corresponding matrices Q, K, and V. The calculation process is: The score of the query matrix Q and the key matrix K is calculated as: S = Q·K T , the normalized gradient stability score is: Use the softmax function to convert the score into probability: P = softmax(S n ), and obtain the weighted value matrix: Attention = V P; the formula for the whole process is: Where softmax represents the softmax activation function, Q and V represent the corresponding matrices compressed by the query vector q and the value vector v, and K T represents the transpose of the corresponding matrix of the key vector compression, d k represents the current input dimension; then, in order to solve the problem that self-attention is insensitive to position information, the position encoding of the same dimension is added to the original input embedding, and its formula is: pos represents the position of the word in the sentence, i represents the current dimension of the position encoding, and d is the dimension of the input vector; the total attention is composed of multiple head attentions: MultiHead(Q′, K′, V′) = concat(head1, ..., head h )W O , concat represents the concatenation operation, W O is the learnable feature transformation matrix, head i Represents the Attention value of each attention head, and h represents a total of h attention heads; 1.4.3) Then pass through the multi-layer perceptron layer to obtain the final semantic information feature map; the process of the multi-layer perceptron layer is: fully connected layer, GELU activation layer, DropPath layer, fully connected layer, DropPath layer; 1.5) Through the learning representation of several Transformer encoders, the final feature map X of the compound molecule is obtained C ; 2) Obtaining a feature map of the protein sequence, specifically including: 2.1) The protein sequence is in FASTA format, where the sequence is composed of amino acids and each amino acid is represented by a single letter; the protein sequence is divided into words using the k-gram method, and the length of each word is k; 2.2) Create a dictionary for the divided words, sort them in ascending order according to the first word that appears, use the dictionary's serial number to replace the original protein sequence's representative word, and then embed the serial number represented by each word; 2.3) Add the position information of the original word in the protein sequence, and pass it through several Transformer encoder modules to learn the semantic information between each word in the protein sequence; 2.4) After learning through several Transformer encoder modules, the final feature map X of the protein sequence is obtained P ; 3) Characteristic graph X of compound molecules C and the characteristic graph X of the protein sequence P Conduct further learning and predict the final results, including: 3.1) The feature map X C and the feature map X P Shallow learning of features through multi-layer perceptrons; 3.2) The two feature maps are superimposed and a CNN module is used for deep feature learning. The CNN module design is as follows: convolution layer, BatchNormalization layer, LeakyReLU function activation layer, maximum pooling layer, convolution layer, BatchNormalization layer, LeakyReLU function activation layer, maximum pooling layer. 3.3) Flatten the feature map and pass it through a fully connected layer to obtain the final prediction result, which is expressed as 0 or 1, where 0 represents no interaction between the compound and the protein, and 1 represents interaction.

Images