A multimodal fusion deep learning model and a multifunctional bioactive peptide prediction method

Abstract

The application discloses a kind of multimodal fusion deep learning model and multifunctional bioactive peptide prediction method, it is characterized in that, including multimodal data input module, peptide sequence coding module, peptide structure coding module and classification module;The peptide sequence and peptide structure of bioactive peptide are input by the multimodal data input module;The peptide sequence coding module uses the feature of multiple scales of peptide sequence to be extracted by fusion of multiscale dilated CNN and BiLSTM model;The characteristic of peptide structure data is extracted by the peptide structure coding module using multilevel CNN model;The classification module concatenates the output of peptide sequence coding module and the output of peptide structure coding module as the input of output layer using the output of peptide sequence coding module and the output of peptide structure coding module.The effective fusion of two kinds of multimodal data of peptide sequence and structure characteristics can effectively extract the data characteristics of different perspectives, so as to better carry out multifunctional bioactive peptide function prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of bioactive peptide prediction, and in particular to a multi-modal fusion deep learning model and a multi-functional bioactive peptide prediction method. BACKGROUND

[0002] Bioactive peptides are small protein fragments, usually containing 2-20 amino acid residues, which play a variety of roles in metabolism and biological processes. In the past few decades, many bioactive peptides with multiple functions have been identified. Accurate identification of the activity of bioactive peptides is of great significance in at least two aspects: it helps to promote the understanding of the mechanism of action of bioactive peptides; and it develops new natural foods and drugs to meet the safety and health needs.

[0003] The use of computer programming in biological research has greatly increased the importance of bioinformatics. In the past few decades, many functional peptides have been identified, making it possible for machine learning algorithms to predict different peptides. Some recent prediction models are specifically designed to predict peptide functions from sequence information alone, without validation or using any prior knowledge as input. In addition, various physicochemical feature-based methods, mainly including amino acid composition, pseudo-amino acid composition, normalized amino acid composition, hydrophobicity, net charge, isoelectric point, alpha-helix propensity, beta-sheet propensity and turn propensity have been proposed for predicting peptides. Structural data can effectively model the functional information of peptides, and the description degree of peptide sequence as a single data is not enough. Multi-modal data of sequence data and structure data can effectively extract data characteristics from different perspectives, so as to better predict peptides, and a single model is difficult to capture the characteristics of multi-modal data. SUMMARY

[0004] The present application aims at the problem that a single model is difficult to capture the characteristics of multi-modal data, and provides a multi-modal fusion deep learning model. In the model, structural properties are introduced, a multi-scale dilated convolution and BiLSTM fusion model is used to obtain multiple activities of bioactive peptides in the sequence, a multi-scale CNN module is used to process the structure input to obtain the characteristics of active peptides, the obtained multi-modal characteristics are processed, information fusion is performed, the complementarity of the sequence and structural properties of active peptides is effectively considered, and the characteristics of bioactive peptides are captured from multi-modal characteristics.

[0005] The technical scheme of the present application is as follows:

[0006] The application discloses a multi-modal fusion deep learning model, which comprises a multi-modal data input module, a peptide sequence coding module, a peptide structure coding module and a classification module; the peptide sequence and the peptide structure of a bioactive peptide are input by the multi-modal data input module; the peptide sequence coding module adopts a multi-scale dilated CNN and a BiLSTM model to fuse and extract features of multiple scales of the peptide sequence; the peptide structure coding module adopts a multi-level CNN model to extract the characteristics of the peptide structure data; and the classification module connects the output of the peptide sequence coding module and the output of the peptide structure coding module as the final features and uses the output of the classification module as the input of an output layer.

[0007] Further, the multi-scale dilated CNN comprises: when applied to a one-dimensional CNN, the multi-scale dilated CNN can be calculated as follows:

[0008]

[0009] wherein y i represents the output of the i-th element in convolution, x i is the i-th element input, omega is the weight of the filter, the length of the filter is K; r is the dilated rate, and r=1 in the dilated convolution is equal to the ordinary convolution, and when the dilated rate r=2, a zero is inserted between adjacent convolution weights.

[0010] Further, the LSTM model comprises the following steps:

[0011] Calculate the forgetting state:

[0012] f t = sigma (W f · [h t-1 , x t ] + b f ),

[0013] v t = tanh (W c · [h t-1 , x t ] + b v ),

[0014] Calculate the input state:

[0015] i t = sigma (W i · [h t-1 , x t ] + b i ),

[0016] Calculate the cell state:

[0017] C t = f t · C t-1 + i t · v t,

[0018] The output gate and hidden state of the current time are calculated:

[0019] O t = sigma(W o [h t-1 ,x t ]+b o ),

[0020] h t = t * tanh(C t ),

[0021] The forward and backward outputs are calculated:

[0022]

[0023] where W and b represent the weights and biases of the training matrix respectively, sigma represents a nonlinear activation function with values in the range [0, 1], h represents a hidden layer unit, f t represents a forget gate unit, v t is a unit state unit, and represents an update gate unit, i.e., an input unit, C t represents a unit state, O t is a synchronization gate that synchronizes information from the previous unit and outputs it. represents an element-wise sum, which is used to sum the elements of the forward and backward outputs.

[0024] Further, the multi-modal data input module pre-processes the peptide sequence before inputting it, fills the peptide with less than 517 residues with a specific feature 'X', and converts all features of the peptide into integers.

[0025] Further, the peptide sequence encoding module inputs an amino acid sequence; and the peptide structure encoding module inputs a peptide molecular fingerprint.

[0026] Further, the classification module is a fully connected layer with five neurons with sigmoid function; the output of each neuron represents the probability of belonging to the corresponding type of peptide.

[0027] The application also includes an active peptide prediction method, which inputs peptide sequence encoding and peptide structure encoding, and uses a multi-modal fusion deep learning model to predict the function of the peptide.

[0028] Compared with the existing technology, the application has the following advantages:

[0029] 1. A multi-modal fusion deep learning model and a multi-functional bioactive peptide prediction method, which effectively fuses two multi-modal data of peptide sequence and structural characteristics, effectively extracts data characteristics from different perspectives, and thus better performs peptide function prediction.

[0030] 2. A multi-modal fusion deep learning model and a multi-functional bioactive peptide prediction method, in order to effectively target the characteristics of sequence data and structural data, different encoders are designed for feature extraction; a multi-scale dilated convolutional CNN and a BiLSTM model are used to extract the characteristics of sequence data, and a multi-level CNN model is used to extract the characteristics of structural data. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 It is a model flowchart of a multi-modal fusion deep learning model and a multi-functional bioactive peptide prediction method. DETAILED DESCRIPTION

[0032] It should be noted that the terms "first" and "second" and the like relational terms are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0033] The features and performances of the present application will be further described in detail below in conjunction with embodiments.

[0034] Please refer to Figure 1 A multi-modal fusion deep learning model, comprising a multi-modal data input module, a peptide sequence encoding module, a peptide structure encoding module and a full connection layer module; the multi-modal data input module inputs the peptide sequence and the peptide structure of the bioactive peptide; the peptide sequence encoding module uses a multi-scale dilated CNN and a BiLSTM model to fuse and extract the features of the peptide sequence in multiple scales; the peptide structure encoding module uses a multi-level CNN model to extract the characteristics of the peptide structure data; and the full connection layer module concatenates the output of the peptide sequence encoding module and the output of the peptide structure encoding module as the input of the output layer.

[0035] The multi-scale dilated CNN comprises: when applied to a one-dimensional CNN, it can be calculated as:

[0036]

[0037] where y i represents the output of the i-th element in the convolution, x i is the i-th element input, ω is the weight of the filter, and the length of the filter is K; r is the dilation rate, and r = 1 in the ordinary convolution when the dilation rate r = 2, a zero is inserted between adjacent convolution weights.

[0038] Compared with the traditional convolution, the dilated convolution can capture the multi-scale context information by setting different dilation rates, and can expand the receptive field without increasing the network parameters. Compared with the ordinary convolution operation, an additional hyperparameter called dilation rate is added in the dilated convolution. Different dilation rates can be regarded as inserting different size holes between each convolution kernel parameter.

[0039] The peptide sequence encoding module first inputs the embedding matrix of the peptide sequence into the dilated convolution block with different dilation rates (dilation rates are 2, 4, 8 respectively) for convolution, and these convolution blocks are arranged in parallel to extract the information of the peptide sequence at different scales. Then, the maximum pooling operation is used to obtain the convolution feature matrix to prevent overfitting.

[0040] Long short-term memory (LSTM) is an improved traditional recurrent neural network (RNN) that can capture the entire historical information of the input data. LSTM solves the problem of possible gradient disappearance or gradient explosion in back propagation by adding input, output and forget gates. However, in the prediction model, LSTM cannot encode information from back to front, i.e., future information is not used. Bi-LSTM solves this problem well, which is composed of two LSTM layers combined together. One LSTM unit processes forward input, and the other unit processes reverse input. Compared with the standard LSTM, BiLSTM can obtain the correlation from the history and the current information, so the network can better understand the context information. The complete LSTM hidden element has a concatenated vector for forward and reverse process output. The Bi-LSTM model includes the following steps:

[0041] Calculate the forget state:

[0042] f t = σ(W f · [h t-1 , x t ] + b f ),

[0043] v t = tanh(W c · [h t-1 , x t ] + b v ),

[0044] Compute input state:

[0045] i t = σ(W i · [h t-1 , x t ] + b i ),

[0046] Compute cell state:

[0047] C t = f t · C t-1 + i t · v t ,

[0048] Compute output gate and hidden state at current time:

[0049] O t = σ(W o [h t-1 , x t ] + b o ),

[0050] h t = O t · tanh(C t ),

[0051] Compute forward and backward outputs:

[0052]

[0053] where W and b denote the weights and biases of the training matrix, respectively, σ denotes a nonlinear activation function with values in the range [0, 1], h denotes the hidden layer unit, f t denotes the forget gate unit, v t is the cell state unit, i denotes the input gate unit, C t denotes the cell state, O t is the output gate that synchronizes information from the previous cell and outputs it. Where denotes element-wise summation, which is used to sum the elements of the forward and backward outputs.

[0054] The multi-modal data input module, before inputting the peptide sequence, pre-processes the peptide sequence, fills the peptide with less than 517 residues with a specific feature 'X', and converts all features of the peptide into integers.

[0055] The peptide sequence encoding module inputs an amino acid sequence; the peptide structure encoding module inputs a peptide molecular fingerprint.

[0056] The classification module is a fully connected layer with five neurons with sigmoid function. The output of each neuron represents the probability of belonging to the corresponding type of peptide. A fully connected layer is used as the prediction layer. The output of the peptide sequence encoding module is concatenated with the output of the structure encoding module as the final features used as input to the output layer. In the multi-label problem, the probability of each node is independent of each other, with binary cross-entropy as the loss function. The sigmoid function is used as the activation function to obtain the score of each node between 0 and 1. Finally, we use 0.5 as the threshold to obtain the predicted label of each class, and the elements in the five-dimensional prediction vector correspond to the labels of ACP, ADP, AHP, AIP and AMP, respectively.

[0057] A multi-layer convolutional model is used as the peptide structure encoding module, using three consecutive one-dimensional convolutional layers with filter numbers of 16, 32 and 32, respectively. After the convolutional layers, there are average pooling layers. Finally, a Dropout layer with a rate of 0.3 is used to avoid overfitting.

[0058] The present application also includes an active peptide prediction method, which inputs peptide sequence encoding and peptide structure encoding, and uses a multi-modal fusion deep learning model to predict the function of the peptide.

[0059] Comparative experiment

[0060] Experimental data set

[0061] The same experimental data set as a deep learning method based on convolutional neural network (CNN) and gated recurrent unit (GRU) using active peptide sequence as input (referred to as MLBP) is used. The data set was retrieved by searching the Google Scholar engine using the keyword bioactive peptide in 2020. The initial data set includes 18 bioactive peptides. Because the number of training samples is too small to train a deep neural network well, the peptide segments with less than 500 residues are removed. Therefore, five functional peptides (antimicrobial peptide AMP, anticancer peptide ACP, antidiabetic peptide ADP, antihypertensive peptide AHP and anti-inflammatory peptide AIP) are retained. The clustering tool CD-HIT is used to remove or reduce redundancy and homology. The sequence identity is set to 0.9. The final number of ACP, ADP, AHP, AIP and AMP is 646, 514, 868, 1678 and 2409, respectively. A total of 80% of the peptides are randomly sampled as the training set, and the remaining 20% are used as the test set.

[0062] Experimental environment

[0063] For training, validation and testing the method, a deep learning server was used, with hardware aspects: Xeon E5-2650v4 CPU, 15GB memory, GPX1080Ti GPU. Software aspects: Ubuntu 18.04, Python 3.6, Tensorflow 1.15.6.

[0064] Comparative method

[0065] The current state-of-the-art method based on deep learning is MLBP, which extracts and fuses the features of the peptide sequence through CNN+BiGRU. This method considers the sequence input of active peptides, but does not consider the structure feature input. At the same time, we also compared four state-of-the-art methods based on machine learning, including the second-order algorithm CLR, the high-order algorithm RAndom k-labELsets (RAKEL), the ranking low-rank learning algorithm support vector machine and binary and robust low-rank relevance (RBRL), and the deep forest-based multi-label learning deep forest (MLDF). The results show that the method we proposed is superior to the state-of-the-art multi-label methods based on deep learning and machine learning.

[0066] Evaluation of the sequence encoding module

[0067] We designed a multi-scale dilated convolution and BiLSTM model as a sequence encoding module to extract sequence features. In order to verify the excellent feature extraction ability of the multi-scale dilated convolution and BiLSTM model we designed for multi-functional active peptide sequences, we deleted the structure encoding module, and kept the sequence feature input module, sequence encoding module and classification module as a multi-functional predictor of bioactive peptides. The same experiment was compared with the current advanced MLBP based on multi-functional bioactive peptide sequences. As shown in Table 1, our sequence encoding model improved by 0.08 in Precision, 0.045 in Coverage, and 0.07 in Accuracy, showing that the sequence encoding module has good sequence feature extraction ability.

[0068] Table 1 Comparison of sequence encoding module and MLBP on test data set

[0069]

[0070] Evaluation of the structure encoding module

[0071] To verify the role of the structure encoding part in the multi-modal fusion learning model, we only kept the sequence feature input when inputting, that is, we kept the structure feature input module, the structure feature encoding module, and the classification module. Compared with the experiment when both sequence and structure features were inputted, the results showed that the model performance when inputting structure and sequence features was better than that when only inputting sequence features in five indicators. This indicates that the structure feature can effectively model the functional information of the peptide.

[0072] Table 2 Comparison of the performance of sequence encoding module and structure encoding module on test dataset

[0073]

[0074] Evaluation of the overall model

[0075] To further evaluate our model, we took the sequence feature and structure feature of two modalities as the input of the multi-modal fusion deep learning model. The results showed that the multi-modal fusion deep learning was better than the current best method, as shown in Table 3. And after the fusion of the two modalities, the five indicators were better than the performance when inputting a single feature. This indicates that the model can effectively fuse the sequence feature and structure feature of the multi-functional active peptide.

[0076] Table 3 Comparison of the performance of multi-modal fusion deep learning and the current best method on test dataset

[0077]

[0078]

[0079] Evaluation of multi-scale dilated convolution and BiLSTM

[0080] Evaluation of multi-scale dilated convolution

[0081] To verify the performance of the multi-scale dilated convolution in extracting peptide sequence features, we performed a smile experiment to evaluate the multi-scale dilated convolution block. First, in our method we replaced the dilated convolution with a normal CNN, the purpose is to compare the performance of multi-scale dilated convolution and ordinary multi-scale convolution in extracting peptide features, the results of Table 4 show that the effect of using multi-scale dilated convolution is obviously better than that of multi-scale ordinary CNN. Considering that the length of the peptide sequence in the benchmark dataset ranges from 5 to 517 residues, the length span is very wide, so we consider using different dilated convolution rates to extract sequence features in different length ranges. Therefore, we designed a model with different dilated rates to test the ability of different dilated rates to extract peptide sequence features. We first set the dilated rate of the multi-scale dilated convolution block to the same rate, that is, all 2, 4 or 8. The results show that the difference between the three is not big and is obviously better than the effect of using ordinary CNN. Considering that the length span of bioactive peptides is very wide, for some too long or too short peptide sequences, if the dilated rate is set to be consistent, the features in them cannot be fully extracted. In order to solve the problem of the difficulty of fully extracting sequence features caused by the wide length span of bioactive peptides, we set the dilated rates of the three convolution blocks in the multi-scale dilated convolution block to 2, 4 and 8 respectively. The results show that the effect of dilated rate of 2, 4 and 8 is obviously better than that of dilated rate of 2, 4 or 8. It shows that different dilated rates can fully extract features of different spans.

[0082] Table 4 Performance comparison of different dilated rates (r) on the test dataset

[0083]

[0084] BiLSTM evaluation

[0085] The role of BiLSTM in this model is to effectively fuse the features extracted by the multi-scale dilated convolution. In order to verify the effect of BiLSTM, we designed 5 kinds of DNN models, and evaluated the effect of each model on the test set, the results show that BiLSTM obtains the best performance. Table 5 lists the performance of these different DNNs.

[0086] Table 5 Performance comparison of LSTM and other DNN models on test

[0087]

[0088] The above embodiments only express the specific implementation of the present application, which is described in more detail and specifically, but cannot be understood as a limitation to the protection scope of the present application. It should be noted that for those skilled in the art, without departing from the technical concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A multi-modal fusion deep learning model, characterized in that, The application relates to a multi-modal fusion deep learning model for predicting the function of a multifunctional bioactive peptide, comprising a multi-modal data input module, a peptide sequence encoding module, a peptide structure encoding module and a classification module; the multi-modal data input module inputs the peptide sequence and the peptide structure of a bioactive peptide; the peptide sequence encoding module adopts a multi-scale dilated CNN and a BiLSTM model to fuse and extract the features of the peptide sequence in multiple scales; the peptide structure encoding module adopts a multi-scale CNN model to extract the features of the peptide structure data; the classification module connects the outputs of the peptide sequence encoding module and the peptide structure encoding module as the final features for the input of an output layer. When applied to a one-dimensional CNN, the multi-scale dilated CNN can be calculated as follows: in In the convolution, the first... The output of each element For the first One element input, These are the filter weights, and the filter length is... ; It refers to the dilation rate, in dilated convolution. Equal to ordinary convolution, when the dilation rate At that time, a zero is inserted between adjacent convolution weights; the BiLSTM model includes the following steps: Calculate the forget state: ， , Calculate the input state: ， Calculate the cell state: ， Calculate the output gate and the hidden state of the current time: ， ， Calculate the forward and reverse outputs: ， where W and b denote the weights and bias of the training matrix respectively, σ represents a nonlinear activation function with values in the range [0, 1], and h represents a hidden layer unit, represents a forget gate unit, is a cell state unit, which denotes an update gate unit, i.e., an input unit, represents a cell state, is a sync gate, which synchronizes information from the previous cell and outputs it; represents an element-wise sum, which is used to sum the elements of the forward and backward outputs; the multi-modal data input module, before inputting the peptide sequence, pre-processes the peptide sequence, fills the peptide with less than 517 residues with the feature 'X', and converts all features of the peptide into integers. 2.The multi-modal fusion deep learning model of claim 1, wherein, The input of the peptide sequence encoding module is an amino acid sequence; the input of the peptide structure encoding module is a peptide molecular fingerprint. 3.The multi-modal fusion deep learning model of claim 1, wherein, The classification module is a fully connected layer with five neurons with sigmoid functions; the output of each neuron represents the probability of belonging to the corresponding type of peptide.

4. A method of predicting multifunctional bioactive peptides, characterized by, The peptide sequence encoding and the peptide structure encoding are input into the multi-modal fusion deep learning model according to any one of claims 1-3 to predict the function of a multifunctional bioactive peptide.

Images