A method for predicting antiviral peptides based on biological segmentation

Abstract

The present invention discloses an antiviral peptide prediction method based on biological segmentation, which belongs to the technical field of antiviral peptide drug prediction, and includes the following steps: S1: constructing a data set; S2: biological segmentation processing; S3: biological segmentation feature representation; S4: constructing a biological segmentation prediction model; S5: model training; S6: using the model for prediction. The present invention uses biological segmentation to represent peptide sequences as sentences in natural language processing, and then uses Word2Vec to obtain the embedded representation of these biological segmentations. Based on the embedded representation of biological segmentations, a biological segmentation prediction model is constructed to achieve higher prediction accuracy for antiviral peptides, and the effectiveness of the proposed biological segmentation method is proved by comparing the biological segmentation prediction model of the present invention with other most advanced models.

Description

Technical Field

[0001] The present invention relates to the technical field of antiviral peptide drug prediction, and in particular to an antiviral peptide prediction method based on biological word segmentation. Background Art

[0002] Antiviral peptides (AVPs) are antimicrobial peptides (AMPs) with potential antiviral infection ability. They have been experimentally confirmed to have the effect of blocking viruses from entering host cells. They play an important role in the medical field and can be used to treat and prevent viral infections, develop drugs, etc. The antiviral mechanisms of antiviral peptides include: (1) Inhibiting the attachment of viruses to host cell receptors. (2) Preventing the interaction between viruses and host cell co-receptors. (3) Inhibiting the fusion of viral envelope and host cell membrane. (4) Inhibiting viral replication by interacting with viral nucleic acids. (5) Participating in the post-translational modification process of some viral proteins. (6) Interfering with viral particle assembly. Most antiviral peptides reduce viral loads through the above mechanisms alone or in combination. Compared with commonly used antiviral drugs, antiviral peptides are not only effective against re-emerging drug-resistant viruses, but also natural, biodegradable by peptidases, and have low toxicity. Most importantly, they can exert a broad spectrum of activity against different viruses. Therefore, antiviral peptides have recently attracted great interest as an alternative to traditional chemical antiviral drugs.

[0003] Traditional methods of obtaining antiviral peptides include natural sources and biological sources, which are usually low in yield, difficult to synthesize, have potential toxicity and side effects, and are difficult to be effectively used in precision medicine and other fields, and are heavily dependent on advanced instruments and skilled personnel. In addition, with the advent of the post-genomic era, the number of peptide sequences in various databases is also increasing rapidly. Therefore, it is necessary to develop a computer tool to efficiently discover new antiviral peptides in protein sequences.

[0004] Traditional bioinformatics methods are based on quantitative structure-activity relationship methods, protein-peptide docking methods, protein-small ligand docking methods, molecular dynamics methods, etc. In recent years, learning technology has been particularly powerful in processing large amounts of biomedical data and has achieved great success in various bioinformatics applications. In recent years, many machine learning and deep learning-based methods have been developed to promote the identification and discovery of antiviral peptides. AVPpred is the first antiviral peptide prediction tool developed using support vector machine (SVM). Chang and Yang first used the random forest (RF) algorithm to predict antiviral peptides. ClassAMP is the first tool to use a multi-classification approach to predict peptides with antibacterial, antifungal and antiviral activities. AI4AVP uses PC6 encoding to convert sequence data into matrices, uses a generative adversarial network to increase the number of antiviral peptides in the positive training data set, and enables the deep learning convolutional neural network model to learn from the negative data set. So far, the only tool reported to predict AVP using a regression method is AVP-IC50Pred, which uses the half-maximal inhibitory concentration (IC50) value to predict antiviral activity.

[0005] Most previous machine learning methods are based on handcrafted sequential features extracted from peptide sequences. Recently, protein language models have been used to obtain embedded representations of peptide sequences to predict AVPs. For example, Du et al. developed the UniDL4BioPep model to predict AVPs based on the ESM-2 model. Lobo et al. developed an AVP prediction model based on the ESM1b and ProtT5 protein language models. Although these methods have achieved promising performance, they usually ignore the secondary structure of peptides, which may provide additional useful information for peptide prediction tasks. Chowdhury et al. used the most abundant features obtained from the physicochemical and structural properties of amino acid sequences to distinguish AVPs from non-AVPs. Through feature selection analysis, they demonstrated that secondary structure is the most critical feature for predicting AVPs.

[0006] How to improve the accuracy of predicting antiviral peptides is a problem that needs to be solved urgently. To this end, the present invention proposes an antiviral peptide prediction method based on biological word segmentation. Summary of the invention

[0007] The technical problem to be solved by the present invention is: how to improve the accuracy of predicting antiviral peptides, and provide a method for predicting antiviral peptides based on biological segmentation. This method takes into account that the secondary structure of small peptides is mainly composed of α helices and loops, uses 1mer and 4mer of peptide sequences for segmentation, and then extracts embeddings from these models to predict AVP. Our results verify that more sequence information can be obtained compared with existing models, and our model achieves higher prediction accuracy.

[0008] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:

[0009] S1: Building a dataset

[0010] Establish a benchmark data set and divide the benchmark data set into two derived data sets, namely a training set and a test set;

[0011] S2: Biological word segmentation processing

[0012] According to the characteristics of protein α-helix structure and ring structure in the secondary structure of peptide, the primary structure sequence of peptide is segmented to obtain two sequences after biological segmentation.

[0013] S3: Biological word segmentation feature representation

[0014] The two sequences after biological word segmentation are input into the Word2Vec prediction model respectively, and the fixed-length words are represented as one-dimensional vectors through the Word2Vec embedding method. The one-dimensional vectors obtained from the two sequences are combined to obtain two feature matrices.

[0015] S4: Building a biological word segmentation prediction model

[0016] A hybrid neural network formed by a CNN network and a Transformer model is used as the first channel of a biological word segmentation prediction model, and another single Transformer model is used as the second channel of the biological word segmentation prediction model, and a biological word segmentation prediction model is formed by combining a fully connected network;

[0017] S5: Model training

[0018] Processing the peptide sequences in the training set in steps S2 and S3 to obtain two feature matrices after biological word segmentation feature representation, and inputting the two feature matrices into the biological word segmentation prediction model for training to obtain a trained biological word segmentation prediction model;

[0019] S6: Using the model to make predictions

[0020] The biological word segmentation prediction model trained in step S5 is used to predict the antiviral peptides in the test set.

[0021] Furthermore, in step S1, the generation process of the benchmark data set is as follows:

[0022] S11: Antiviral peptide data were collected through starPepDB and UniProt databases;

[0023] S12: Record the distribution of positive and negative sample lengths of antiviral peptide sequences through experiments.

[0024] Furthermore, in the step S2, the biological word segmentation process is as follows: the protein sequence is selected to be biologically segmented in a manner of every four residues as a word and one residue as a word, thereby realizing the segmentation of the primary structure sequence of the peptide, wherein four residues are represented as 4mer and one residue is represented as 1mer.

[0025] Furthermore, in step S3, the Word2Vec prediction model is a CBOW model, and the input of CBOW is represented by biological words w0, w1, w2, w3, ..., w n The target word w i Prediction is done by the following objective function:

[0026]

[0027] Among them, w i is the central word, i.e. the target word, and w i-t ,...,w i-1 w i+1 ,...,w i+t represents the context word, the conditional probability P(w i |w i-t ,...,w i-1 w i+1 ,...,w i+t ) is defined as follows:

[0028]

[0029] Among them, h i is the value of the N-dimensional vector in the hidden layer, and the calculation formula is as follows:

[0030]

[0031] Where C is the window size and the weight matrix V is located between the input layer and the hidden layer.

[0032] Furthermore, in step S3, the two characteristic matrices are respectively a characteristic matrix of 1mer and a characteristic matrix of 4mer.

[0033] Furthermore, in step S4, the CNN network includes a convolutional layer, a pooling layer and a fully connected layer connected in sequence, and the CNN network is activated by using a ReLU activation function, and the ReLU activation function is defined by the following formula:

[0034] R(z)=max(0,z)

[0035] Among them, R(z) is the output of the ReLU activation function, and z represents the input value.

[0036] Furthermore, in step S4, the Transformer model encodes the input sequence through a multi-head attention mechanism, residual connection, layer normalization and a feedforward neural network. The multi-head attention mechanism is a collection of multiple different self-attention mechanisms. The self-attention function expression is as follows:

[0037]

[0038] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of query and key, used to scale the attention score, which is calculated by taking the dot product between the query and key and dividing by To obtain.

[0039] Furthermore, in step S6, when predicting the antiviral peptides in the test set, the specific processing process is as follows:

[0040] S61: inputting the obtained 1mer feature matrix into the first channel for feature extraction, and inputting the obtained 4mer feature matrix into the second channel for feature extraction;

[0041] S62: Input the extracted features into the fully connected network to obtain the final prediction probability.

[0042] Compared with the prior art, the present invention has the following advantages: the antiviral peptide prediction method based on biological segmentation uses biological segmentation to represent peptide sequences as sentences in natural language processing, and then uses Word2Vec to obtain the embedded representation of these biological segmentations. Based on the embedded representation of biological segmentation, a biological segmentation prediction model is constructed to achieve higher prediction accuracy for antiviral peptides, and the effectiveness of the proposed biological segmentation method is proved by comparing the biological segmentation prediction model of the present invention with other most advanced models. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of the process of the antiviral peptide prediction method based on biological word segmentation in Example 1 of the present invention;

[0044] Figure 2 is a histogram of the length distribution of positive and negative samples of the antiviral peptide data in Example 2 of the present invention;

[0045] Figure 3 is an example diagram of the secondary structure predicted by Jpred4 of the data set in Example 2 of the present invention;

[0046] Figure 4 is a schematic diagram of the secondary structure composition of the antiviral peptide residues in the data set in Example 2 of the present invention;

[0047] Figure 5is a schematic diagram of the biological word segmentation method in the second embodiment of the present invention;

[0048] Figure 6 This is a schematic diagram of 4mer word segmentation using a biological word segmentation method in Example 2 of the present invention;

[0049] Figure 7 This is a schematic diagram of 1mer segmentation using the biological segmentation method in Example 2 of the present invention;

[0050] Figure 8 is a flowchart of the Transformer encoding layer in Embodiment 2 of the present invention;

[0051] Fig. 9 : It is a confusion matrix diagram of different neural network frameworks on the test set in Example 2 of the present invention, wherein (A) is the confusion matrix of CNN and BiGRU hybrid neural network, (B) is the confusion matrix of CNN and LSTM hybrid neural network, (C) is the confusion matrix of CNN and TCN hybrid neural network, (D) is the confusion matrix of CNN and BiLSTM hybrid neural network, (E) is the confusion matrix of CNN and GRU hybrid neural network, and (F) is the confusion matrix of CNN and Transformer hybrid neural network in the model of the present invention;

[0052] Fig.10 1 is a diagram of the sampling results of the T-SEN input and output layers in the second embodiment of the present invention, where (A) is the input and (B) is the output;

[0053] Fig.11 is a bar graph of the data distribution of anticancer peptides in Example 2 of the present invention;

[0054] Fig.12 is a bar graph of the data distribution of DPP-IV inhibitory peptides in Example 2 of the present invention;

[0055] Fig.13 is a secondary structure composition diagram of peptide residues in the anticancer peptide data set in Example 2 of the present invention;

[0056] Fig.14 It is a secondary structure composition diagram of peptide residues in the DPP-IV inhibitory peptide data set in Example 2 of the present invention. DETAILED DESCRIPTION

[0057] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0058] Embodiment 1

[0059] like Figure 1As shown, this embodiment provides a technical solution: a method for predicting antiviral peptides based on biological word segmentation, comprising the following steps:

[0060] S1: Dataset Construction

[0061] The data of antiviral peptides were collected for training. In order to predict antiviral peptides more accurately, the dataset used by SergioA et al. was used. The dataset was obtained from the starPepDB and UniProt databases. The dataset contains 2944 positive samples and 2944 negative samples, of which the training set accounts for about 80% of the total dataset (4642 samples), and the rest is used as a test set for performance evaluation and comparison of the model; the details of the benchmark dataset are shown in Table 1.

[0062] Table 1: Details of the benchmark datasets

[0063] Dataset Positive Sample Negative samples source Training set 2321 2321 starPepDB Test Set 623 623 UniProt

[0064] We chose to obtain training and test data from starPepDB and UniProt databases for the following reasons: First, to the best of our knowledge, starPepDB is the largest database compiled so far, and when starPepDB is updated, corresponding operations will be performed to ensure that the data is updated and avoid overfitting that may occur due to the long-term use of these datasets. The UniProt database is one of the most informative and resource-rich protein databases, which combines data from three major databases such as Swiss-Prot, TrEMBL, and PIR-PSD. Second, researchers have already used this data in their studies, and we will subsequently compare our results with previous models. In addition, both datasets are publicly available, which ensures that subsequent researchers can download the datasets themselves.

[0065] As a binary classification experiment, we analyzed the distribution of sample sequence lengths. Figure 2 As shown, the sequence length of the antiviral peptides ranges from 5 to 100. The analysis results show that in the sequence length interval (0,25], the proportion of positive and negative samples is the largest, accounting for about 3 / 4 of the total number of samples respectively. The length distribution of positive and negative samples is similar.

[0066] In addition, we used JPred4 to predict the secondary structure of AVP. Figure 3 is an example of a prediction result. Then, we analyzed the secondary structure composition of all samples in the dataset. Figure 4 As shown, among all the 118372 residues, most of them belong to α helix (protein α helical structure) and loop (protein ring structure), reaching 46.4% and 45.6% respectively. Only a small part forms β fold (protein β fold structure).

[0067] S2: Biological word segmentation processing

[0068] Inspired by the secondary structure composition of AVP peptides, we use the method of biological word segmentation to represent peptide sequences. Specifically, considering that the secondary structures of these peptides mainly include α-helices and loops, the peptide sequences are represented as natural language sentences, four residues (4mer) are one word (in natural language processing, peptide sequences and k-mers correspond to sentences and words, respectively), and one residue (1mer) is one word.

[0069] The workflow of the biological word segmentation prediction model in this embodiment mainly includes the following steps: (1) constructing an antiviral peptide dataset and processing the sequence into a fixed length; (2) feature representation: considering that the secondary structure of these peptides mainly includes α-helix and loop, the peptide sequence is represented as a natural language sentence, four residues (4mer) is a word (in natural language processing, peptide sequences and k-mers correspond to sentences and words, respectively), and one residue (1mer) is a word. These sentences (sequences) are then input into the Word2Vec model respectively, where fixed-length words are represented as one-dimensional vectors. Therefore, the peptides can be represented as two feature matrices respectively; (3) Neural network: in order to further extract meaningful feature information from these data, the two feature matrices are respectively input into a hybrid neural network composed of CNN and Transformer and a separate Transformer; (4) Prediction: the features extracted from the main neural network (biological word segmentation prediction model) are input into the fully connected network to obtain the final prediction probability. The entire workflow is as follows: Figure 5 shown.

[0070] S3: Biological word representation based on Word2Vec

[0071] Most AVP peptides are mainly composed of α-helices and loops. We choose to split the peptide sequence into words with four residues and one residue respectively. We take four consecutive residues as a word because in the α-helical structure, it rotates and rises along the main axis in a regular manner, and every 3.7 amino acids rotate and rise. Therefore, we assume that more extensive information can be extracted by using four residues as words in the protein language model. In the field of natural language processing, multi-granularity processing of text can often extract more information from sentences. For example, text can be processed using word-level and sentence-level granularity, which means that when analyzing and processing text information, we should not only pay attention to the semantic information and structure of the overall text, but also pay attention to information with smaller or larger granularity. In this way, more sequential information can be obtained and the prediction performance can be improved.

[0072] Given the biological words of the peptide sequence, the Word2Vec prediction model is used to generate the embedding representation of the word. The Word2Vec prediction model is a machine learning model based on a feedforward neural network, which can be used to generate vector representations of words in text and has been widely used in bioinformatics problems. The Word2Vec prediction model includes two models: the continuous bag of words model (CBOW) and the Skip-gram model. In this study, CBOW is used to build our Word2Vec prediction model, which can extract sequence features to predict the target word based on the context. The model consists of three layers: input layer, hidden layer, and output layer. First, multi-scale representation and feature extraction are performed at the word level. Subsequently, the word-level features are input into the pre-trained model for multi-granular semantic analysis using CBOW. It provides unlabeled training data and encodes the semantic information of each word in the corpus. Specifically, the input of CBOW is represented by biological words as w0, w1, w2, w3, ..., w n The target word w i The following objective function can be used for prediction:

[0073]

[0074] Among them, w i is the central word, i.e. the target word, and w i-t ,...,w i-1 w i+1 ,...,w i+t represents the context word. The conditional probability P(w i |w i-t ,...,w i-1 w i+1 ,...,w i+t ) is defined as follows:

[0075]

[0076] Among them, h i is the value of the N-dimensional vector in the hidden layer, and the calculation formula is as follows:

[0077]

[0078] Where C is the window size and the weight matrix V is located between the input layer and the hidden layer. For example, consider a sequence ACEKLHET, and our model performs 1mer, 4mer segmentation on it with a step size of 1. Figure 6 , 7 shown.

[0079] Our model may benefit from other Word2Vec prediction models (e.g., Skip-gram, fast-text). However, we choose the CBOW model because it uses the average of the context word vectors to predict the target word. In contrast, Skip-gram requires predicting each context word separately. Compared with the skip-gram model, our model has the advantage of uniformly organizing the information distributed in the dataset to pre-train the embedding layer. In addition, compared with Skip-gram and fast-Text, the CBOW model is more efficient in training. In addition, the CBOW model is better at capturing the semantic relationship between common words during the learning process.

[0080] S4: Building a biological word segmentation prediction model

[0081] The obtained 1mer and 4mer feature matrices are respectively input into the hybrid neural network of CNN and Transformer and Transformer for information extraction. Convolutional neural network (CNN) is a powerful deep learning model that is widely used in various tasks, including object detection, speech recognition, computer vision, image classification and bioinformatics. The main components of CNN include convolutional layers, pooling layers and fully connected layers. It can automatically learn and identify features from data without manual feature extraction. The multi-head attention in the Transformer model is equivalent to a collection of multiple different self-attentions, such as Figure 8 As shown. Given an input sequence X=(x1,x2,...,x n ), the output of the self-attention function is a weighted sum sequence Y = (y1, y2, ..., y n ), where each y i are all weighted sums of the input sequences. The weighted sum is obtained by weighted summation of all input positions, and the weight is determined by the attention score. The self-attention function expression is as follows:

[0082]

[0083] Where: Q is the query matrix, K is the key matrix, V is the value matrix, d k is the dimension of query and key, used to scale the attention score. The attention score is obtained by calculating the dot product between the query and the key, and then dividing by Scaling, since our task is to make classification predictions, only the encoder part in the Transformer is needed to complete feature extraction. Therefore, the decoder part is directly replaced by the Softmax module. The Softmax function is used to obtain standardized weights, and finally these weights are applied to the value matrix to obtain the weighted sum as the output of the self-attention function.

[0084] S5: Using the model to make predictions

[0085] The prediction model in step S4 is used to predict the antiviral peptides in the dataset, and the extracted information is input into a fully connected network to derive the final prediction probability for identifying the antiviral peptides.

[0086] Embodiment 2

[0087] like Figures 2 to 14 As shown, this embodiment provides a technical solution: a method for predicting antiviral peptides based on biological word segmentation, the main technical solution includes the following contents:

[0088] 1. Materials and Methods

[0089] 1.1 Dataset

[0090] High-quality benchmark datasets are crucial for developing machine learning methods for predicting AVPs. Pinacho-Castellanos et al. established comprehensive data from the largest experimentally validated non-redundant peptide dataset. In this example, we used the same dataset to build our model for fair comparison. Positive samples (AVPs) and negative samples (non-AVPs) are from starPepDB and UniProt, respectively. The dataset contains 2944 positive samples and 2944 negative samples, of which the training set accounts for approximately 80% of the total dataset (4642 samples), and the rest is used as a test dataset for model performance evaluation and comparison. Table 1 shows the details of the benchmark dataset.

[0091] Table 1 Detailed information of the benchmark datasets

[0092] Dataset Positive Sample Negative samples source Training set 2321 2321 starPepDB Test Set 623 623 UniProt

[0093] As a binary classification experiment, we analyzed the distribution of sample sequence lengths. Figure 2 As shown, the sequence length of the antiviral peptides ranges from 5 to 100. The analysis results show that in the sequence length interval (0,25], the proportion of positive and negative samples is the largest, accounting for about 3 / 4 of the total number of samples respectively. The length distribution of positive and negative samples is similar.

[0094] In addition, we used JPred4 to predict the secondary structure of AVP. Figure 3is an example of a prediction result. Then, we analyzed the secondary structure composition of all samples in the dataset. Figure 4 As shown, among all the 118372 residues, most of them belong to α helix (protein α helical structure) and loop (protein ring structure), reaching 46.4% and 45.6% respectively. Only a small part forms β fold (protein β fold structure), see Appendix 2 for details.

[0095] Table 2 Secondary structure composition of peptide residues in the antiviral peptide dataset

[0096] sequence α-helix (H) β-fold (E) ring total 1-300 2801 596 2862 6259 300-600 2788 493 2633 5,914 600-900 2936 537 2668 6141 900-1200 3174 414 2690 6278 1200-1500 3052 484 2736 6272 1500-1800 2818 593 3151 6562 1800-2100 3078 543 2690 6311 2100-2400 2122 592 3163 5866 2400-2700 1941 205 2734 4880 2700-3000 2583 528 2587 5698 3000-3300 2870 320 2392 5582 3300-3600 1753 595 2987 5335 3600-3900 4757 480 3788 9025 3900-4200 1963 337 3032 5332 4200-4500 2269 492 1689 4450 4500-4800 3238 353 2264 5855 4800-5100 3050 706 3611 7367 5100-5400 2756 388 2386 5530 5400-5700 3582 537 2755 6874 5700-5888 1445 284 1112 2841 1-5888 54976 9477 53930 118372

[0097] In this study, we chose to obtain training and test data from starPepDB and UniProt databases for the following reasons: First, as far as we know, starPepDB is the largest database compiled so far, and when starPepDB is updated, corresponding operations are performed to ensure data updates to avoid overfitting that may occur due to long-term use of these data sets. The UniProt database is the most information-rich and resource-rich protein database, which is composed of data from three major databases: Swiss-Prot, TrEMBL, and PIR-PSD. Secondly, researchers have used these data in their studies, and we will compare them with previous models in the future. In addition, both data are open source, ensuring that subsequent researchers can download the reproduction code by themselves.

[0098] 2. Model workflow

[0099] Inspired by the secondary structure of AVP peptides, we used the method of biological word segmentation to represent peptide sequences to establish our biological word segmentation prediction model for the first time. The workflow of the biological word segmentation prediction model mainly includes the following steps: (1) constructing an antiviral peptide dataset and processing the sequences into fixed lengths; (2) feature representation: considering that the secondary structures of these peptides mainly include α-helices and loops, the peptide sequences are represented as sentences in natural language processing, with four residues (4mer) as one word and one residue (1mer) as one word (in natural language processing, peptide sequences and k-mers correspond to sentences and words, respectively). These sentences are then input into the Word2Vec prediction model, where fixed-length words are represented as one-dimensional vectors. Therefore, the peptides can be represented as two feature matrices respectively; (3) main neural network: in order to further extract meaningful feature information from these data, the embedding information obtained by the two Word2Vec prediction models is input into two network channels. The first channel is a hybrid neural network composed of CNN and Transformer (hereinafter referred to as Transformer1). The second channel is Transformer (hereinafter referred to as Transformer2); (4) Prediction: The features extracted from the main neural network are input into the fully connected network to obtain the final prediction probability. The entire workflow is as follows Figure 5 shown.

[0100] 2.1 Biological Word Representation Based on Word2Vec

[0101] Recognizing that most AVP peptides are mainly composed of alpha helices and loops, we chose to split the peptide sequence into words with four residues and one residue respectively. We regard four consecutive residues as a word. Therefore, we assume that more extensive information can be extracted by using four residues as words in the language model. In the field of natural language processing, multi-granularity processing of text can often extract more information from a sentence. For example, text can be processed using word-level and sentence-level granularity, which means that when analyzing and processing text information, we should not only pay attention to the semantic information and structure of the overall text, but also pay attention to information with smaller or larger granularity. In this way, more sequential information can be obtained and the prediction performance can be improved.

[0102] Given the biological words of the peptide sequence, the Word2Vec prediction model is used to generate the embedding representation of the word. The Word2Vec prediction model is a machine learning model based on a feedforward neural network, which can be used to generate vector representations of words in text and has been widely used in bioinformatics problems. The Word2Vec prediction model includes two models: continuous bag of words (CBOW) and Skip-gram. In this study, CBOW is used to build our Word2Vec prediction model, which can extract sequence features to predict the target word based on the context. The model consists of three layers: input layer, hidden layer, and output layer. First, multi-scale representation and feature extraction are performed at the word level. Subsequently, the word-level features are input into the pre-trained model for multi-granular semantic analysis using CBOW. It provides unlabeled training data and encodes the semantic information of each word in the corpus. Specifically, the input of CBOW is represented by biological words as w0, w1, w2, w3, ..., w n The target word w i The following objective function can be used for prediction:

[0103]

[0104] Among them, w i is the central word, i.e. the target word, and w i-t ,...,w i-1 w i+1 ,...,w i+t represents the context word. The conditional probability P(w i |w i-t ,...,w i-1 w i+1 ,...,w i+t ) is defined as follows:

[0105]

[0106] Among them, h i is the value of the N-dimensional vector in the hidden layer, and the calculation formula is as follows:

[0107]

[0108] Where C is the window size and the weight matrix V is located between the input layer and the hidden layer. For example, consider a sequence ACEKLHET, and our model performs 1mer, 4mer segmentation on it with a step size of 1. Figure 6 , 7 shown.

[0109] Our model may benefit from other Word2Vec prediction models (e.g., Skip-gram, fast-text). However, we choose the CBOW model because it uses the average of the context word vectors to predict the target word. In contrast, Skip-gram requires predicting each context word separately. Compared with the skip-gram model, our model has the advantage of uniformly organizing the information distributed in the dataset to pre-train the embedding layer. In addition, compared with Skip-gram and fast-Text, the CBOW model is more efficient in training. In addition, the CBOW model is better at capturing the semantic relationship between common words during the learning process.

[0110] 2.2 Convolutional Neural Networks

[0111] Convolutional neural networks (CNNs) are a class of powerful deep learning models that are widely used in a variety of tasks, including object detection, speech recognition, computer vision, image classification, and bioinformatics. The main components of CNNs include convolutional layers, pooling layers, and fully connected layers. It can automatically learn and identify features from data without manually extracting features. In this embodiment, the input matrix size of the CNN layer is (100,150), which represents the features of 1mer biological word segmentation through Word2Vec. Our model uses a three-layer convolutional neural network, and adds normalization layers and dropout layers to speed up the training process to stabilize the model and prevent overfitting. They are activated by using the ReLU activation function because the ReLU activation function is sparse compared to the sigmoid function and reduces the possibility of gradient vanishing. The ReLU activation function is defined by the following formula:

[0112] R(z)=max(0,z) (4)

[0113] Among them, R(z) is the output of the ReLU activation function, and z represents the input value. The dropout value is set to 0.2. For the feature matrix represented by 1mer biological words, after being input into the CNN neural network, the feature matrix is ​​then input into the Transformer encoder.

[0114] 2.3 Transformer

[0115] The Transformer encoder encodes the input sequence through components such as multi-head attention mechanism, residual connection, layer normalization, and feedforward neural network. Transformer is able to better capture information in the sequence at different positions and attention points, while improving training stability through residual concatenation and normalization. The use of multi-head attention mechanism is similar to the use of multiple convolution kernels in the same convolution layer in CNN. This feature allows the model to label different features in different subspaces and avoid suppressing these features through average pooling.

[0116] The multi-head attention mechanism is equivalent to the aggregation of multiple different self-attention mechanisms, such as Figure 8 Given an input sequence X = (x1, x2, ..., x n ), the output of the self-attention function is the weighted sum sequence Y = (y1, y2, ..., y n ), where each y i is the weighted sum of the input sequence. This weighted sum is obtained by performing a weighted summation over all input positions, with the weights determined by the attention scores. The self-attention function is expressed as follows:

[0117]

[0118] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of query and key, used to scale the attention score. The attention score is calculated by taking the dot product between the query and key and dividing it by Finally, these weights are applied to the value matrix to obtain a weighted sum, which is used as the output of the self-attention function. These components help encode the information of the input sequence and encourage the model to better capture the relationships in the sequence.

[0119] 2.4 Model combination

[0120] The obtained 1mer and 4mer feature matrices are input into the hybrid neural network of CNN and Transformer1 and Transformer2 for information extraction. We input the extracted information into the fully connected layer and derive it through the Softmax activation function for predicting probability. The Softmax activation function is defined as follows:

[0121]

[0122] Among them, S(x i ) is the output of the Softmax function, and the result interval is (0,1).

[0123] 3. Evaluation indicators

[0124] We use MCC (Matthews correlation coefficient), SP (specificity), SN (sensitivity), ACC (accuracy) and AUC as evaluation indicators. Their calculation formulas are as follows:

[0125]

[0126] Among them, TP (True Positive) is the positive sample predicted by the model as positive class. TN (True Negative) is the negative sample predicted by the model as negative class. FP (False Positive) is the negative sample predicted by the model as positive. FN (False Negative) is the positive sample predicted by the model as negative class. AUC is the area under the ROC curve.

[0127] 4. Results and Discussion

[0128] 4.1 Performance of Different Token Sizes

[0129] As mentioned earlier, 4mer and 1mer are used to split peptide sequences according to two main secondary structures: α helix and loop. To verify our hypothesis, we also use 2mer and 3mer as words for word segmentation. Embeddings of different word sizes are input into the first channel composed of CNN and Transformer1 to demonstrate their effectiveness. As shown in Table 2, from 1mer to 4mer, as the number of residues increases, the training set evaluation index AUC generally shows an upward trend, indicating that 4mer contains more relevant information. Table 3 is the cross-validation performance of the model represented by 1mer, 2mer, 3mer and 4mer generated by CNN and Transformer1.

[0130] Model ACC SN SP MCC AUC 1mer 0.801 0.742 0.860 0.608 0.902 2mer 0.831 0.899 0.763 0.673 0.905 3mer 0.826 0.813 0.840 0.668 0.918 4mer 0.879 0.912 0.848 0.762 0.940

[0131] We assume that the embeddings of 1mer and 4mer correspond to α-helices and loops, and the embedding combination of 1mer and 4mer can represent extensive information of the peptide sequence. To further demonstrate the effect of 1mer and 4mer embeddings, different embedding combinations such as 1mer and 2mer, 1mer and 3mer, 2mer and 1mer, and 2mer and 3mer are also used to build the model. We input the embeddings of k-mer (k = 1, 2, 3, 4) into different channels. For example, we input the 1mer embedding into the CNN+Transformer1 channel, and the 2mer, 3mer, and 4mer embeddings into the Transformer2 channel, respectively. In a similar way, we input the embedding of 2mer into the CNN+Transformer1 channel, and the embedding of 3mer and 4mer into the Transformer2 channel, respectively. Then, in turn, we feed the 1mer embedding into the Transformer2 channel, and feed the 2mer, 3mer, and 4mer into the CNN+Transformer1 channel, respectively, and so on. As shown in Table 4, for the first six combinations, the model based on the combination of 1mer and 4mer outperformed the other models in terms of ACC, SN, MCC, and AUC results. For the last six combinations, the model based on the combination of 4mer and 1mer also outperformed the other models. This proves that selecting four residues as words and one residue as a combination word can capture more information and produce the best results. Therefore, we selected the model based on the combination of 1mer and 4mer as our final model.

[0132] Table 4 Cross-validation performance of the model based on 1mer, 2mer, 3mer and 4mer representation combinations generated by AVPpred_RBW

[0133]

[0134]

[0135] 4.2 Performance of different neural network frameworks

[0136] Different deep learning networks have been used for antiviral peptide prediction [33,35,50]. In order to select the optimal model, we analyzed the performance of CNN combined with different neural networks (such as BiLSTM, TCN, LSTM, GRU and BiGRU). Specifically, we used these networks to replace the Transformer (Transformer1 and Transformer2) in the model. As shown in Table 5, the CNN and Transformer based models outperformed the other models according to the AUC value, ACC value and MCC value. Fig. 9It can be clearly seen that our model has the largest number of correctly classified samples for AVP and non-AVP in the independent test set. In addition, the models based on other networks also show good performance, which once again proves the effectiveness of the embedding combination of 1mer and 4mer.

[0137] Table 5 Cross-validation performance of models based on different deep neural networks

[0138] Neural Networks ACC SN SP MCC AUC CNN+BiLSTM1+BiLSTM2 0.968 0.980 0.957 0.937 0.989 CNN+TCN1+TCN2 0.951 0.955 0.946 0.902 0.983 CNN+LSTM1+LSTM2 0.964 0.966 0.963 0.930 0.991 CNN+GRU1+GRU2 0.942 0.970 0.914 0.887 0.987 CNN+BiGRU1+BiGRU2 0.966 0.973 0.958 0.932 0.990 CNN+Transformer1+Transformer2 0.969 0.983 0.954 0.938 0.991

[0139] 4.3 Ablation Analysis of the Model

[0140] In this study, we designed a two-channel hybrid neural network of CNN and Transformer to build our model. In order to analyze the contribution of different modules to our model, we conducted an ablation study. Specifically, we deleted three modules respectively to obtain three model variants. Table 6 shows the cross-validation performance of these model variants, which shows that the AUC values ​​of the three model variants are all smaller than our original model. Our results illustrate the importance of all three modules of our model.

[0141] Table 6 Ablation study results of models based on different depth networks on the training set

[0142]

[0143]

[0144] 4.4 Comparison of existing models

[0145] To further validate the generalization of our model, we compared our model with several state-of-the-art models on independent test sets, such as UniDL4BioPep, ABPDiscover. The results show that our model is far superior to the two existing prediction models shown in Table 7. Our model achieved an accuracy of 90.3%, an MCC of 0.806, and an AUC of 95.4% on the independent test set, with an improvement of 6.1%, 15.6%, and 4.7% in accuracy, MCC, and AUC, respectively, which indicates that our model performs well in identifying antiviral peptides. At the same time, in order to reveal the structural transformation of feature representation during the model's feature processing, we used T-SNE to visualize the input features and the features generated from the fully connected layer, as shown in Fig.10 The results show that the positive and negative samples from the training and independent test sets are mixed in the input layer and finally clearly separated after the fully connected layer.

[0146] Table 7. Comparison with other state-of-the-art models on an independent test set

[0147] Model ACC SN SP MCC AUC UniDL4BioPep 0.842 0.916 0.79 0.694 0.907 ABPDiscover 0.828 0.764 0.892 0.662 0.896 Ours 0.903 0.929 0.876 0.806 0.954

[0148] In addition, we analyzed the performance enhancement and attributed it to the following two factors. First, the combination of Word2Vec embeddings of 4mer and 1mer can effectively capture sequence information. Second, CNN performs well in capturing local features, while Transformer can effectively handle the global context. By combining the two, the relationship between sequences and functions can be better captured and the performance of classification tasks can be improved.

[0149] 5.1 Promotion of anticancer peptides and DPP-IV inhibitory peptides

[0150] To validate the effectiveness of the proposed bio-words, the proposed model was extended to predict two other bioactive peptides, namely anticancer peptides and DPP-IV inhibitory peptides. Anticancer peptides (ACPs) are part of the antimicrobial peptide (AMP) group with anticancer activity. Inhibition of dipeptidyl peptidase IV (DPP-IV, EC3.4.14.5) is recognized as a new approach for the treatment of type 2 diabetes (T2D). So far, peptide-like DPP-IV inhibitors have been shown to normalize blood glucose concentrations in T2D subjects. The anticancer peptide dataset includes 970 positive samples and 970 negative samples taken from AntiCP2.0. The data distribution is as follows Fig.11 As shown. There are obvious differences between positive and negative samples. The length of most positive samples is in the interval (10, 20), while the length of more than half of the negative samples is in the interval (10, 30). In addition, we collected DPP-IV inhibitory peptides from iDPPIV-SCM, including 665 positive samples and 665 negative samples. Table 7 shows the detailed information of the anticancer peptides and DPP-IV inhibitory peptides datasets. Fig.12 The length distribution of positive and negative samples is shown. Positive samples only exist in the interval (0,50), and only a small part exists in the interval (50,100). For negative samples, most samples are in the interval (0,50), and only about 1.5% of negative samples exist in the interval (50,100). The summary of the DPP-IV inhibitory peptide and ACP datasets is shown in Table 8. We continue to use the JPred4 model to predict the secondary structure content in the ACP and DPP-IV inhibitory peptide datasets. The results are shown in Fig.13 and 14 Given in.

[0151] Table 8 DPP-IV inhibitory peptide and ACP datasets used for model promotion

[0152] Bioactive Peptides Data Positive Negative Source Anticancer peptides Train 689 689 AntiCP 2.0 Test 172 172 AntiCP 2.0 DPP-IV inhibitory peptide Train 532 532 iDPPIV-SCM Test 133 133 iDPPIV-SCM

[0153] We compared the performance of our model with existing models. Our model (ours) was built based on the training set using the same pipeline as AVP. Table 9 shows the results of cross-validation of our proposed framework on two other bioactive peptide training sets. Table 10 shows the prediction results of the model on an independent test set, which shows that our model outperforms other existing state-of-the-art models based on the AUC value. These results indicate that the effectiveness of the proposed pipeline is mainly attributed to the representation of biological words and prove that selecting four residues as words and one residue as words for splicing can capture more information and produce the best results.

[0154] Table 9 Performance comparison of existing models of biological word segmentation prediction model on the training set

[0155] Bioactive Peptides ACC SN SP MCC AUC Anticancer peptides 0.915 0.926 0.903 0.830 0.949 DPP-IV inhibitory peptide 0.949 0.955 0.943 0.899 0.984

[0156] Table 10 Performance comparison of biological word segmentation prediction models on existing models on independent test datasets

[0157] Bioactive Peptides Model ACC SN SP MCC AUC Anticancer peptides AntiCP 2.0 0.754 0.774 0.734 0.510 N / A UniDL4BioPep 0.735 0.734 0.737 0.471 0.805 iACP-FSCM 0.825 0.726 0.903 0.646 0.812 ours 0.765 0.733 0.797 0.530 0.834 DPP-IV inhibitory peptide UniDL4BioPep 0.853 0.861 0.846 0.707 0.938 iDPPIV-SCM 0.797 0.789 0.805 0.594 0.847 ours 0.883 0.887 0.878 0.767 0.960

[0158] 6. Conclusion

[0159] The emergence of antiviral peptides provides a new potential means for the prevention and treatment of viral infections, a promising avenue for exploring new antiviral treatments, and a more flexible and extensive solution to the ever-changing viral threats. Inspired by the secondary structure of peptides as well as multi-scale representation and multi-granularity semantic analysis, we propose to use the method of biological segmentation to represent peptide sequences as natural language sentences. Then Word2Vec is used to obtain the embedding representation of these biological words. Based on the embedding of the biological segmentation method, a model biological segmentation prediction model is constructed. The effectiveness of the proposed biological segmentation method is demonstrated by comparing our model with other state-of-the-art models. We believe that our model can be an effective tool for identifying AVPs, and the proposed biological segmentation method can be an effective representation for the identification of other bioactive peptides.

[0160] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for predicting antiviral peptides based on biological word segmentation, characterized in that: The following steps are involved: S1: Building a dataset Establish a benchmark data set and divide the benchmark data set into two derived data sets, namely a training set and a test set; S2: Biological word segmentation processing According to the characteristics of protein α-helix structure and ring structure in the secondary structure of peptide, the primary structure sequence of peptide is segmented to obtain two sequences after biological segmentation. S3: Biological word segmentation feature representation The two sequences after biological word segmentation are input into the Word2Vec prediction model respectively, and the fixed-length words are represented as one-dimensional vectors through the Word2Vec embedding method. The one-dimensional vectors obtained from the two sequences are combined to obtain two feature matrices. S4: Building a biological word segmentation prediction model A hybrid neural network formed by a CNN network and a Transformer model is used as the first channel of a biological word segmentation prediction model, and another single Transformer model is used as the second channel of the biological word segmentation prediction model, and a biological word segmentation prediction model is formed by combining a fully connected network; S5: Model training Processing the peptide sequences in the training set in steps S2 and S3 to obtain two feature matrices after biological word segmentation feature representation, and inputting the two feature matrices into the biological word segmentation prediction model for training to obtain a trained biological word segmentation prediction model; S6: Using the model to make predictions The biological word segmentation prediction model trained in step S5 is used to predict the antiviral peptides in the test set.

2. The method for predicting antiviral peptides based on biological word segmentation according to claim 1, characterized in that: In step S1, the generation process of the benchmark data set is as follows: S11: Antiviral peptide data were collected through starPepDB and UniProt databases; S12: Record the distribution of positive and negative sample lengths of antiviral peptide sequences through experiments.

3. The method for predicting antiviral peptides based on biological word segmentation according to claim 1, characterized in that: In step S2, the biological word segmentation process is as follows: select the protein sequence for biological word segmentation in a manner of every four residues as a word and one residue as a word, thereby realizing the word segmentation of the primary structure sequence of the peptide, wherein four residues are represented as 4mer and one residue is represented as 1mer.

4. The method for predicting antiviral peptides based on biological word segmentation according to claim 1, characterized in that: In step S3, Word2Ve c The prediction model is the CBOW model. The input of CBOW is represented by biological words w0, w1, w2, w3, ..., w n The target word w i Prediction is done by the following objective function: Among them, w i is the central word, i.e. the target word, and w i-t , ..., w i-1 , w i+1 , ..., w i+t represents the context word, the conditional probability P(w i |w i-t , ..., w i-1 , w i+1， ..., w i+t ) is defined as follows: Among them, h i is the value of the N-dimensional vector in the hidden layer, and the calculation formula is as follows: Where C is the window size and the weight matrix V is located between the input layer and the hidden layer.

5. The method for predicting antiviral peptides based on biological word segmentation according to claim 1, characterized in that: In step S3, the two characteristic matrices are respectively a characteristic matrix of 1mer and a characteristic matrix of 4mer.

6. The method for predicting antiviral peptides based on biological word segmentation according to claim 5, characterized in that: In step S4, the CNN network includes a convolutional layer, a pooling layer, and a fully connected layer connected in sequence. The CNN network is activated by using a ReLU activation function, and the ReLU activation function is defined by the following formula: R(z)=max(0,z) Among them, R(z) is the output of the ReLU activation function, and z represents the input value.

7. The method for predicting antiviral peptides based on biological word segmentation according to claim 6, characterized in that: In step S4, the Transformer model encodes the input sequence through a multi-head attention mechanism, residual connection, layer normalization, and a feedforward neural network. The multi-head attention mechanism is a collection of multiple different self-attention mechanisms. The self-attention function expression is as follows: Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of query and key, used to scale the attention score, which is calculated by taking the dot product between the query and key and dividing by To obtain.

8. The method for predicting antiviral peptides based on biological word segmentation according to claim 7, characterized in that: In step S6, when predicting the antiviral peptides in the test set, the specific processing process is as follows: S61: inputting the obtained 1mer feature matrix into the first channel for feature extraction, and inputting the obtained 4mer feature matrix into the second channel for feature extraction; S62: Input the extracted features into the fully connected network to obtain the final prediction probability.

Images