A method for predicting post-translational modification of protein based on multi-head attention

Abstract

The application discloses a protein post-translational modification prediction method based on multi-head attention, and belongs to the technical field of biological information, and comprises the following steps: preprocessing protein sequences with post-translational modification annotation information to obtain a legal character set, a second protein sequence data set and a benchmark data set; obtaining three kinds of vocabulary tables based on the legal character set; performing multi-scale division on protein sequence samples in the second protein sequence data set to obtain a single amino acid scale data set, a dipeptide scale data set and a tripeptide scale data set; training a corpus by using a Skip-gram model structure to obtain three kinds of word vector models; performing label prediction on protein sequence fragments of the benchmark data set by using a multi-head attention model to obtain protein sequence fragment prediction results; and the multi-scale word vector is used to comprehensively utilize polypeptide composition information in the original protein sequence, and the dependence on biological feature engineering is reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of biological information, and particularly relates to a protein post-translational modification prediction method based on multi-head attention. BACKGROUND

[0002] The prediction of whether post-translational modification occurs on the amino acid of the protein sequence through biological physical and chemical experiments needs protein extraction and fractionation after pretreatment of the experimental sample, enrichment of the detected peptide segment, and mass spectrometry analysis of the enriched peptide segment. This method has high time and economic costs and is not suitable for numerous modified post-translational prediction.

[0003] Although some existing methods for predicting post-translational modification of amino acids in protein sequences achieve protein sequence prediction through calculation, they have an outstanding advantage in efficiency and cost overhead compared to biological experimental methods for determining acylation sites. However, these calculation methods often require rich biological background knowledge to construct features from the original protein sequence according to biological feature extraction methods.

[0004] The existing methods for predicting post-translational modification of amino acids in protein sequences have the following problems: the processing of protein sequences heavily relies on biological background feature engineering, and the polypeptide composition information in the protein sequence is also ignored. SUMMARY

[0005] To overcome the above-mentioned deficiencies in the prior art, the application provides a protein post-translational modification prediction method based on multi-head attention, which comprehensively utilizes the polypeptide composition information in the original protein sequence through multi-scale word vectors and reduces the dependence on biological feature engineering.

[0006] To achieve the above-mentioned purposes, the application adopts the following technical solutions:

[0007] The application provides a protein post-translational modification prediction method based on multi-head attention, comprising the following steps:

[0008] S1, pretreating a protein sequence with post-translational modification annotation information to obtain a legal character set D aa , a second protein sequence data set D' protein , and a benchmark data set D;

[0009] S2, obtaining a first vocabulary W aa , a second vocabulary W 1 , and a third vocabulary W 2 based on the legal character set D 3 ;

[0010] S3, obtaining a first word embedding matrix E 1Second vocabulary list W 2 and the third vocabulary list W 3 The second protein sequence dataset D′ was analyzed separately. protein Protein sequence samples were divided into multiple scales to obtain single amino acid-scale datasets. Dipeptide-scale dataset and tripeptide-scale dataset

[0011] S4. Using the Skip-gram model structure, respectively, the datasets at the single amino acid scale are processed. Dipeptide-scale dataset and tripeptide-scale dataset The corpus was used for training to obtain the first word vector model, the second word vector model, and the third word vector model;

[0012] S5, based on the first vocabulary list W 1 Second vocabulary list W 2 and the third vocabulary list W 3 We used the first word vector model, the second word vector model, and the third word vector model to perform multi-head attention model label prediction on protein sequence fragments in the benchmark dataset D, and obtained the protein sequence fragment prediction results.

[0013] The beneficial effects of this invention are as follows: The multi-head attention-based protein post-translational modification prediction method provided by this invention extracts information at multiple scales from single amino acids, dipeptides, and tripeptides in protein sequences, and extracts features from word vectors at each scale through three channels. It also combines a channel attention mechanism to output features fused from multiple channels, thereby improving the prediction effect of amino acid post-translational modifications on proteins. This scheme constructs datasets at three scales—single amino acid, dipeptide, and tripeptide—from the collected original protein sequence dataset as a corpus, trains multi-scale word vectors, and uses only word vectors as features of the original sequence, no longer relying on traditional biological feature extraction methods.

[0014] Further, step S1 includes the following steps:

[0015] S11. Obtain a set of protein sequences with post-translational modification annotation information;

[0016] S12. Define the expression for the label information (Label) of protein sequence samples in the protein sequence set:

[0017] Label = {pos1,pos2,…,pos} i ,…,pos L}

[0018] Where, posi indicates that a certain post-translational modification occurs at the i-th site in the annotation information of the protein sequence sample, where i = 1, 2, …, L, and L represents the total number of sites in the protein sequence sample where a certain post-translational modification occurs;

[0019] S13, removing protein sequence samples in the protein sequence set that do not belong to the legal character set D aa , to obtain a first protein sequence dataset D protein ;

[0020] S14, performing CD-HIT on the protein sequence dataset D protein to remove redundancy and obtain a second protein sequence dataset D' protein ;

[0021] S15, setting a sliding window with a window radius of K' for each protein sequence sample in the second protein sequence dataset D' protein , and using the sliding window to crop each protein sequence sample to obtain a plurality of protein sequence fragments each having a length of 2K' + 1;

[0022] S16, obtaining a set of amino acid sites where post-translational modification occurs according to the annotation information Label of each protein sequence sample in the protein sequence set, and determining whether the amino acid at the center of each protein sequence fragment is in the set of amino acid sites where post-translational modification occurs, and if so, placing the corresponding protein sequence fragment in a positive sample dataset, otherwise placing the corresponding protein sequence fragment in a negative sample dataset;

[0023] S17, obtaining protein sequence fragments consistent in number with the protein sequence fragments in the positive sample dataset from the negative sample dataset by random sampling, and combining the obtained protein sequence fragments with the positive sample dataset to obtain a benchmark dataset D.

[0024] The beneficial effects of the above further scheme are: a method for preprocessing original protein sequences is provided, and the legal character set D aa , the second protein sequence dataset D' protein , and the benchmark dataset D provide a basis for obtaining a vocabulary and a scale dataset.

[0025] Further, the protein sequence samples in the protein sequence set are represented by the letter abbreviations of the amino acids that make up the sequence, and the set of twenty common amino acid abbreviations G, A, V, L, I, P, F, Y, W, S, T, C, M, R, and H in living organisms is taken as the legal character set D aa .

[0026] The beneficial effect of the further scheme is to provide a range of legal character sets and a representation method of protein sequence samples.

[0027] Further, the step S15 comprises the following steps:

[0028] S151, sliding a window with a radius of K' on the second protein sequence dataset D' protein to set a sliding window with a radius of K' on each protein sequence sample;

[0029] S152, sliding the sliding window on the corresponding protein sequence sample;

[0030] S153, selecting a sequence with a length of K' on both sides of the central point in the window for cutting to obtain a plurality of protein sequence fragments with a length of 2K'+1.

[0031] The beneficial effect of the further scheme is to provide a method of sliding window cutting equal-length protein sequence fragments, providing a basis for obtaining the reference dataset D.

[0032] Further, the step S2 comprises the following steps:

[0033] S21, respectively constructing a first vocabulary W 1 , a second vocabulary W 2 and a third vocabulary W 3 , and initializing the first vocabulary W 1 , the second vocabulary W 2 and the third vocabulary W 3 to empty sets;

[0034] S22, respectively constructing words in single amino acid, dipeptide and tripeptide dimensions based on the twenty amino acids in the legal character set D aa .

[0035] S23, placing the words composed of single amino acids in the first vocabulary W 1 .

[0036] S24, placing the words composed of dipeptides composed of two amino acids in the second vocabulary W 2 .

[0037] S25, placing the words composed of tripeptides composed of three amino acids in the third vocabulary W 3 .

[0038] The beneficial effect of the further scheme is to provide a specific construction method of the first vocabulary W 1 , the second vocabulary W 2 and the third vocabulary W 3 , providing a basis for obtaining the scale dataset and realizing protein sequence fragment prediction.

[0039] Further, the step S3 comprises the following steps:

[0040] S31, representing each protein sequence sample in the second protein sequence dataset D' by amino acid composition, to obtain an amino acid composition represented protein sequence sample Seq: protein

[0041] Seq=[aa1, aa2, …, aai i′ , …, aa n ]

[0042] Seq∈D' protein , aa i′ ∈D aa

[0043] wherein aa i′ represents an amino acid at the i'th position in the amino acid composition represented protein sequence sample Seq, wherein i'=1, 2, 3, …, n, n represents the total number of amino acids in the amino acid composition represented protein sequence sample Seq;

[0044] S32, dividing the amino acid composition represented protein sequence sample Seq at three scales of single amino acid, dipeptide and tripeptide respectively, to obtain a single amino acid sequence Seq 1 , a dipeptide sequence Seq 2 and a tripeptide sequence Seq 3 composed of words in the first vocabulary W 1 , the second vocabulary W 2 and the third vocabulary W 3 respectively:

[0045] Seq 1 =[aa1, aa2, …, aa i′ , …, aa n ]=Seq

[0046] Seq 2 =[(aa1aa2), (aa2aa3), …, (aa i′ aa i′+1 ), …, (aa n-1 aa n ]

[0047] Seq 3 =[(aa1aa2aa3), (aa2aa3aa4), …, (aa i′ aa i′+1 aa i′+2 ), …, (aa n-2 aa n-1 aa​n )]

[0048] aa i′ ∈W 1 , aa i′ aa i′+1 ∈W 2 , aa i′ aa i′+1 aa i′+2 ∈W 3

[0049] wherein, aa′ i′ represents an amino acid at i′-th position in a single amino acid sequence Seq 1 represented by a word in a first vocabulary W 1 aa i′ aa i′+1 represents a dipeptide at i′-th position in a dipeptide sequence Seq 2 represented by a word in a second vocabulary W 2 aa i′ aa i′+1 aa i′+2 represents a tripeptide at i′-th position in a tripeptide sequence Seq 3 represented by a word in a third vocabulary W 3 wherein, i′=1, 2, 3, …, n;

[0050] S33, sequentially performing single amino acid, dipeptide and tripeptide three-scale partitioning on protein sequence samples Seq in the second protein sequence dataset D′ protein respectively, to obtain a single amino acid scale dataset a dipeptide scale dataset and a tripeptide scale dataset

[0051] The beneficial effects of the above further scheme are: providing a method for performing multi-scale partitioning on protein sequence samples in the second protein sequence dataset D′ protein to obtain a single amino acid scale dataset a dipeptide scale dataset and a tripeptide scale dataset , providing a basis for obtaining a word vector model and performing protein sequence fragment prediction.

[0052] Further, the step S5 comprises the following steps:

[0053] S51, performing single amino acid, dipeptide and tripeptide three-scale partitioning on each protein sequence fragment in the reference dataset D respectively;

[0054] S52, respectively utilizing a first vocabulary W1 , the second vocabulary W 2 and the third vocabulary W 3 The words in the three vocabularies are used to reconstruct the protein sequence fragments divided by the three scales to obtain a plurality of amino acid sample fragment sequences a plurality of dipeptide sample fragment sequences and a plurality of tripeptide sample fragment sequences Wherein, i" represents the i" protein sequence fragment in the reference data set;

[0055] S53, converting each amino acid sample fragment sequence into an amino acid sample fragment word vector using a first word vector model, converting each dipeptide sample fragment sequence into a dipeptide sample fragment word vector using a second word vector model, and converting each tripeptide sample fragment sequence into a tripeptide sample fragment word vector using a third word vector model;

[0056] S54, constructing an amino acid sample fragment word vector matrix based on the amino acid sample fragment word vector constructing a dipeptide sample fragment word vector matrix based on the dipeptide sample fragment word vector constructing a tripeptide sample fragment word vector matrix based on the tripeptide sample fragment word vector

[0057] S55, adding position encoding information to each amino acid sample fragment word vector, each dipeptide sample fragment word vector and each tripeptide sample fragment word vector using a rotation position encoding module RoPE, to obtain an amino acid sample fragment word vector matrix after adding position encoding a dipeptide sample fragment word vector matrix and a tripeptide sample fragment word vector matrix

[0058] S56, inputting the amino acid sample fragment word vector matrix after adding position encoding into a multi-head attention module Multi-Att of a single amino acid sequence channel, inputting the dipeptide sample fragment word vector matrix into a multi-head attention module Multi-Att of a dipeptide sequence channel, and inputting the tripeptide sample fragment word vector matrix into a multi-head attention module Multi-Att of a tripeptide sequence channel, and learning the word vector matrix features under the three scales through the multi-head attention modules in each channel:

[0059]

[0060] wherein, represents the word vector matrix feature extracted by the single amino acid sequence channel for the i'th protein sequence fragment, wherein j=1 represents the single amino acid sequence channel, j=2 represents the dipeptide sequence channel, and j=3 represents the tripeptide sequence channel;

[0061] S57, the word vector matrix feature extracted by the single amino acid sequence channel for the i'th protein sequence fragment, the word vector matrix feature extracted by the dipeptide sequence channel for the i'th protein sequence fragment, and the word vector matrix feature extracted by the tripeptide sequence channel for the i'th protein sequence fragment are spliced, and the spliced result is input into an attention module Chanel-Att to learn the attention weight of the tripeptide sequence channel, to obtain the i'th protein sequence fragment feature after channel attention weight adjustment:

[0062]

[0063] wherein O i″ represents the i'th protein sequence fragment feature after channel attention weight adjustment;

[0064] S58, the i'th protein sequence fragment feature after channel attention weight adjustment is input into a full connection layer to obtain a protein sequence fragment prediction result:

[0065] Label i″ = soft max(w·O i″ +b)

[0066] wherein Label i″ represents the prediction label of the i'th protein sequence fragment, and w and b represent the neuron weight matrix and the bias of the last linear layer network respectively, wherein when the i'th protein sequence fragment is a positive sample data, its prediction label is 1, and when the i'th protein sequence fragment is a negative sample data, its prediction label is 0.

[0067] The beneficial effects of the above further scheme are: a method for predicting amino acid post-translational modification after translation of each protein sequence fragment is provided, and the polypeptide composition information in the original protein sequence is comprehensively utilized by using multi-scale word vectors, and the dependence on biological feature engineering is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is a step diagram of a protein post-translational modification prediction method based on multi-head attention in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The specific embodiments of the present application are described below to facilitate the understanding of the present application for those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0070] As Figure 1 shown, in one embodiment of the present application, the present application provides a protein post-translational modification prediction method based on multi-head attention, comprising the following steps:

[0071] At present, there is a data open source platform for protein amino acid post-translational modification problems, PTM Viewer, but only a small number of samples cannot meet the demand; Therefore, the present embodiment also collects the protein Uniport id with post-translational modification phenomenon and the coordinate information of the amino acid with PTM on the protein from the published experimental papers; Then, the complete amino acid sequence of the protein is retrieved in the Uniport database through the protein Uniport id; Combined with the amino acid sequence and the PTM position occurring thereon, a raw sample set A with one or more PTM sites is formed; The data obtained from the open source data platform PTM Viewer is taken as a raw sample set B; In order to ensure the correctness of the samples in the data set, the scheme is in accordance with the legal character set D aa The error samples in the raw sample set A and the raw sample set B are screened; After screening, the legal samples in the raw sample set A and the raw sample set B are merged together to form a raw data set D raw ;

[0072] S1, pre-processing protein sequence with post-translational modification annotation information to obtain legal character set D aa , the second protein sequence data set D' protein and the benchmark data set D;

[0073] The step S1 comprises the following steps:

[0074] S11, obtaining protein sequence set with post-translational modification annotation information based on the raw data set D raw ;

[0075] S12, defining the expression of the annotation information Label of the protein sequence sample in the protein sequence set:

[0076] Label = {pos1, pos2, …, pos i , …, pos L}

[0077] pos i represents that a certain post-translational modification occurs at the i-th site in the annotation information of the protein sequence sample, wherein i = 1, 2, …, L, and L represents the total number of sites in the protein sequence sample where a certain post-translational modification occurs;

[0078] S13, removing the protein sequence samples in the protein sequence set that do not belong to the legal character set D aa containing amino acid abbreviation characters, to obtain a first protein sequence dataset D protein ;

[0079] The protein sequence samples in the protein sequence set are represented by the letter abbreviations of the amino acids that constitute the sequence, and the set of twenty common amino acid abbreviations in vivo, G (glycine), A (alanine), V (valine), L (leucine), I (isoleucine), P (proline), F (phenylalanine), Y (tyrosine), W (tryptophan), S (serine), T (threonine), C (cysteine), M (methionine), N (asparagine), Q (glutamine), D (aspartic acid), E (glutamic acid), K (lysine), R (arginine), and H (histidine), are taken as the legal character set D aa ;

[0080] S14, removing redundancy from the protein sequence dataset D protein using CD-HIT to obtain a second protein sequence dataset D' protein In this embodiment, the de-redundancy parameters selected by CD-HIT are 0.3;

[0081] S15, setting a sliding window with a window radius of K' for each protein sequence sample in the second protein sequence dataset D' protein and cropping each protein sequence sample using the sliding window to obtain a plurality of protein sequence fragments each having a length of 2K'+1. In this embodiment, the sliding window radius is set to K' = 25, and the length of the protein sequence fragment is 51.

[0082] The step S15 includes the following steps:

[0083] S151, setting a sliding window with a window radius of K' for each protein sequence sample in the second protein sequence dataset D' protein ;

[0084] S152, sliding the sliding window on the corresponding protein sequence sample;

[0085] S153, selecting the sequence with a length of K' on both sides of the amino acid at the center point in the window for cropping to obtain a plurality of protein sequence fragments each having a length of 2K'+1.

[0086] S16. Based on the label information Label of each protein sequence sample in the protein sequence set, obtain the set of amino acid sites where post-translational modifications occur, and determine whether the amino acid at the center of each protein sequence fragment is in the set of amino acid sites where post-translational modifications occur. If so, place the corresponding protein sequence fragment in the positive sample dataset; otherwise, place the corresponding protein sequence fragment in the negative sample dataset.

[0087] S17. By randomly sampling, protein sequence fragments with the same number of protein sequence fragments as those in the positive sample dataset are obtained from the negative sample dataset, and the obtained protein sequence fragments are merged with the positive sample dataset to obtain the benchmark dataset D; In this embodiment, random sampling is used to balance the positive and negative sample data in the dataset.

[0088] S2, based on the legal character set D aa Obtain the first vocabulary list W 1 Second vocabulary list W 2 and the third vocabulary list W 3 ;

[0089] Step S2 includes the following steps:

[0090] S21. Construct the first vocabulary list W respectively. 1 Second vocabulary list W 2 and the third vocabulary list W 3 and the first vocabulary W 1 Second vocabulary list W 2 and the third vocabulary list W 3 All are initialized to an empty set;

[0091] S22, Based on the legal character set D aa The twenty amino acids in the text are used to form words at three scales: single amino acid, dipeptide, and tripeptide.

[0092] S23. Place words consisting of a single amino acid in the first vocabulary list W. 1 In China, its vocabulary size is 20;

[0093] S24. Place words consisting of dipeptides made up of two amino acids in the second vocabulary list W. 2 In China, its vocabulary size is 400;

[0094] S25. Place words composed of tripeptides (three amino acids) in the third vocabulary list W. 3 In China, its vocabulary size is 8000;

[0095] S3, based on the first vocabulary list W 1 Second vocabulary list W 2 and the third vocabulary list W 3respectively, to obtain single-amino acid scale data set protein dipeptide scale data set and tripeptide scale data set

[0096] The step S3 comprises the following steps:

[0097] S31, representing each protein sequence sample in the second protein sequence data set D' protein by amino acid composition to obtain an amino acid composition represented protein sequence sample Seq:

[0098] Seq=[aa1,aa2,…,aa i′ ,…,aa n ]

[0099] Seq∈D′ protein ,aa i′ ∈D aa

[0100] wherein aa i′ represents an amino acid at the i'th position in the amino acid composition represented protein sequence sample Seq, wherein i' = 1, 2, 3, …, n, and n represents the total number of amino acids in the amino acid composition represented protein sequence sample Seq;

[0101] S32, dividing the amino acid composition represented protein sequence sample Seq at single-amino acid, dipeptide and tripeptide scales respectively to obtain single-amino acid sequence Seq 1 , dipeptide sequence Seq 2 and tripeptide sequence Seq 3 composed of words in the first vocabulary W 1 , the second vocabulary W 2 and the third vocabulary W 3 respectively:

[0102] Seq 1 =[aa1,aa2,…,aa i′ ,…,aa n ]=Seq

[0103] Seq 2 =[(aa1aa2),(aa2aa3),…,(aa i′ aa i′+1 ),…,(aa n-1 aa n )]

[0104] Seq 3 ​= [(aa1aa2aa3), (aa2aa3aa4), …, (aa i′ aa i′+1 aa i′+2 ), …, (aa n-2 aa n-1 aa n )]

[0105] aa i′ ∈W 1 ,aa i′ aa i′+1 ∈W 2 ,aa i′ aa i′+1 aa i′+2 ∈W 3

[0106] wherein aa′ i′ represents an amino acid at the i′-th position in the single amino acid sequence Seq 1 represented by a word in the first vocabulary W 1 aa i′ aa i′+1 represents a dipeptide at the i′-th position in the dipeptide sequence Seq 2 represented by a word in the second vocabulary W 2 aa i′ aa i′+1 aa i′+2 represents a tripeptide at the i′-th position in the tripeptide sequence Seq 3 represented by a word in the third vocabulary W 3 wherein i′ = 1, 2, 3, …, n;

[0107] For each of the three sequences Seq 1 , Seq 2 and Seq 3 obtained after the above multi-scale partitioning, the length of each sequence is adjusted to n by padding, and the three sequences can form three new data sets composed of different scale vocabularies, respectively, a single amino acid scale data set a dipeptide scale data set and a tripeptide scale data set and the size of the three new data sets is equal to the size of the original data set |D protein |;

[0108] S33, sequentially partitioning the protein sequence samples Seq in the second protein sequence data set D′ protein in single amino acid, dipeptide and tripeptide scales, respectively, to obtain a single amino acid scale data set a dipeptide scale data set and tripeptide-scale dataset

[0109] S4. Using the Skip-gram model structure, respectively, the datasets at the single amino acid scale are processed. Dipeptide-scale dataset and tripeptide-scale dataset The corpus was used for training to obtain a first word vector model, a second word vector model, and a third word vector model; in this embodiment, the amino acid word vectors trained at the three scales each have a dimension of 100; the trained word vectors are then used in the first vocabulary W. 1 Second vocabulary list W 2 and the third vocabulary list W 3 The corresponding multi-scale amino acid words can only learn word vectors for multi-scale amino acid words that appear in the corpus. Other words that do not appear in the corpus are initialized to 100-dimensional 0 vectors by default. However, those default-initialized amino acid word vectors are not used in the model's amino acid word embedding. Therefore, such initialization in this embodiment will not affect the use of word vectors.

[0110] S5, based on the first vocabulary list W 1 Second vocabulary list W 2 and the third vocabulary list W 3 The first, second, and third word vector models were used to predict the labels of protein sequence fragments in the benchmark dataset D using a multi-head attention model, and the prediction results of the protein sequence fragments were obtained.

[0111] Step S5 includes the following steps:

[0112] S51. Divide each protein sequence fragment in the benchmark dataset D into three scales: single amino acid, dipeptide, and tripeptide.

[0113] S52, using the first vocabulary list W respectively 1 Second vocabulary list W 2 and the third vocabulary list W 3 The words in the text were used to reconstruct the protein sequence fragments after being divided at three scales, resulting in several amino acid sample fragment sequences. Several dipeptide sample fragment sequences and several tripeptide sample fragment sequences Where i″ represents the i″-th protein sequence fragment in the benchmark dataset;

[0114] The amino acid sample fragment sequence in this embodiment Dipeptide sample fragment sequence and tripeptide sample fragment sequence the dimensions of the three are 1x51, 1x50 and 1x49 respectively, and the three are padded into a sequence of 1x51, and then the first vocabulary W 1 , the second vocabulary W 2 and the third vocabulary W 3 are used to express the word vectors;

[0115] S53, converting each amino acid sample fragment sequence into an amino acid sample fragment word vector using a first word vector model, converting each dipeptide sample fragment sequence into a dipeptide sample fragment word vector using a second word vector model, and converting each tripeptide sample fragment sequence into a tripeptide sample fragment word vector using a third word vector model;

[0116] S54, constructing an amino acid sample fragment word vector matrix based on the amino acid sample fragment word vectors constructing a dipeptide sample fragment word vector matrix based on the dipeptide sample fragment word vectors constructing a tripeptide sample fragment word vector matrix based on the tripeptide sample fragment word vectors

[0117] S55, adding position encoding information to each amino acid sample fragment word vector, each dipeptide sample fragment word vector and each tripeptide sample fragment word vector using a rotation position encoding module RoPE, respectively, to obtain an amino acid sample fragment word vector matrix a dipeptide sample fragment word vector matrix and a tripeptide sample fragment word vector matrix after adding position encoding information; a dipeptide sample fragment word vector matrix and a tripeptide sample fragment word vector matrix The dimensions of the amino acid sample fragment word vector matrix

[0118] S56, inputting the amino acid sample fragment word vector matrix after adding position encoding information into a multi-head attention module Multi-Att of a single amino acid sequence channel, inputting the dipeptide sample fragment word vector matrix into a multi-head attention module Multi-Att of a dipeptide sequence channel, and inputting the tripeptide sample fragment word vector matrix into a multi-head attention module Multi-Att of a tripeptide sequence channel, and learning the word vector matrix features under three scales through the multi-head attention modules in each channel:

[0119]

[0120] wherein, represents the feature extracted from the i'th protein sequence fragment in the j channel, and the dimension is 1x51x100, wherein j=1 represents a single amino acid sequence channel, j=2 represents a dipeptide sequence channel, and j=3 represents a tripeptide sequence channel;

[0121] S57, the features extracted from the i'th protein sequence fragment by the single amino acid sequence channel, the features extracted from the i'th protein sequence fragment by the dipeptide sequence channel, and the features extracted from the i'th protein sequence fragment by the tripeptide sequence channel are spliced and input into an attention module Chanel-Att to learn the attention weight of the tripeptide sequence channel, and the i'th protein sequence fragment feature after channel attention weight adjustment is obtained:

[0122]

[0123] wherein O i″ represents the i'th protein sequence fragment feature after channel attention weight adjustment, and the dimension is 1x51x100;

[0124] S58, the i'th protein sequence fragment feature after channel attention weight adjustment is input into a fully connected layer to obtain a protein sequence fragment prediction result:

[0125] Label i″ = soft max(w·O i″ +b)

[0126] wherein Label i″ represents the predicted label of the i'th protein sequence fragment, and w and b represent the neuron weight matrix and bias of the last linear layer network, respectively, wherein when the i'th protein sequence fragment is a positive sample data, the predicted label is 1, and when the i'th protein sequence fragment is a negative sample data, the predicted label is 0.

[0127] In order to verify the superiority of the prediction accuracy of the protein post-translational modification of amino acid in the present application, the present application method and the method based on convolutional neural network (CNN) are compared. The experimental data set has 754 positive samples and an equal amount of randomly sampled positive samples, a total of 1508, and the test conditions are shown in Table 1:

[0128] Table 1

[0129]

[0130] From Table 1, the prediction performance of the present application method is improved by about 13% compared with the existing method, which has been greatly improved.

[0131] The application is divided into single amino acid, dipeptide and tripeptide three scales to obtain multi-scale word vectors from protein sequences, then after adding position information through rotation position coding, features are extracted from word vectors in three scales based on multi-head attention in three channels; then the weight of three channel features is adjusted through channel attention, finally the fused features of three channels are input into a linear layer to realize label prediction of the input sequence, so that the combination of amino acid, dipeptide and tripeptide multi-scale sequence composition information in the protein sequence is realized, and the prediction performance of post-translational modification of amino acid on the protein is improved.

[0132] The beneficial effects of the application are:

[0133] (1), the application extracts the multi-scale information of single amino acid, dipeptide and tripeptide in the protein sequence, extracts the features in the word vector under each scale through three channels, and outputs the fused features of multiple channels by combining the channel attention mechanism, thereby improving the prediction effect of post-translational modification of amino acid on the protein.

[0134] (2), the application constructs single amino acid, dipeptide and tripeptide data sets of three scales from the collected original protein sequence data set according to three scales, trains multi-scale word vectors, and only uses word vectors as the features of the original sequence, without relying on traditional biological feature extraction methods, thereby solving the dependence of the existing method on the biological feature extraction method when realizing the prediction of post-translational modification of amino acid on the protein.

[0135] Although the above describes the specific embodiments of the application for the purpose of facilitating the understanding of the application by those skilled in the art, it should be clear that the application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the application defined and determined by the appended claims, and all the application and creation utilizing the concept of the application are within the scope of protection.

Claims

1. A method for predicting post-translational modification of proteins based on multi-head attention, characterized in that, The method comprises the following steps: S1, preprocessing protein sequence with post-translational modification annotation information to obtain a legal character set , a second protein sequence data set , and a benchmark data set D ; S2, based on the legal character set obtaining a first vocabulary , a second vocabulary , and a third vocabulary ; S3, based on the first vocabulary , the second vocabulary , and the third vocabulary , respectively, to protein sequence samples in a second protein sequence dataset , to obtain a single-amino-acid-scale dataset , a dipeptide-scale dataset , and a tripeptide-scale dataset ; S4, training the corpus composed of single amino acid scale dataset, dipeptide scale dataset and tripeptide scale dataset respectively by using Skip-gram model structure to obtain first word vector model, second word vector model and third word vector model; , dipeptide scale dataset , and tripeptide scale dataset ; S5、based on the first vocabulary , the second vocabulary and the third vocabulary , respectively, using the first word vector model, the second word vector model and the third word vector model to perform label prediction of the multi-head attention model on the protein sequence fragments in the benchmark dataset D , to obtain protein sequence fragment prediction results; The step S1 comprises the following steps: S11, acquire a protein sequence set with post-translational modification annotation information; S12、define the label information of the protein sequence sample in the protein sequence set Label Expression: in, The annotation information of protein sequence samples is the first i At each of the following sites, some kind of post-translational modification occurred, among which, i =1, 2, ..., L , L This indicates the total number of sites in a protein sequence sample where a certain post-translational modification has occurred; S13, removing protein sequence samples in the protein sequence set that include characters not belonging to the legal character set S14, removing protein sequence samples in the protein sequence set that include characters not belonging to the legal character set ; S14, de-reducing the protein sequence dataset by CD-HIT de-reducing, resulting in a second protein sequence dataset ; S15, to the second protein sequence dataset each protein sequence sample is set a window radius of and each protein sequence sample is cropped by using the sliding window to obtain a plurality of protein sequence fragments with a length of 2 +1. S16, labeling information of each protein sequence sample in the protein sequence set Label obtaining a set of amino acid sites subjected to post-translational modification, and determining whether the amino acid at the center of each protein sequence fragment is in the set of amino acid sites subjected to post-translational modification, if yes, placing the corresponding protein sequence fragment in the positive sample dataset, otherwise, placing the corresponding protein sequence fragment in the negative sample dataset; S17, obtaining protein sequence fragments consistent with the number of protein sequence fragments in the positive sample data set from the negative sample data set by random sampling, and merging the obtained protein sequence fragments with the positive sample data set to obtain a benchmark data set D . 2.The multi-headed attention-based protein post-translational modification prediction method of claim 1, wherein, The protein sequence samples in the protein sequence set are represented by the letter abbreviations of the amino acids that make up the sequence, and the set of twenty common amino acid abbreviations characters G, A, V, L, I, P, F, Y, W, S, T, C, M, R, and H in organisms are taken as the legal character set . 3.The multi-headed attention-based protein post-translational modification prediction method of claim 2, wherein, The step S15 comprises the following steps: S151, setting a window radius for each protein sequence sample in the second protein sequence dataset a sliding window of​ S152, slide the sliding window on the corresponding protein sequence sample; S153, selecting sequences of length 2+1 on both sides of the amino acid at the center point in the window, and cutting to obtain a plurality of protein sequence fragments of length 2+1. 4.The multi-headed attention-based protein post-translational modification prediction method of claim 3, wherein, The step S2 comprises the following steps: S21, respectively constructing a first vocabulary , a second vocabulary , and a third vocabulary , and initializing the first vocabulary , the second vocabulary , and the third vocabulary to empty sets; S22, based on the legal character set of the twenty amino acids in the set of legal characters, at three scales: single amino acid, dipeptide, and tripeptide. S23, placing words consisting of single amino acids in the first vocabulary in the middle; S24, placing a word consisting of a dipeptide of two amino acids in the second vocabulary in the middle; S25, placing a tripeptide consisting of three amino acids into the third vocabulary in the middle. 5.The multi-headed attention-based protein post-translational modification prediction method of claim 4, wherein, The step S3 comprises the following steps: S31, representing each protein sequence sample in the second protein sequence dataset by amino acid composition, obtaining protein sequence samples represented by amino acid composition :​ wherein, represents a sample of a protein sequence represented by the amino acid composition at the th position in the = 1, 2, 3,..., n , n represents a sample of a protein sequence represented by the amino acid composition total number of amino acids; S32, Protein sequence samples represented by amino acid composition at three scales of single amino acid, dipeptide and tripeptide respectively The division is performed to obtain single amino acid sequences , dipeptide sequences and tripeptide sequences composed of words in the first vocabulary , the second vocabulary and the third vocabulary respectively: wherein, represents a single amino acid sequence represented by a word in a first vocabulary represents an amino acid at position in a sequence of represents a dipeptide sequence represented by a word in a second vocabulary represents a dipeptide at position in a sequence of represents a tripeptide sequence represented by a word in a third vocabulary represents a tripeptide at position in a sequence of = 1, 2, 3, …, n ;​​​ S33、sequentially performing single amino acid, dipeptide and tripeptide scale partitioning on the protein sequence samples in the second protein sequence dataset obtaining single amino acid scale dataset , dipeptide scale dataset and tripeptide scale dataset respectively.​ 6.The multi-headed attention-based protein post-translational modification prediction method of claim 5, wherein, The step S5 comprises the following steps: S51, the reference dataset is divided into three scales of single amino acid, dipeptide and tripeptide according to each protein sequence fragment in the reference dataset respectively. D S51, the reference dataset is divided into three scales of single amino acid, dipeptide and tripeptide according to each protein sequence fragment in the reference dataset respectively. S52, Using the first vocabulary list respectively Second vocabulary list and the third vocabulary list The words in the text were used to reconstruct the protein sequence fragments after being divided at three scales, resulting in several amino acid sample fragment sequences. Several dipeptide sample fragment sequences and several tripeptide sample fragment sequences ,in, Represents the first in the benchmark dataset A protein sequence fragment; S53, converting each amino acid sample segment sequence into an amino acid sample segment word vector using a first word vector model S54, converting each dipeptide sample segment sequence into a dipeptide sample segment word vector using a second word vector model S55, converting each tripeptide sample segment sequence into a tripeptide sample segment word vector using a third word vector model S56, converting each amino acid sample segment sequence into an amino acid sample segment word vector using a first word vector model S54, constructing an amino acid sample segment word vector matrix based on the amino acid sample segment word vectors constructing a dipeptide sample segment word vector matrix based on the dipeptide sample segment word vectors constructing a tripeptide sample segment word vector matrix based on the tripeptide sample segment word vectors ; S55, respectively adding position coding information to each amino acid sample segment word vector, each dipeptide sample segment word vector and each tripeptide sample segment word vector by using a rotation position coding module RoPE, to obtain an amino acid sample segment word vector matrix after adding position coding , a dipeptide sample segment word vector matrix , and a tripeptide sample segment word vector matrix ; S56, the amino acid sample segment word vector matrix after adding position coding In the multi-head attention module Multi-Att inputting a single amino acid sequence channel, the dipeptide sample segment word vector matrix is inputted In the multi-head attention module Multi-Att inputting a dipeptide sequence channel, the tripeptide sample segment word vector matrix is inputted In the multi-head attention module Multi-Att inputting a tripeptide sequence channel, and the word vector matrix features in three scales are learned through the multi-head attention module in each channel wherein, represents j the word vector matrix feature extracted for the protein sequence fragment in the channel, wherein, j= 1 represents a single amino acid sequence channel, j = 2 represents a dipeptide sequence channel, j = 3 represents a tripeptide sequence channel; S57, concatenate the word vector matrix features extracted by the single amino acid sequence channel for the first protein sequence segment, the word vector matrix features extracted by the dipeptide sequence channel for the first protein sequence segment, and the word vector matrix features extracted by the tripeptide sequence channel for the first protein sequence segment, and input the concatenation result into the attention module Chanel-Att to learn the attention weight of the tripeptide sequence channel, to obtain the first protein sequence segment feature after channel attention weight adjustment: wherein, represents the i-th channel attention weight adjusted protein sequence fragment feature; and represents the i-th channel attention weight adjusted protein sequence fragment feature; and S58, the first [section / item] after adjusting the channel attention weights. Each protein sequence fragment feature is input into a fully connected layer to obtain the protein sequence fragment prediction result: wherein, represents the predicted label of the th protein sequence fragment, and respectively represent the weight matrix and bias of the neurons of the last linear layer network, wherein the predicted label of the th protein sequence fragment is 1 when it is a positive sample data, and the predicted label of the th protein sequence fragment is 0 when it is a negative sample data.

Images