A method for protein generation based on position-specific weight matrix

Abstract

The application discloses a protein generation method based on a position-specific weight matrix, comprising the following steps: obtaining a training sample set; training a transformer model based on the training sample set through a cross-entropy loss function, simultaneously outputting an amino acid sequence probability distribution, obtaining a predicted amino acid sequence through top-k sampling, performing multiple sequence alignment on the predicted amino acid sequence by adopting a psi-blast method to obtain a position-specific weight matrix, performing information entropy calculation on the position-specific weight matrix to obtain a weight-specific matrix information value; constructing a total loss function through the weight-specific matrix information value and a cross loss function, updating model parameters based on the training sample set through the total loss function to obtain an amino acid sequence generation model; taking a leading sequence as input and sequentially passing through the amino acid sequence generation model to generate an amino acid sequence, and folding the amino acid sequence through a trRosetta model to obtain protein secondary and tertiary structures.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer-aided protein generation, and particularly relates to a protein generation method based on a position-specific weight matrix. BACKGROUND

[0002] Bioinformatics is a research hotspot in the intersection of life sciences and computer science. Bioinformatics research results have been widely used in gene discovery and prediction, gene data storage management, data retrieval and mining, gene expression data analysis, protein structure prediction, gene and protein homology relationship prediction, sequence analysis and alignment, etc. Genome defines all the proteins that constitute the organism, and gene defines the amino acid sequence that constitutes the protein. Although proteins are composed of linear sequences of amino acids, they only have corresponding activities and corresponding biological functions when they are folded into specific spatial structures.

[0003] Protein generation has always been one of the most concerned directions in the field of gene and protein characterization research, and almost all protein engineering involves modification of natural protein structures. Protein generation is of great significance to protein structure research, protein evolution, and disease treatment intervention. On the one hand, it can explore and verify the correspondence between protein structure and function; on the other hand, it can enrich the diversity of natural protein databases.

[0004] Understanding the spatial structure of proteins is not only beneficial to understanding the function of proteins, but also beneficial to understanding how proteins perform functions. It is very important to determine the structure of proteins. At present, the accumulation of protein sequence database data is very fast, but the number of known structures is relatively small. Although there has been significant progress in protein structure determination technology, the process of determining protein structure through experimental methods is still very complex and costly. Therefore, the number of experimentally determined protein structures is much less than the known protein sequences.

[0005] In the field of protein generation, the current mainstream methods can be divided into two categories: traditional statistical methods and machine learning methods. Traditional statistical methods include Markov probability models, etc., which use biological statistical methods and tools to find pattern information of protein sequences and use biophysical formulas for guidance, but cannot learn long-distance amino acid residue interaction relationships. The accuracy and confidence of the protein generated by the machine learning method exceed that of the traditional statistical method, mainly including long short-term memory network, adversarial neural network, and graph neural network, etc., and the features are obtained by using pre-trained language models.

[0006] The addition of GNN, transformer makes long-distance feature discovery possible, and the accuracy of protein generation is greatly increased. At present, ProGen published in 2020 is the best machine learning model in the field of protein generation, and the generated protein has been verified by machine learning and biological experiments. Its transformer architecture adopts attention mechanism and cancels the cycle structure of RNN, which can pay attention to the long-distance interaction relationship of protein sequences.

[0007] However, although the ProGen model has high accuracy, it also has some problems. The ProGen model is a transformer architecture, which relies on large-scale and large-parameter language models. The model has 1.2 billion parameters and has completed pre-training on a database of 2.8 million proteins. It relies heavily on machine learning knowledge and has weak interpretability. In the current machine learning method, the problem of ignoring biological pattern features, large parameters and high training cost is widespread, and the generalization and interpretability are also poor. SUMMARY

[0008] The present application provides a protein generation method based on position-specific weight matrix, which enhances the interpretability of the model by introducing position-specific weight matrix (PSSM).

[0009] A protein generation method based on position-specific weight matrix, comprising:

[0010] (1) obtaining a protein sequence set of a selected function as a sample set, and obtaining a training sample set from the sample set;

[0011] (2) training the transformer model based on the training sample set through the cross-entropy loss function, while outputting the amino acid sequence probability distribution, top-k sampling is performed on the amino acid sequence probability distribution to obtain the predicted amino acid sequence, and the psi-blast search method is used to perform multiple sequence alignment on the predicted amino acid sequence to obtain the position-specific weight matrix of the predicted amino acid sequence, and the information entropy of each row of multiple sequence alignment values in the position-specific weight matrix is calculated to obtain the information entropy value of different amino acid sequence lengths, and the information entropy of different amino acid sequence lengths is averaged to obtain the weight-specific matrix information value;

[0012] (3) constructing a total loss function, constructing a total loss function through the weight-specific matrix information value and the cross-entropy loss function, and training the transformer model based on the training sample set through the total loss function to update the model parameters and obtain an amino acid sequence generation model;

[0013] (4) When applied, the leading sequence is taken as input, and the amino acid sequence generation model and top-K sampling are sequentially passed through to complete the amino acid sequence generation, and the predicted protein secondary and tertiary structures are obtained by folding the generated amino acid sequence through the trRosetta model.

[0014] The protein sequence set of selected functions is obtained from the kaggle or UniPort development dataset according to the Go label.

[0015] The transformer model comprises an input embedding layer, an intermediate hidden layer and an output prediction layer connected in sequence, the intermediate hidden layer is composed of a plurality of transformer modules, each transformer module comprises a multi-head attention layer, a first Dropout layer, a first Add&Norm layer, a feedforward layer, a second Dropout layer and a second Add&Norm layer connected in sequence.

[0016] The position-specific weight matrix is obtained by performing multiple sequence alignment on the predicted amino acid sequence and the protein database using the psi-blast search method, including: performing multiple sequence alignment on the predicted amino acid sequence and the protein database using the psi-blast search method, constructing an initial position-specific weight matrix based on the alignment result, performing multiple sequence alignment on the initial position-specific weight matrix and the protein database again using the psi-blast search method, constructing a new position-specific weight matrix based on the alignment result, and continuously performing multiple sequence alignment until the alignment result does not change to obtain the position-specific weight matrix.

[0017] The position-specific weight matrix P is:

[0018]

[0019] Wherein, N is the length of the amino acid sequence, r i.j is the probability value of the jth amino acid at the ith position of the sequence, i.e. the sequence alignment value, i∈[1,N],j∈[1,20].

[0020] The information entropy of each row in the position-specific weight matrix is calculated to obtain the information entropy value P i of the ith amino acid sequence length:

[0021]

[0022] Wherein, r i.j is the probability value of the jth amino acid at the ith position of the sequence, i.e. the sequence alignment value, i∈[1,N],j∈[1,20].

[0023] The weight-specific matrix information value H is obtained by averaging the information entropy of different amino acid sequences.

[0024]

[0025] Wherein, P i is the information entropy value of the length of the i-th amino acid sequence, N is the length of the amino acid sequence, r i.j is the probability value of the j-th amino acid at the i-th position of the sequence, that is, the sequence alignment value, i [1, N], j [1, 20].

[0026] The total loss function S is:

[0027] S = lambda * L + (1 - lambda) * H

[0028]

[0029] Wherein, H is the weight-specific matrix information value, lambda is the adjustment parameter, y (m) is the prediction value of the m-th amino sequence, is the actual value of the m-th amino sequence, and M is the amino sequence index.

[0030] Compared with the prior art, the beneficial effects of the present application are:

[0031] The present application retains the advantages of the transformer model structure, and introduces a position-specific weight matrix based on the model. In the model training process, the protein is subjected to multiple sequence alignment in the natural protein database, similar sequences with the same module information are matched, and a position-specific weight matrix with sequence pattern probability distribution information is obtained. The trained model has more biological pattern information, and the model has stronger interpretability. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The flowchart of the protein generation method based on the position-specific weight matrix is provided for the specific embodiment;

[0033] Figure 2 The flowchart of the transformer model training is provided for the specific embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and technical effect of the present application clearer, the present application is further described in detail in combination with the drawings of the specification.

[0035] The present application provides a protein generation method based on a transformer model combined with a position-specific weight matrix, comprising:

[0036] S1: The application adopts Go label, obtains a set of protein sequences of a specific function to be generated from a development data set of kaggle or UniPort, and divides the set of protein sequences into a training sample set and a test sample set.

[0037] S2: The training sample set is taken as input, an initial transformer model is obtained by training a transformer model through a cross-entropy loss function, the training sample is input into the initial transformer model to obtain an amino acid sequence probability distribution, including probabilities on 20 basic amino acids and a sequence end symbol, and a predicted amino acid sequence is obtained by top-k sampling on the amino acid sequence probability distribution.

[0038] S3: An initial PSSM matrix is obtained by performing multiple sequence alignment on the predicted amino acid sequence and a protein database by using a psi-blast search method, the initial PSSM matrix and the protein database are searched again by using the psi-blast search method, that is, multiple sequence alignment is performed, the PSSM matrix is reconstructed by using the search result, and the above process is repeated until there is no new search result, that is, the PSSM matrix does not change, to obtain the PSSM matrix. The psi-blast search method can make the protein sequence with high conservation have a higher score, and can effectively find homologous proteins and distant homologues across species.

[0039] The specific steps for obtaining the initial PSSM matrix are as follows: a series of similar amino acid sequences are obtained by searching the predicted amino acid sequence by using the psi-blast search method, the series of similar amino acid sequences are taken as a protein family, and then an archive is constructed, and the matrix form of the archive is the initial PSSM.

[0040] The position-specific weight matrix P is:

[0041]

[0042] Wherein, N is the length of the amino acid sequence, r i.j is the probability value of the jth amino acid at the i th position of the sequence, that is, the sequence alignment value, i∈[1,N],j∈[1,20].

[0043] The information entropy of each row of the PSSM matrix is calculated to obtain the information entropy value P i of the i th amino acid sequence length as:

[0044]

[0045] The information entropy of different amino acid sequences is averaged to obtain the weight-specific matrix information value H as:

[0046]

[0047] The information entropy is used to quantify the uncertainty of the generated protein containing pattern information. The goal is to generate a protein with as much pattern information as the protein database, so it is necessary to maximize the information entropy to make it as certain as possible, so the weight-specific matrix information value is taken as the PSSM loss value.

[0048] The PSSM loss value is added to the cross-entropy loss function of the transformer model to construct the total loss function S as:

[0049] S=λ*L+(1-λ)*H

[0050]

[0051] Where H is the weight-specific matrix information value, λ is the adjustment parameter, y (m) is the mth amino acid sequence prediction value, is the mth amino acid sequence actual value, M is the amino acid sequence index. The final loss function adjusts the value of the PSSM loss value and the cross-entropy loss by λ, and the best value of λ is determined by the grid search method.

[0052] The transformer model is trained based on the training sample set to update the model parameters to obtain the amino acid sequence generation model.

[0053] The Adam function is used for parameter optimization. At the same time, the epoch number and batch size of the model training are adjusted according to the actual training situation, and drop out and early stopping are used to prevent model overfitting, with patience=10, and dropout is 0.3 during training.

[0054] S4: The leader sequence is first input into the amino acid sequence generation model to obtain the predicted amino acid sequence probability distribution based on the leader sequence, then the next amino acid of the leader sequence is obtained by acting on the predicted amino acid sequence probability distribution through top-K sampling, then the leader sequence and the next amino acid are input into the amino acid sequence generation model as a new sequence, and the above steps are repeated until the end token of a protein sequence to complete the generation of the amino acid sequence, and the highest scoring new amino acid sequence is obtained.

[0055] S5: The specific embodiments of the present application adopt the Sequence logo and trRosetta primary and secondary sequence verification methods to verify the confidence of the generated amino acid sequence through the test sample set. The generated amino acid sequence is used to complete the folding of the amino acid sequence by the trRosetta model to obtain the spatial stability and secondary and tertiary structure prediction of the generated sequence.

[0056] Among them, the transformer model provided by the present application includes a transformer encoder and a transformer decoder, and the specific structure is an input embedding layer, an intermediate hidden layer and an output prediction layer connected in turn, the intermediate hidden layer is composed of a plurality of transformer modules, each transformer module includes a multi-head attention layer, a first Dropout layer, a first Add&Norm layer, a feedforward layer, a second Dropout layer and a second Add&Norm layer connected in turn.

[0057] The calculation principle of the PSSM matrix provided by the present application is as follows:

[0058] There are sequence fragments with specific patterns in protein sequences, which are called sequence motifs. Sequence motifs are closely related to biological functions. For example, the N-glycosylation site motif always conforms to the following specific pattern: starting with Asn (aspartic acid), followed by any amino acid except Pro (proline), followed by Ser (serine) or Thr (threonine), followed by any amino acid except Pro.

[0059] Motifs are obtained from multiple sequence alignment results and are not adjacent on the amino acid sequence, but may be closely combined in three-dimensional structure. Traditional statistical methods and machine learning methods such as LSTM and GRU easily ignore these long-distance information. At the same time, two sequences may be very different in evolutionary relationship, but the three-dimensional space structure is consistent, which is easily ignored by LSTM and GRU models that ignore long-distance information.

[0060] The present application adopts PSI-BLAST (Position-Specific Iterated BLAST) for motif analysis. Unlike the BLAST query sequence-based search method, the profile-based search method is adopted. PSI-BLAST first starts from the query sequence, obtains a series of similar sequences as the same protein family, and then constructs a Profile. The matrix form of this profile is called a position-specific score matrix (PSSM). PSI-BLAST searches the database with a position-specific weight matrix each time, and then reconstructs the PSSM using the search results. Then the database is searched again with the new PSSM, and the process is repeated until no new results are generated. Psiblast can score highly conserved protein sequences and effectively find homologous proteins and distant homologues across species.

[0061] The present application adds a position-specific weight matrix in the machine learning model, that is, in the training process, the protein sequence is subjected to multiple sequence alignment in the natural protein database, similar sequences with the same module information are matched, and a PSSM matrix with sequence pattern probability distribution information is obtained. In this way, more biological pattern information can be included in the model, and the model is more interpretable.

Claims

1. A method for protein generation based on a position-specific weight matrix, characterized by, The method comprises the following steps: (1) obtaining a protein sequence set with a selected function as a sample set, and obtaining a training sample set from the sample set; (2) training a transformer model based on the training sample set through a cross-entropy loss function, simultaneously outputting an amino acid sequence probability distribution, performing top-k sampling on the amino acid sequence probability distribution to obtain a predicted amino acid sequence, performing multiple sequence alignment on the predicted amino acid sequence by using a psi-blast search method to obtain a position-specific weight matrix of the predicted amino acid sequence, calculating information entropy of a plurality of sequence alignment values in each row of the position-specific weight matrix to obtain information entropy values of different amino acid sequence lengths, and taking a mean value of the information entropy values of different amino acid sequence lengths to obtain a weight-specific matrix information value; (3) constructing a total loss function, constructing the total loss function through the weight-specific matrix information value and the cross-entropy loss function, training the transformer model based on the training sample set through the total loss function to update model parameters, and obtaining an amino acid sequence generation model; (4) in application, a leading sequence is taken as input, and amino acid sequence generation is completed by sequentially passing through the amino acid sequence generation model and top-K sampling, and the generated amino acid sequence is folded by a trRosetta model to obtain predicted protein secondary and tertiary structures.

2. The method of generating a protein based on a position-specific weight matrix according to claim 1, wherein, The protein sequence set with the selected function is obtained from kaggle or UniPort development data according to a Go label.

3. The method of generating a protein based on a position-specific weight matrix according to claim 1, wherein, The transformer model comprises an input embedding layer, an intermediate hidden layer and an output prediction layer connected in sequence, the intermediate hidden layer is composed of a plurality of transformer modules, each transformer module comprises a multi-head attention layer, a first Dropout layer, a first Add&Norm layer, a feedforward layer, a second Dropout layer and a second Add&Norm layer connected in sequence.

4. The method of generating proteins based on a position-specific weight matrix according to claim 1, wherein, The position-specific weight matrix is obtained by performing multiple sequence alignment on the predicted amino acid sequence and a protein database by using the psi-blast search method, comprising: performing multiple sequence alignment on the predicted amino acid sequence and the protein database by using the psi-blast search method, constructing an initial position-specific weight matrix based on the alignment result, performing multiple sequence alignment on the initial position-specific weight matrix and the protein database again by using the psi-blast search method, constructing a new position-specific weight matrix based on the alignment result, and continuously performing multiple sequence alignment until the alignment result does not change to obtain the position-specific weight matrix.

5. The method for generating a protein based on a position-specific weight matrix according to claim 1 or 4, wherein The position-specific weight matrix P is: where N is the length of the amino acid sequence, r i.j is the sequence alignment value of the jthamino acid at the ithposition of the sequence, i∈[1, N], j∈[1, 20].

6. The method of generating a protein based on a position-specific weight matrix according to claim 1, wherein, The information entropy value P of the length of the i-th amino acid sequence is calculated by information entropy calculation on a plurality of sequence alignment values of each row in the position-specific weight matrix i is: wherein r i.j is the sequence alignment value of the jthamino acid at the ithposition of the sequence, i ∈ [1, N], j ∈ [1, 20].

7. The method of claim 1, wherein the position-specific weight matrix is generated based on a protein sequence alignment of a plurality of proteins. The weight-specific matrix information value H is obtained by taking a mean value of information entropy of different amino acid sequences: wherein P i is the information entropy value of the length of the i-th amino acid sequence, N is the length of the amino acid sequence, r i.j is the sequence alignment value of the j-th amino acid at the i-th position of the sequence, i∈[1, N], j∈[1, 20].

8. The method of generating proteins based on a position-specific weight matrix according to claim 1, wherein, The total loss function S is: S = λ * L + (1 - λ) * H where H is the weight-specific matrix information value, λ is the adjustment parameter, y (m) is the predicted value of the mth amino sequence, is the actual value of the mth amino sequence, and M is the amino sequence index.

Images