A method for predicting protein secondary structure based on single-amino-acid sequence characteristics

Abstract

The application discloses a protein secondary structure prediction method based on single-amino acid sequence characteristics, and belongs to the field of bioinformatics. The application solves the problem that the prediction effect of a model obtained by using an existing method is poor due to the limitation of evolutionary information feature acquisition and the fact that the model ignores the interdependent relationship between secondary structure labels in an amino acid sequence. In the feature processing stage, the application fuses feature representations generated by a protein language model and features obtained by using a hybrid coding mode of word embedding and position embedding. The fused features contain rich evolutionary information and fully acquire position information of amino acid residues in the sequence and the dependent relationship between the amino acid residues. The fused features are sequentially output through a Transformer encoder, a CNN and a BiLSTM to output global feature information, and the interaction between secondary structures of proteins in the sequence is considered by introducing a CRF, so that the protein secondary structure prediction accuracy is ensured. The method of the application can be applied to protein secondary structure prediction.

Description

Technical Field

[0001] This invention belongs to the field of bioinformatics, specifically relating to a method for predicting protein secondary structure based on single amino acid sequence features. Background Technology

[0002] Proteins, as the primary executors of various functions in organisms, have structures that determine their function and stability. Secondary structure, acting as a bridge between primary and tertiary structures, is a crucial step in predicting protein tertiary structure and an indispensable part of understanding the relationship between protein structure and function. The Protein Secondary Structure Prediction (PSSP) task describes the local conformation of proteins by assigning secondary structure tags, thereby classifying each amino acid in the protein sequence. Based on different hydrogen bonding modes, secondary structures are generally classified into three basic types: H (helix), E (sheet), and C (irregular coil). Further subdivision yields eight categories of protein secondary structures: G (3-helix), H (α-helix), I (π-helix), E (β-sheet), B (β-bridge), S (bend), T (turn), and C (coil).

[0003] Initially, traditional machine learning methods such as Support Vector Machines, neural networks, and k-nearest neighbors were widely used in protein secondary structure prediction (PSSP). However, these early models were insufficient to meet practical needs. With the rapid development and improvement of deep learning technology, many deep learning-based methods have been introduced into the field of protein secondary structure prediction. In recent years, research on protein secondary structure prediction has two main directions: one is to use the amino acid sequence as the sole input to the prediction task, without using additional evolutionary information such as multiple sequence alignment (MSA), position-specific scoring matrices (PSSM), and hidden Markov models (HMM); the other focuses on improving the accuracy of predicting the secondary structure classification probability of each amino acid by optimizing and integrating different deep learning architectures.

[0004] Because proteins exhibit homology, prediction methods incorporating evolutionary information can fully leverage the structural similarities between homologous proteins in nature, thereby significantly improving the accuracy of protein structure prediction. However, considering only the amino acid sequence and completely disregarding evolutionary information yields limited results. To overcome this challenge, many methods attempt to utilize pre-trained language models to acquire evolutionary information about amino acid sequences. Some methods attempt to use natural language models to assist prediction, but this direct transfer of natural language models to the field of structure prediction fails to effectively capture protein features. Furthermore, many methods, in exploring lighter and more efficient prediction models, often focus on improving and fusing multiple deep learning models to accurately obtain the probability of secondary structure tags corresponding to each amino acid, and then using SoftMax layers to assign the corresponding three-state or eight-state secondary structure prediction results. However, the outputs of the SoftMax layer are independent, so the model cannot learn the interdependencies between secondary structure tags in the amino acid sequence, which limits the improvement of prediction accuracy to some extent.

[0005] In summary, although existing methods have achieved certain successes in the field of protein secondary structure prediction, limitations remain in the acquisition of evolutionary information features, and the models ignore the interdependencies between secondary structure tags in the amino acid sequence, resulting in poor prediction performance using existing methods. Therefore, developing a new technology to address these issues is of significant research and application value for protein secondary structure prediction. Summary of the Invention

[0006] The purpose of this invention is to address the problem that existing methods result in poor prediction performance due to limitations in obtaining evolutionary information features and the neglect of interdependencies between secondary structure tags in amino acid sequences. Therefore, this invention proposes a protein secondary structure prediction method based on single amino acid sequence features.

[0007] The technical solution adopted by this invention to solve the above-mentioned technical problems is: a method for predicting protein secondary structure based on single amino acid sequence features, the method specifically including the following steps:

[0008] Step S1: Collect protein secondary structure data files, process the collected protein secondary structure data files, and then obtain the amino acid sequence and the secondary structure tag of each amino acid in the processed protein secondary structure data files.

[0009] The obtained amino acid sequences and the secondary structure tags of each amino acid in the sequences are used as the training sample set;

[0010] Step S2: Use the amino acid sequences in the training sample set as input to the protein language model, and generate feature representations for each amino acid sequence through the protein language model.

[0011] Sine and cosine position encoding is performed on each amino acid sequence in the training sample set to obtain the position encoding result. Then, the position encoding result is concatenated with the word embedding matrix to obtain the concatenated feature corresponding to each amino acid sequence.

[0012] The spliced ​​features are then fused with the feature representations output by the protein language model to obtain the fused feature representations for each amino acid sequence.

[0013] Step S3: Use the fused feature representation as input to the Transformer model, and output the enhanced features through the encoder of the Transformer model;

[0014] Step S4: Use the enhanced features from step S3 as input to the CNN model, output the local features of each extracted amino acid sequence through the CNN model, and then use the local features as input to the BiLSTM model, output the global features of each amino acid sequence through the BiLSTM model.

[0015] Step S5: Based on the global features in step S4, obtain the protein secondary structure prediction results for each amino acid sequence in the training sample set. Calculate the loss based on the protein secondary structure prediction results and the secondary structure labels of the training sample set. Stop training when the loss function converges to obtain the trained protein language model, Transformer model, CNN model, and BiLSTM model.

[0016] Step S6: Use the amino acid sequence to be predicted as the input of the protein language model, and use the method in step S2 to obtain the fused feature representation of the amino acid sequence to be predicted, and use the fused feature representation as the input of the trained Transformer model.

[0017] The features output by the encoder of the Transformer model are then passed sequentially through the trained CNN model and BiLSTM model, and the protein secondary structure prediction results for the amino acid sequence to be predicted are obtained based on the features output by the BiLSTM model.

[0018] The beneficial effects of this invention are:

[0019] To extract rich feature information from amino acid sequences, this invention integrates two feature representations in the feature processing stage. First, it utilizes a protein language model to generate feature representations rich in evolutionary information, which can replace additional evolutionary information feature inputs and contribute to improved prediction accuracy. Second, it employs a hybrid encoding method combining word embedding and position embedding to fully capture the positional information of amino acid residues in the sequence and their dependencies. This feature processing approach helps improve the model's prediction performance and can also provide empirical support and rich reference for other related research in the field of proteins.

[0020] This invention utilizes a Transformer encoder to enhance the feature representation obtained in the feature processing stage and further captures the long-range dependencies between amino acid residues in the sequence. Then, a multi-layer CNN architecture is used to fully acquire local features of the amino acid sequence, and the bidirectional information integration capability of BiLSTM is used to fully capture global feature information of the amino acid sequence. After sufficient feature extraction, CRF is introduced as the core method in the prediction stage. This approach can take into account the interactions between protein secondary structures in the sequence, improving the prediction accuracy of protein secondary structures and obtaining the optimal secondary structure sequence.

[0021] The model provided by this invention exhibits good generalization performance and can be used to predict the secondary structure sequences of proteins. It can also provide excellent preliminary support for subsequent biological tasks such as predicting tertiary protein structures and drug development. Furthermore, the methods proposed in the feature processing and feature extraction sections of this invention have a certain degree of versatility and can be applied in a wider range of fields. Attached Figure Description

[0022] Figure 1 This is a flowchart of a protein secondary structure prediction method based on single amino acid sequence features according to the present invention;

[0023] Figure 2 This is a flowchart of the protein language model generating feature representations;

[0024] Figure 3 This is a flowchart of the CRF process;

[0025] Figure 4a This is a comparison chart of the accuracy scores of three-state predictions for different sequence feature encoding sizes;

[0026] Figure 4b This is a comparison chart of the Sov scores for three-state predictions based on the feature encoding sizes of different sequences;

[0027] Figure 4c This is a comparison chart of the eight-state prediction accuracy scores for different sequence feature encoding sizes;

[0028] Figure 4dThis is a comparison chart of the Sov scores for octet predictions of different sequence feature encoding sizes;

[0029] Figure 5a This is a graph showing the accuracy score of the three-state prediction under different conditions of the number of encoder blocks and attention heads in the Transformer model.

[0030] Figure 5b It is the Sov score map of the three-state prediction under the condition that the number of encoder blocks and attention heads of the Transformer model are different;

[0031] Figure 5c This is a graph showing the accuracy score of eight-state prediction under different conditions of the number of encoder blocks and attention heads in the Transformer model.

[0032] Figure 5d The Sov score map of the eight-state prediction under the condition that the number of encoder blocks and attention heads of the Transformer model are different;

[0033] Figure 6 This is a graph showing the impact of the number of convolutional layers in a CNN on prediction performance.

[0034] Figure 7 This is a graph showing the impact of the number of hidden layers in a BiLSTM model on its prediction performance. Detailed Implementation

[0035] Specific implementation method one: Combining Figure 1 This embodiment describes a method for predicting protein secondary structure based on single amino acid sequence features. The method specifically includes the following steps:

[0036] Step S1: Collect protein secondary structure data files, process the collected protein secondary structure data files, and then obtain the amino acid sequence and the secondary structure tag of each amino acid in the processed protein secondary structure data files.

[0037] The obtained amino acid sequences and the secondary structure tags of each amino acid in the sequences are used as the training sample set;

[0038] Step S2: Use the amino acid sequences in the training sample set as input to the protein language model, and generate feature representations for each amino acid sequence through the protein language model.

[0039] Sine and cosine position encoding is performed on each amino acid sequence in the training sample set to obtain the position encoding result. Then, the position encoding result is concatenated with the word embedding matrix to obtain the concatenated feature corresponding to each amino acid sequence.

[0040] The spliced ​​features are then fused with the feature representations output by the protein language model to obtain the fused feature representations for each amino acid sequence.

[0041] Step S3: Use the fused feature representation as input to the Transformer model, and output the enhanced features through the encoder of the Transformer model; the enhanced features can help the model obtain the long-range dependencies of the sequence.

[0042] Step S4: Use the enhanced features from step S3 as input to the CNN model, output the local features of each extracted amino acid sequence through the CNN model, and then use the local features as input to the BiLSTM model, output the global features of each amino acid sequence through the BiLSTM model.

[0043] Step S5: Based on the global features in step S4, obtain the protein secondary structure prediction results for each amino acid sequence in the training sample set. Calculate the loss based on the protein secondary structure prediction results and the secondary structure labels of the training sample set. Stop training when the loss function converges to obtain the trained protein language model, Transformer model, CNN model, and BiLSTM model.

[0044] Step S6: Use the amino acid sequence to be predicted as the input of the protein language model, and use the method in step S2 to obtain the fused feature representation of the amino acid sequence to be predicted, and use the fused feature representation as the input of the trained Transformer model.

[0045] The features output by the encoder of the Transformer model are then passed sequentially through the trained CNN model and BiLSTM model, and the protein secondary structure prediction results for the amino acid sequence to be predicted are obtained based on the features output by the BiLSTM model.

[0046] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the specific process of step S1 is as follows:

[0047] Step S11: Obtain the protein secondary structure data file from the NetSurfP-2.0 dataset, and then perform redundancy removal processing on the obtained protein secondary structure data file to obtain the processed protein secondary structure data file.

[0048] Step S12: Obtain from the processed data file a FASTA file containing the amino acid sequence of the protein, a JSON file containing the amino acid sequence of the protein, and secondary structure tags for each amino acid in the sequence.

[0049] The other steps and parameters are the same as in Specific Implementation Method 1.

[0050] The NetSurfP-2.0 dataset contains structural information for 10,792 proteins, including their primary amino acid sequences and corresponding tri- and octave-state secondary structure tag sequences. This structural information was derived from X-ray crystallography, a process with a resolution of at least [resolution missing]. Furthermore, it is necessary to ensure that there are no chain breaks. When it is necessary to predict the three-state secondary structure of a protein, the model is trained based on the three-state secondary structure tag sequence; when it is necessary to predict the eight-state secondary structure of a protein, the model is trained based on the eight-state secondary structure tag sequence.

[0051] Specific Implementation Method 3: This implementation method differs from Specific Implementation Method 1 or 2 in that the protein language model is a finely tuned ProtT5-XL-Half_Uniref50-Enc model.

[0052] Other steps and parameters are the same as in specific implementation method one or two.

[0053] Specific implementation method four: Combination Figure 2 This embodiment is described below. The difference between this embodiment and one of the specific embodiments one to three is that the protein language model includes N attention layers, and each attention layer has multi-head attention.

[0054] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0055] The ProtT5-XL-Half_Uniref50-Enc model is based on the T5 architecture. In this invention, N is 24, each attention layer has 32 attention heads, and it has been trained and fine-tuned on more than 45 million proteins provided by the BFD dataset and the UniRef50 dataset.

[0056] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the specific process of step S2 is as follows:

[0057] Step S21: For any amino acid sequence S, denote the number of amino acids contained in the amino acid sequence S as L. Process the amino acid sequence S through a protein language model to generate a 1024-dimensional embedding for each amino acid in the amino acid sequence S (i.e., the output of the protein language model). Concatenate the embeddings generated for L amino acids to obtain the protein embedding of the amino acid sequence S, which is the feature representation of the amino acid sequence S. The dimension of the feature representation of the amino acid sequence S is L×1024.

[0058] Similarly, the feature representation of each amino acid sequence is obtained; the generated protein embedding contains evolutionary information, which can be used to assist the model in completing the prediction task.

[0059] Step S22: Perform sine and cosine position encoding on the amino acid sequence S. Using sine and cosine position encoding can effectively reflect the positional information of each amino acid in the sequence, thereby helping the model capture the long-range dependencies of the amino acid sequence.

[0060]

[0061] Where pos represents the amino acid position in amino acid sequence S, PE (pos,2i) It is the 2ith element in the position encoding vector corresponding to the amino acid at position pos, PE (pos,2i+1) It is the (2i+1)th element in the position encoding vector corresponding to the amino acid at position pos, where 2i represents an even number and 2i+1 represents an odd number, d model This represents the dimension of the word vector; the dimension of the word vector is 512.

[0062] Step S23: Encode the amino acid at position pos into a word embedding representation, then concatenate the position encoding vector corresponding to the amino acid at position pos with the word embedding representation corresponding to the amino acid at position pos (i.e., add the elements at corresponding positions in both), resulting in a concatenated feature with a dimension of 1×512; for example... Figure 4a , Figure 4b , Figure 4c and Figure 4d As shown, the prediction performance is best when the feature size is 512;

[0063] Then, based on all the amino acids in the amino acid sequence S, the spliced ​​features with a dimension of L×512 are obtained;

[0064] Step S24: The protein embedding corresponding to amino acid sequence S and the spliced ​​feature corresponding to amino acid sequence S are fused to obtain the fused feature representation corresponding to amino acid sequence S. That is, the protein embedding with a size of L×1024 and the spliced ​​feature with a size of L×512 are fused to form a feature representation with a size of L×1536.

[0065] Similarly, the fusion-enhanced feature representations for each amino acid sequence are obtained.

[0066] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0067] The fused feature representation contains both evolutionary information and amino acid position information.

[0068] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the encoder of the Transformer model includes M encoder blocks, and each encoder block includes multi-head attention, a feed-forward network, and add&norm.

[0069] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0070] The multi-head attention mechanism within the encoder block consists of eight heads, which are computed sequentially for deeper encoding. Each head of attention is calculated using the same method, as detailed below:

[0071]

[0072] Among them, h t For the input data, Q is the query vector, K is the key vector, V is the value vector, and W is the input vector. Q W K and W V All are weighting coefficients.

[0073] Enhancing features through an encoder can yield richer and more comprehensive sequence information. For example... Figure 5a , Figure 5b , Figure 5c and Figure 5d As shown, the optimal number of Encoderblocks and the number of multi-head attention heads can be selected by observing the performance and efficiency of the model under different parameter configurations.

[0074] In the first stage of deep feature extraction, Convolutional Neural Networks (CNNs) leverage their powerful ability to capture local features. Multi-layer CNN architectures, by stacking multiple convolutional layers, can abstract more complex and abstract protein features layer by layer, such as... Figure 6 As shown, the appropriate number of convolutional layers can be determined by comparing the prediction performance of CNNs with different numbers of convolutional layers.

[0075] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One to Six in that the CNN model sequentially includes a first one-dimensional convolutional layer, a first ReLU activation function layer, a first Dropout layer, a second one-dimensional convolutional layer, a second ReLU activation function layer, a second Dropout layer, a third one-dimensional convolutional layer, a third ReLU activation function layer, and a third Dropout layer.

[0076] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0077] Y l =f(w*X+b)

[0078] Y l =f(w*Y l-1 +b)

[0079] Among them, Y l Y represents the output of each convolutional layer, f represents the activation function, and Y represents the output of each convolutional layer. l-1 This represents the output of the previous convolutional layer, X represents the input element, w represents the convolutional kernel, and b is the bias term, which adds a constant offset to each convolutional kernel.

[0080] Specific Implementation Method Eight: This implementation method differs from one of the specific implementation methods one to seven in that the BiLSTM model consists of a forward LSTM and a backward LSTM.

[0081] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0082] The BiLSTM architecture is used to capture long-range dependencies in amino acid sequences. Long Short-Term Memory (LSTM) networks contain three gating mechanisms: an input gate, a forget gate, and an output gate. These gating units work together to update and preserve sequence information. The forward LSTM operation is as follows:

[0083] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0084] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0085] C t =f t ⊙C t-1 +i t ⊙tanh(W c ·[h t-1 ,x t ]+b c )

[0086] o t =σ(W o @[h t-1 ,x t ]+b o )

[0087] h t =o t ⊙tanh(C t )

[0088] In this formula, the first formula represents the calculation process of the forget gate, the second and third formulas represent the calculation process of the input gate, and the fourth and fifth formulas represent the calculation process of the output gate. t It is the activation value of the output gate, h. t This represents the hidden state that is finally updated after passing through the gating unit.

[0089] In the BiLSTM architecture, the forward LSTM processes the input in the natural order of time steps, while the backward LSTM processes the same input in reverse order. Thus, the final hidden state at each time step is actually determined by the information from all the time steps before and after it. This integration of bidirectional information flow allows BiLSTM to fully capture the long-range dependencies of amino acid sequence features during the feature extraction stage. For example... Figure 7 As shown, the optimal number of hidden layers is determined by comparing the prediction performance of the BiLSTM model with different numbers of hidden layers.

[0090] Specific Implementation Method Nine: Combining Figure 3 This embodiment is described below. The difference between this embodiment and specific embodiments one through eight is that in step S5, based on the global features of step S4, the protein secondary structure prediction result for each amino acid sequence in the training sample set is obtained, specifically as follows:

[0091] Step S51: The global features obtained in step S4 are linearly transformed through a fully connected layer to make the dimension of the transformed features consistent with the dimension of the label. Each element in the linearly transformed features is used as the emission score of each category in the secondary structure label.

[0092] For example, when predicting three-state secondary structures, the FNN (fully connected layer) maps the feature vector to a probability distribution with a value of 3, also known as the emission score (state score), which represents the probability of predicting each amino acid as a certain three-state secondary structure.

[0093] Step S52: CRF learns the probability transition matrix between secondary structure labels through training. The elements in the probability transition matrix represent transition scores.

[0094] Step S53: Calculate the probability P(Y|X) that the predicted result of amino acid sequence X is secondary structure sequence Y based on the emission fraction and transfer fraction;

[0095] After obtaining the emission and transfer scores, the probability of a particular amino acid being labeled as a specific secondary structure tag can be calculated. Given the amino acid sequence X = (x1, x2, ..., x...). T ) and the secondary structure label sequence Y = (y1, y2, ..., y T The calculation process is as follows:

[0096]

[0097] Among them, f k (y t ,y t-1 ,X,t) is the k-th characteristic function, used to describe the dependencies between secondary structure tags and between secondary structure tags and amino acid residues. θ k It is the characteristic function f k The weights are learned from the training data. Z(X) is the partition function, used for normalization to ensure that the sum of the probabilities of all possible label sequences is 1. The specific formula is as follows:

[0098]

[0099] Where Q is the set of all possible secondary structure sequences, and Y... ′ ∈Q;

[0100] Step S54: Calculate the final secondary structure prediction result Y using the Viterbi algorithm and probability P(Y|x). * .

[0101] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0102] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that the specific process of step S54 is as follows:

[0103]

[0104] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.

[0105] The Viterbi algorithm recursively calculates the state probability at each time step and selects the path with the highest cumulative probability at each step. Finally, it backtracks to obtain the optimal state sequence of the entire sequence, that is, finds the path with the maximum score. This algorithm can greatly improve the computational efficiency.

[0106] Example

[0107] This embodiment proposes a method for predicting protein secondary structure based on single amino acid sequence features, specifically as follows:

[0108] Step S1: Collect protein secondary structure data files from the NetSurfP-2.0 dataset. The NetSurfP-2.0 dataset contains structural information for 10792 proteins, covering their primary amino acid sequences and corresponding tri- and octave-state secondary structure tag sequences. This protein structural information is derived from X-ray crystallography, and the resolution in this process is at least [resolution missing]. And it is necessary to ensure that there is no chain breakage.

[0109] The collected protein secondary structure data files are processed to obtain the amino acid sequences and secondary structure tags for each amino acid in the processed protein secondary structure data files. The obtained amino acid sequences and secondary structure tags for each amino acid in the sequences are used as training sample sets. It should be noted that when it is necessary to predict the three-state secondary structure of a protein, the model is trained based on the three-state secondary structure tag sequences, and when it is necessary to predict the eight-state secondary structure of a protein, the model is trained based on the eight-state secondary structure tag sequences.

[0110] Step S2: The amino acid sequences in the training sample set are used as input to the fine-tuned protein language model ProtT5-XL-Half_Uniref50-Enc. The ProtT5-XL-Half_Uniref50-Enc model is based on the T5 architecture, including 24 attention layers, each with 32 attention heads, and has been trained and fine-tuned on more than 45 million proteins provided by the BFD dataset and the UniRef50 dataset. The protein language model generates feature representations for each amino acid sequence.

[0111] Sine and cosine position encoding is performed on each amino acid sequence in the training sample set to obtain the position encoding result. Then, the position encoding result is concatenated with the word embedding matrix to obtain the concatenated feature corresponding to each amino acid sequence.

[0112] The spliced ​​features are then fused with the feature representations output by the protein language model to obtain the fused feature representations for each amino acid sequence.

[0113] The following example, using any amino acid sequence S, illustrates the specific process of step S2:

[0114] Step S21: Let L be the number of amino acids in amino acid sequence S. Process amino acid sequence S through a protein language model to generate a 1024-dimensional embedding for each amino acid in amino acid sequence S (i.e., the output of the protein language model). Concatenate the embeddings generated for L amino acids to obtain the protein embedding of amino acid sequence S, that is, to obtain the feature representation of amino acid sequence S, and the dimension of the feature representation of amino acid sequence S is L×1024.

[0115] The generated protein embeddings contain evolutionary information, which can be used to assist the model in completing prediction tasks;

[0116] Step S22: Perform sine and cosine position encoding on the amino acid sequence S. Using sine and cosine position encoding can effectively reflect the positional information of each amino acid in the sequence, thereby helping the model capture the long-range dependencies of the amino acid sequence.

[0117]

[0118] Where pos represents the amino acid position in amino acid sequence S, PE (pos,2i) It is the 2ith element in the position encoding vector corresponding to the amino acid at position pos, PE (pos,2i+1) It is the (2i+1)th element in the position encoding vector corresponding to the amino acid at position pos, where 2i represents an even number and 2i+1 represents an odd number, d model This represents the dimension of the word vector; the dimension of the word vector is 512.

[0119] Step S23: Encode the amino acid at position pos into a word embedding representation, then concatenate the position encoding vector corresponding to the amino acid at position pos with the word embedding representation corresponding to the amino acid at position pos (i.e., add the elements at corresponding positions in both), resulting in a concatenated feature with a dimension of 1×512; for example... Figure 4a , Figure 4b , Figure 4c and Figure 4d As shown, the prediction performance is best when the feature size is 512;

[0120] Then, based on all the amino acids in the amino acid sequence S, the spliced ​​features with a dimension of L×512 are obtained;

[0121] Step S24: The protein embedding corresponding to amino acid sequence S and the spliced ​​feature corresponding to amino acid sequence S are fused to obtain the fused feature representation corresponding to amino acid sequence S. That is, the protein embedding with a size of L×1024 and the spliced ​​feature with a size of L×512 are fused to form a feature representation with a size of L×1536.

[0122] Step S3: The fused feature representation is used as input to the Transformer model. The Transformer model's encoder includes six encoder blocks, each containing an 8-head attention (multi-head attention), a feed-forward network, and an add&norm. The 8-head attention is computed sequentially for deeper encoding.

[0123] The calculation method for attention is the same for all heads, and the specific calculation method is as follows:

[0124]

[0125] Among them, h t For the input data, Q is the query vector, K is the key vector, V is the value vector, and W is the input vector. Q W K and W V All are weighting coefficients.

[0126] Enhancing features through an encoder can yield richer and more comprehensive sequence information. For example... Figure 5a , Figure 5b , Figure 5c and Figure 5d As shown, the optimal number of Encoderblocks and the number of multi-head attention heads can be selected by observing the performance and efficiency of the model under different parameter configurations.

[0127] In the first stage of deep feature extraction, Convolutional Neural Networks (CNNs) leverage their powerful ability to capture local features. Multi-layer CNN architectures, by stacking multiple convolutional layers, can abstract more complex and abstract protein features layer by layer, such as... Figure 6 As shown, the appropriate number of convolutional layers can be determined by comparing the prediction performance of CNNs with different numbers of convolutional layers.

[0128] The encoder of the Transformer model outputs enhanced features; these enhanced features can help the model obtain long-range dependencies of sequences.

[0129] Step S4: Use the enhanced features from step S3 as input to the CNN model, output the local features of each extracted amino acid sequence through the CNN model, and then use the local features as input to the BiLSTM model, output the global features of each amino acid sequence through the BiLSTM model.

[0130] The CNN model consists of the following layers in sequence: the first one-dimensional convolutional layer, the first ReLU activation function layer, the first Dropout layer, the second one-dimensional convolutional layer, the second ReLU activation function layer, the second Dropout layer, the third one-dimensional convolutional layer, the third ReLU activation function layer, and the third Dropout layer.

[0131] Y l =f(w*X+b)

[0132] Y l =f(w*Y l-1 +b)

[0133] Among them, Y l Y represents the output of each convolutional layer, f represents the activation function, and Y represents the output of each convolutional layer. l-1 This represents the output of the previous convolutional layer, X represents the input element, w represents the convolutional kernel, and b is the bias term, which adds a constant offset to each convolutional kernel.

[0134] The BiLSTM model consists of a forward LSTM and a backward LSTM. The BiLSTM architecture is used to capture long-range dependencies in amino acid sequences. The Long Short-Term Memory (LSTM) network contains three gating mechanisms: an input gate, a forget gate, and an output gate. These gating units work together to update and preserve sequence information. The forward LSTM works as follows:

[0135] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0136] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0137] C t =f t ⊙C t-1 +i t ⊙tanh(W c ·[h t-1 ,x t ]+b c )

[0138] o t =σ(W O ·[h t-1 ,x t ]+b o )

[0139] h t =o t ⊙tanh(C t )

[0140] In this formula, the first formula represents the calculation process of the forget gate, the second and third formulas represent the calculation process of the input gate, and the fourth and fifth formulas represent the calculation process of the output gate. t It is the activation value of the output gate, h. t This represents the hidden state that is finally updated after passing through the gating unit.

[0141] In the BiLSTM architecture, the forward LSTM processes the input in the natural order of time steps, while the backward LSTM processes the same input in reverse order. Thus, the final hidden state at each time step is actually determined by the information from all the time steps before and after it. This integration of bidirectional information flow allows BiLSTM to fully capture the long-range dependencies of amino acid sequence features during the feature extraction stage. For example... Figure 7 As shown, the optimal number of hidden layers is determined by comparing the prediction performance of the BiLSTM model with different numbers of hidden layers.

[0142] Step S5: Based on the global features from step S4, obtain the protein secondary structure prediction results for each amino acid sequence in the training sample set; specifically:

[0143] Step S51: The global features obtained in step S4 are linearly transformed through a fully connected layer to make the dimension of the transformed features consistent with the dimension of the label. Each element in the linearly transformed features is used as the emission score of each category in the secondary structure label.

[0144] For example, when predicting three-state secondary structures, the FNN (fully connected layer) maps the feature vector to a probability distribution with a value of 3, also known as the emission score (state score), which represents the probability of predicting each amino acid as a certain three-state secondary structure.

[0145] Step S52: CRF learns the probability transition matrix between secondary structure labels through training. The elements in the probability transition matrix represent transition scores.

[0146] Step S53: Calculate the probability P(Y|X) that the predicted result of amino acid sequence X is secondary structure sequence Y based on the emission fraction and transfer fraction;

[0147] After obtaining the emission and transfer scores, the probability of a particular amino acid being labeled as a specific secondary structure tag can be calculated. Given the amino acid sequence X = (x1, x2, ..., x...). T ) and the secondary structure label sequence Y = (y1, y2, ..., y T The calculation process is as follows:

[0148]

[0149] Among them, f k (y t ,y t-1 ,X,t) is the k-th characteristic function, used to describe the dependencies between secondary structure tags and between secondary structure tags and amino acid residues. θ k It is the characteristic function f kThe weights are learned from the training data. Z(X) is the partition function, used for normalization to ensure that the sum of the probabilities of all possible label sequences is 1. The specific formula is as follows:

[0150]

[0151] Where Q is the set of all possible secondary structure sequences, and Y... ′ ∈Q;

[0152] Step S54: Calculate the final secondary structure prediction result Y using the Viterbi algorithm and probability P(Y|X). * :

[0153]

[0154] Based on the protein secondary structure prediction results Y * The loss is calculated using the secondary structure labels of the training sample set until the loss function converges, at which point training stops, resulting in a well-trained protein language model, Transformer model, CNN model, and BiLSTM model.

[0155] Step S6: Use the amino acid sequence to be predicted as the input of the protein language model, and use the method in step S2 to obtain the fused feature representation of the amino acid sequence to be predicted, and use the fused feature representation as the input of the trained Transformer model.

[0156] The features output by the encoder of the Transformer model are then passed sequentially through the trained CNN model and BiLSTM model, and the protein secondary structure prediction results for the amino acid sequence to be predicted are obtained based on the features output by the BiLSTM model.

[0157] The performance of the method of the present invention is evaluated using two key indicators: the percentage of each amino acid residue in the three-state and eight-state secondary structures that is accurately predicted and the fragment overlap (Sov).

[0158] The percentage of each amino acid residue in the three- and eight-state secondary structures that is accurately predicted is generally referred to as accuracy (Q). m The specific calculation method is as follows:

[0159]

[0160] Among them, Q m The precision is represented by m, the number of types of secondary structures, and N, the total number of residues in the target amino acid sequence. i This represents the number of correctly predicted amino acids.

[0161] The Sov score (Sovereignty over overlap) is frequently used to compare predicted protein secondary structures. Its main purpose is to measure the similarity between the predicted and actual protein secondary structures. The specific calculation method for the Sov score is as follows:

[0162]

[0163] Where, N Sov This represents the total number of amino acid residues in the protein sequence. S1 is the actual fragment, S2 is the predicted fragment, S0 represents the fragment with the same structure between S1 and S2, maxov(S1,S2) represents the largest fragment overlap between S1 and S2, minov(S1,S2) represents the smallest fragment overlap between S1 and S2, and σ(S1,S2) represents the similarity score between S1 and S2. The calculation process of σ(S1,S2) is as follows:

[0164]

[0165] Based on the condition that the sequence is the only input, this invention was compared and evaluated with similar methods on the CASP12, NEW364, TS115 and CB513 test sets, and the comparison results are shown in Table 1.

[0166] Table 1. Comparison results of the present invention and similar methods on four test sets.

[0167]

[0168] For methods where the sequence is not the only input (e.g., PSSM is introduced as input information), this invention was compared with these methods on the CASP11, CASP12 and CB513 test sets, and the comparison results are shown in Table 2.

[0169] Table 2 Comparison results of the present invention and similar methods on three test sets.

[0170]

[0171]

[0172] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for predicting protein secondary structure based on single amino acid sequence features, characterized in that, The method specifically includes the following steps: Step S1: Collect protein secondary structure data files, process the collected protein secondary structure data files, and then obtain the amino acid sequence and the secondary structure tag of each amino acid in the processed protein secondary structure data files. The obtained amino acid sequences and the secondary structure tags of each amino acid in the sequences are used as the training sample set; The specific process of step S1 is as follows: Step S11: Obtain the protein secondary structure data file from the NetSurfP-2.0 dataset, and then perform redundancy removal processing on the obtained protein secondary structure data file to obtain the processed protein secondary structure data file. Step S12: Obtain from the processed data file a FASTA file containing the amino acid sequence of the protein, a JSON file containing the amino acid sequence of the protein, and secondary structure tags for each amino acid in the sequence. Step S2: Use the amino acid sequences in the training sample set as input to the protein language model, and generate feature representations for each amino acid sequence through the protein language model. Sine and cosine position encoding is performed on each amino acid sequence in the training sample set to obtain the position encoding result. Then, the position encoding result is concatenated with the word embedding matrix to obtain the concatenated feature corresponding to each amino acid sequence. The spliced ​​features are then fused with the feature representations output by the protein language model to obtain the fused feature representations for each amino acid sequence. The specific process of step S2 is as follows: Step S21: For any amino acid sequence S, denote the number of amino acids contained in the amino acid sequence S as L. Process the amino acid sequence S through a protein language model to generate a 1024-dimensional embedding for each amino acid in the amino acid sequence S. Concatenate the embeddings generated for L amino acids to obtain the protein embedding of the amino acid sequence S, which is the feature representation of the amino acid sequence S. The dimension of the feature representation of the amino acid sequence S is L×1024. Similarly, the characteristic representation of each amino acid sequence is obtained; Step S22: Perform sine and cosine position encoding on the amino acid sequence S: Where pos represents the amino acid position in amino acid sequence S, PE (pos,2i) It is the 2ith element in the position encoding vector corresponding to the amino acid at position pos, PE (pos,2i+1) It is the (2i+1)th element in the position encoding vector corresponding to the amino acid at position pos, where 2i represents an even number and 2i+1 represents an odd number, d model The dimension representing the word vector; Step S23: Encode the amino acid at position pos into a word embedding representation, and then concatenate the position encoding vector corresponding to the amino acid at position pos and the word embedding representation corresponding to the amino acid at position pos to obtain a concatenated feature with a dimension of 1×512. Then, based on all the amino acids in the amino acid sequence S, the spliced ​​features with a dimension of L×512 are obtained; Step S24: Fuse the protein embedding corresponding to amino acid sequence S and the spliced ​​feature corresponding to amino acid sequence S to obtain the fused feature representation corresponding to amino acid sequence S. Similarly, the fusion-enhanced feature representations for each amino acid sequence are obtained. Step S3: Use the fused feature representation as input to the Transformer model, and output the enhanced features through the encoder of the Transformer model; Step S4: Use the enhanced features from step S3 as input to the CNN model, output the local features of each extracted amino acid sequence through the CNN model, and then use the local features as input to the BiLSTM model, output the global features of each amino acid sequence through the BiLSTM model. Step S5: Based on the global features in step S4, obtain the protein secondary structure prediction results for each amino acid sequence in the training sample set. Calculate the loss based on the protein secondary structure prediction results and the secondary structure labels of the training sample set. Stop training when the loss function converges to obtain the trained protein language model, Transformer model, CNN model, and BiLSTM model. In step S5, based on the global features from step S4, the protein secondary structure prediction results for each amino acid sequence in the training sample set are obtained, specifically as follows: Step S51: The global features obtained in step S4 are linearly transformed through a fully connected layer to make the dimension of the transformed features consistent with the dimension of the label. Each element in the linearly transformed features is used as the emission score of each category in the secondary structure label. Step S52: CRF learns the probability transition matrix between secondary structure labels through training. The elements in the probability transition matrix represent transition scores. Step S53: Calculate the probability P(Y|X) that the predicted result of amino acid sequence X is secondary structure sequence Y based on the emission fraction and transfer fraction; Step S54: Calculate the final secondary structure prediction result Y using the Viterbi algorithm and probability P(Y|X). * ; The specific process of step S54 is as follows: Step S6: Use the amino acid sequence to be predicted as the input of the protein language model, and use the method in step S2 to obtain the fused feature representation of the amino acid sequence to be predicted, and use the fused feature representation as the input of the trained Transformer model. The features output by the encoder of the Transformer model are then passed sequentially through the trained CNN model and BiLSTM model, and the protein secondary structure prediction results for the amino acid sequence to be predicted are obtained based on the features output by the BiLSTM model.

2. The protein secondary structure prediction method based on single amino acid sequence features according to claim 1, characterized in that, The protein language model is a finely tuned ProtT5-XL-Half_Uniref50-Enc model.

3. The protein secondary structure prediction method based on single amino acid sequence features according to claim 2, characterized in that, The protein language model includes N attention layers, and each attention layer has multi-head attention.

4. The protein secondary structure prediction method based on single amino acid sequence features according to claim 3, characterized in that, The encoder of the Transformer model includes M encoder blocks, each of which includes multi-head attention, a feedforward neural network, and Add & Norm.

5. The protein secondary structure prediction method based on single amino acid sequence features according to claim 4, characterized in that, The CNN model includes, in sequence, a first one-dimensional convolutional layer, a first ReLU activation function layer, a first Dropout layer, a second one-dimensional convolutional layer, a second ReLU activation function layer, a second Dropout layer, a third one-dimensional convolutional layer, a third ReLU activation function layer, and a third Dropout layer.

6. The method for predicting protein secondary structure based on single amino acid sequence features according to claim 5, characterized in that, The BiLSTM model consists of a forward LSTM and a backward LSTM.

Images