A protein torsion angle prediction method based on embedded features and dynamic convolutional networks

Abstract

The present invention discloses a protein torsion angle prediction method based on embedded features and dynamic convolutional networks. For a given protein amino acid sequence, the method first uses a pre-trained protein language model to generate embedded features rich in biological information, and then uses a full-dimensional dynamic convolutional network that integrates an attention mechanism to directly predict the torsion angle. In particular, by using full-dimensional dynamic convolution, the designed prediction network can simultaneously learn the attention of spatial dimensions, input channel dimensions, and output channel dimensions and thereby improve the feature extraction capability of the network. In addition, since the network outputs the torsion angle rather than the sine and cosine values ​​of the torsion angle, the time-consuming use of the arctan function to calculate the torsion angle is avoided. In summary, the torsion angle prediction method proposed in the present invention can not only accurately predict the torsion angle based on the amino acid sequence, but is also applicable to any embedded features generated by a pre-trained protein language model.

Description

Technical Field

[0001] The present invention belongs to the technical field of bioinformatics and relates to a method for predicting protein torsion angles, and in particular to a method for predicting protein torsion angles based on embedded features and a dynamic convolutional network. Background Art

[0002] Proteins are biological macromolecules composed of 20 amino acids. They are essential components of living organisms and regulate nearly every biological process within the cell. Proteins play crucial roles in biological processes such as metabolic reactions and pathway regulation, and their biological functions are closely related to their three-dimensional structure. Many applications, such as drug and enzyme design, require knowledge of protein structure to determine their functions. Determining protein three-dimensional structure is known as the protein structure prediction problem. Currently, commonly used experimental methods for determining protein three-dimensional structure include X-ray crystallography, electron microscopy, and nuclear magnetic resonance. However, these experimental methods are extremely time-consuming and expensive. In particular, due to the rapid development of protein sequencing technology, the number of known protein sequences has reached over 300 million, while the number of known protein structures is only around 200,000. To bridge the significant gap between sequence and structure data, it is imperative to develop reliable and efficient computational methods to predict protein three-dimensional structure and its properties, such as torsion angles, solvent accessibility, and supersecondary structure, based on the amino acid sequence of the protein.

[0003] The torsion angle is the dihedral angle formed by four consecutive atoms in the protein backbone, and its value range is -180 degrees to 180 degrees. For a given protein, the bond length and angle between adjacent backbone atoms are fixed, and the only uncertain thing is the torsion angle. Once the torsion angle is determined, its backbone structure can be determined. The torsion angle determines the overall shape and stability of the protein and affects various interactions within the protein molecule. In addition, the torsion angle has a high correlation with the secondary structure of the protein (especially the helical structure and folded structure). The dihedral angle state of a specific residue also has a strong correlation with the adjacent residues in its sequence. In particular, this correlation helps to determine the local conformation of the protein. The secondary structure of a protein is a coarse-grained description of the local backbone structure, and the boundaries between different secondary structure fragments are uncertain. Compared with the secondary structure, the torsion angle with a continuous value can eliminate the uncertainty of the boundary and distinguish different ring conformations. In summary, accurately predicted torsion angles can not only greatly reduce the space for conformational searches and thus help reconstruct the three-dimensional structure of proteins, but can also be used to improve protein folding recognition and multiple sequence alignment [Betancourt, MR and J. Skolnick, Local Propensities and Statistical Potentials of Backbone Dihedral Angles in Proteins. Journal of Molecular Biology, 2004. 342(2): p. 635-649.].

[0004] Many successful prediction algorithms based on neural network methods have been proposed for torsion angle prediction. According to the model framework adopted by the prediction algorithm, current torsion angle prediction methods can be roughly divided into two categories: sliding window-based prediction algorithms and sequence-to-sequence prediction algorithms [Hasan, A., et al., SAINT-Angle: self-attention augmented inception-inside-inception network and transfer learning improve protein backbone torsion angle prediction. Bioinform Adv, 2023. 3(1): p. vbad042.]. However, existing methods usually use PSSM spectrum features generated by time-consuming multiple sequence alignment operations to represent proteins, and the prediction network outputs the sine and cosine values ​​of the torsion angle rather than the torsion angle, so it is necessary to further use the arctan function to calculate the torsion angle. Summary of the Invention

[0005] To address the time-consuming and poorly effective prediction issues of existing protein torsion angle prediction models, which use PSSM spectral features for protein representation and the arctan function for torsion angle calculation, this paper proposes a protein torsion angle prediction method based on embedded features and a dynamic convolutional network. This method first uses a pre-trained protein language model to generate embedded features rich in biological information, and then directly predicts torsion angles using a full-dimensional dynamic convolutional network integrated with an attention mechanism. The proposed torsion angle prediction method not only avoids the time-consuming use of multiple sequence alignment to generate PSSM spectral features and the arctan function for torsion angle calculation, but also enables accurate torsion angle prediction based solely on amino acid sequence.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A protein torsion angle prediction method based on embedded features and dynamic convolutional networks, including:

[0008] Step 1: Select protein chains from the RCSB PDB protein structure database as training data based on the PISCES server;

[0009] Step 2: Obtain the torsion angle of each protein chain;

[0010] Step 3: Use the embedded features derived from the pre-trained protein language model as the feature representation of the protein chain;

[0011] Step 4: Design protein torsion angle prediction network architecture based on dynamic convolution;

[0012] Step 5: Design a loss function for training the protein torsion angle prediction network based on mean absolute error;

[0013] Step 6: Optimize the network settings and determine the final torsion angle prediction model;

[0014] Step 7: Predict protein torsion angles based on the final torsion angle prediction model.

[0015] Furthermore, in step 1, the selection criteria for the protein chain is that the resolution of the protein structure is less than The maximum R-value was 1.0, the maximum protein chain length was 800, the minimum protein chain length was 50, and the threshold for sequence identity was 25%, and protein chains with more than 25% sequence identity with protein chains in the test set were removed.

[0016] Furthermore, the step 2 includes:

[0017] Obtain the corresponding structure file based on the mmCIF format according to the PDBID, then convert the structure file into the corresponding dssp file based on the DSSP program, and finally extract the torsion angle information from the dssp file based on the chain ID. For the torsion angle for which the torsion angle information cannot be obtained, its value is set to NAN.

[0018] Furthermore, the step 4 includes:

[0019] First, basic building blocks are designed based on full-dimensional dynamic 1D convolution, and then the protein torsion angle prediction network architecture is designed based on the basic building blocks.

[0020] Furthermore, the processing flow within the basic building block includes:

[0021] First, the input tensor of size N×C×L is evenly divided into two parts along the channel dimension. The left part remains unchanged, and the right part is sent to a convolution block consisting of a dynamic 1-dimensional convolution with a convolution kernel size of 3, a ReLU activation function and a normalization layer. The feature tensor of the left part and the output of the convolution block are concatenated through the channel to form a new feature tensor of size N×C×L, which is then sent to a convolution block with a convolution kernel size of 1 and a Dropout layer in turn. The output tensor size of the basic building block is still N×C×L, where N represents the batch size, C represents the channel dimension, and L represents the maximum length of the protein chain in the batch.

[0022] Furthermore, skip connections are introduced into the convolutional blocks to avoid gradient vanishing and enhance training stability.

[0023] Furthermore, the processing flow within the protein torsion angle prediction network includes:

[0024] For a given protein chain, it is first fed into a pre-trained protein language model to obtain embedded features, and then a 1D convolution with a convolution kernel size of 1 is used to convert the dimension of the embedded features to C. Then, n stacked basic building blocks are used, and then processed in sequence through a normalization layer, a 1D convolution, a ReLU function, and a 1D convolution. Finally, the Hardtanh activation function is used to limit the output range of the torsion angle to between -1 and 1.

[0025] Furthermore, the loss function is:

[0026]

[0027] Where T ijk and Y ijk They represent the true value and predicted value of the torsion angle in the jth residue of the i-th protein chain in the batch, respectively. k equals 0, which means the corresponding torsion angle is Phi; k equals 1, which means the corresponding torsion angle is Psi. Represents calculation and where N is the batch size and L is the maximum length of protein chains in the batch.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention proposes a protein torsion angle prediction method based on embedded features and dynamic convolutional networks. The method first uses a pre-trained protein language model to generate embedded features rich in biological information, and then uses a full-dimensional dynamic convolutional network that integrates an attention mechanism to directly predict the torsion angle. In particular, by using full-dimensional dynamic convolution, the designed prediction network can simultaneously learn the attention of spatial dimensions, input channel dimensions, and output channel dimensions, thereby improving the network's feature extraction capabilities. In addition, because the network outputs the torsion angle rather than the sine and cosine values ​​of the torsion angle, the time-consuming use of the arctan function to calculate the torsion angle is avoided. The present invention can accurately predict the torsion angle based solely on the amino acid sequence and is applicable to any embedded features generated by a pre-trained protein language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a basic flow chart of a protein torsion angle prediction method based on embedded features and dynamic convolutional networks according to an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of the basic building block network architecture designed for an embodiment of the present invention;

[0032] Figure 3 Schematic diagram of the protein torsion angle prediction network architecture designed for an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0034] like Figure 1 As shown, a protein torsion angle prediction method based on embedded features and dynamic convolutional network includes:

[0035] Step S101: Determine the training, validation, and test sets for protein torsion angles. To facilitate evaluation of the proposed protein torsion angle prediction method, the present invention uses four commonly used datasets: CASP12, CASP13, CASP14, and CB513, as test sets. These four test sets contain 47, 41, 33, and 513 protein chains, respectively, and the sequence identity between any two protein chains in each test set is less than 25%.

[0036] In order to construct the corresponding training set and validation set, the present invention selects protein chains from the RCSB PDB protein structure database based on the PISCES server. The selection criterion is that the resolution of the protein structure is less than The maximum R-value was 1.0, the maximum protein chain length was 800, the minimum protein chain length was 50, and the sequence identity threshold was 25%. We submitted this request to the PISCES server on March 13, 2023, and the server returned 10,981 protein chains. Furthermore, to eliminate redundancy between the returned protein chains and the protein chains in the four test sets, we further removed protein chains with at least 25% sequence identity with the protein chains in the test sets, leaving 10,355 protein chains. From these protein chains, we randomly selected 512 protein chains as the validation set, and the remaining 9,843 protein chains as the training set.

[0037] Step S102: Obtain the torsion angle of each protein chain in the training set, validation set, and test set. For each amino acid in the protein chain, it corresponds to two different torsion angles Phi and Psi. In order to obtain the torsion angle information of a given data set, the present invention first downloads the corresponding structure file based on the mmCIF format from https: / / www.rcsb.org / according to the PDBID, and then converts the structure file into the corresponding dssp file based on the DSSP program provided by the website https: / / github.com / PDB-REDO / dssp, and finally extracts the torsion angle information from the dssp file based on the chain ID. It should be noted that due to the lack of relevant atomic coordinate information in some structural data, the torsion angle information of some amino acid residues cannot be obtained. For this part of the torsion angle, we set its value to NAN so that it does not participate in the training and evaluation of the torsion angle prediction model.

[0038] Step S103: Generate a feature representation vector for the protein chain. In order to predict the protein torsion angle based on the amino acid sequence, each amino acid in the protein chain needs to be converted into a numerical vector. Currently, PSSM spectral features are commonly used for torsion angle prediction. However, generating PSSM spectral features requires performing a multiple sequence alignment operation on a non-redundant protein sequence database such as uniref90. In particular, considering that uniref90 currently contains 170 million non-redundant protein sequences, the decompressed database file reaches 74.8GB, so generating PSSM spectral features is not only extremely time-consuming, but also requires a large storage space. Recently, pre-trained protein language models such as ESM-2 and Ankh, which are trained based on self-supervised deep learning technology and large-scale protein sequence data, have been proven to be able to successfully learn the physicochemical properties of amino acids. In particular, compared with PSSM spectral features, the embedded features derived based on the pre-trained protein language model can not only provide better feature representation, but also perform fast feature generation. Therefore, the present invention adopts embedded features as the feature representation of the protein chain. To determine the best embedding feature representation, we construct embedding features based on the pre-trained models ESM-1b, ProtT5-XL-U50, ESM2_t33_650M_UR50D, ESM2_t36_3B_UR50D, ESM2_t48_15B_UR50D, Ankh_base and Ankh_large [Ahmed Elnaggar, HE, Wafaa Salah-Eldin, Walid Moustafa, Mohamed Elkerdawy, Charlotte Rochereau, Burkhard Rost, Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling.arXiv, 2023.arXiv:2301.06568.]. For a protein chain of length L, the embedding feature matrices generated by these pre-trained models have sizes of L×1280, L×1024, L×1280, L×2560, L×5120, L×768, and L×1536, respectively.

[0039] Step S104: Designing a protein torsion angle prediction network architecture based on dynamic convolution. The present invention first designs a protein torsion angle prediction network architecture based on the full-dimensional dynamic 1-dimensional convolution (ODConv1D) [Chao Li and Aojun Zhou and Anbang Yao, Omni-Dimensional Dynamic Convolution, International Conference on Learning Representations, 2022]. Figure 2 The basic building block shown. For an input tensor of size N×C×L (where N represents the size of the minibatch, C represents the dimension of the channel, and L represents the maximum length of the protein chain in the minibatch), the basic building block first divides it equally into two parts along the channel dimension, left and right. The left part remains unchanged to enhance the feature reuse capability of the network, and the right part is fed into a convolution block consisting of a dynamic 1D convolution with a convolution kernel size of 3, a ReLU activation function, and a layer normalization layer. In particular, the present invention introduces a skip connection in the convolution block to avoid gradient disappearance and enhance the stability of training. The feature tensor of the left part and the output of the convolution block are spliced ​​through the channel to form a new feature tensor of size N×C×L. The feature tensor is sequentially fed into a convolution block with a convolution kernel size of 1 and a Dropout layer. The output tensor size of the basic building block is still N×C×L. In particular, unlike standard convolution, which uses static convolution kernels for all inputs, full-dimensional dynamic 1D convolution uses an attention mechanism to dynamically weight the convolution kernels, thereby achieving input-dependent convolution operations. Specifically, full-dimensional dynamic 1D convolution can be defined as:

[0040] Y=(α w1 ⊙α f ⊙α c ⊙α s ⊙W1+...+α wm ⊙α f ⊙α c ⊙α s ⊙W m )*X

[0041] in and represent input features and output features respectively, represents the parameters of the i-th convolution kernel (i=1,...,m), k represents the size of the convolution kernel, m represents the number of convolution kernels, and * represents the convolution operation. In addition, represents the attention scalar of the i-th convolution kernel, represents the attention of the output channel dimension, represents the attention of the input channel dimension, In order to obtain the four types of attention, we first use a 1-dimensional adaptive average pooling operation to convert the input feature X into a dimension of C. in The feature vector is then fed into the fully connected layer, the 1-dimensional batch normalization layer, the ReLU activation function layer, and four independent head branches. Each head branch contains a fully connected layer. f , α c and α s , the Sigmoid function is used after the fully connected layer. In order to generate the normalized attention α wi (i=1,...,m), the Softmax function is used after the fully connected layer.

[0042] Figure 3 A protein torsion angle prediction framework based on basic building block design is given. For a given amino acid sequence, it is first fed into a pre-trained protein language model to obtain embedded features. Then, a 1-dimensional convolution with a convolution kernel size of 1 is used to convert the dimension of the embedded features to C. Then, n stacked basic building blocks are used, and the optimal value of n is determined by experiments on the validation set. Finally, two real values ​​will be output for each amino acid residue position. The purpose of introducing the Hardtanh activation function is to limit the output range of the torsion angle to between -1 and 1, so as to ensure the stability of the network training. The result is multiplied by 180 to obtain the final predicted torsion angles Phi and Psi. Note that existing methods usually predict the sine and cosine values ​​of the torsion angle, so 4 output values ​​are required for each amino acid residue position. In order to obtain the predicted value of the torsion angle, it is necessary to calculate it using the inverse tangent function arctan. The method of the present invention avoids this operation.

[0043] Step S105: Design the loss function for training the prediction network. is the prediction output of the torsion angle prediction network, is the corresponding target tensor value. Note that the value range of each element in Y is from -1 to 1, while the value range of each element in T is from -180 to 180. Based on the mean absolute error, the loss function can be defined as:

[0044]

[0045] Where T ijk and Y ijk They represent the true value and predicted value of the torsion angle in the jth residue of the i-th protein chain in the mini-batch, respectively. k equals 0, which means the corresponding torsion angle is Phi, and k equals 1, which means the corresponding torsion angle is Psi. and The purpose of minimizing the value between is to reduce the periodicity of the angle. Note that existing methods usually reduce the periodicity of the angle by directly predicting the sine and cosine values ​​of the torsion angle. In essence, the effects of the two are similar.

[0046] Step S106: Optimize the network settings and determine the final torsion angle prediction model. Based on the PyTorch deep learning framework, the present invention uses the AdamW optimizer with a fixed learning rate of 0.001 for network training, and the number of protein chains in the mini-batch is set to 32. In order to reduce overfitting, training is stopped when the evaluation accuracy on the validation set does not improve for 8 consecutive epochs. The proposed torsion angle prediction network contains three parameters: the number of basic building blocks n, the channel parameter C, and the number of convolution kernels m. We set their default values ​​to 6, 1024, and 1, respectively. Based on the default settings of the network and the accuracy on the validation set, we first compared the prediction performance of the embedded features generated by 7 pre-trained models ESM-1b, ProtT5-XL-U50, ESM2_t33_650M_UR50D, ESM2_t36_3B_UR50D, ESM2_t48_15B_UR50D, Ankh_base, and Ankh_large. Experimental results show that the embedded features derived from the pre-trained model Ankh_large can achieve the highest accuracy on the validation set. Therefore, this paper uses it as the default feature for the proposed torsion angle prediction model.

[0047] Step S107: Predicting protein torsion angles based on the final torsion angle prediction model.

[0048] In summary, the present invention proposes a protein torsion angle prediction method based on embedded features and a dynamic convolutional network. This method first uses a pre-trained protein language model to generate embedded features rich in biological information. It then directly predicts torsion angles using a full-dimensional dynamic convolutional network integrated with an attention mechanism. This torsion angle prediction method not only avoids the time-consuming use of multiple sequence alignment to generate PSSM spectral features and the arctan function to calculate torsion angles, but also enables accurate torsion angle prediction based solely on amino acid sequences.

[0049] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A protein torsion angle prediction method based on embedded features and dynamic convolutional networks, characterized in that: include: Step 1: Select protein chains from the RCSB PDB protein structure database as training data based on the PISCES server; Step 2: Obtain the torsion angle of each protein chain; Step 3: Use the embedding features derived from the pre-trained protein language model Ankh_large as the feature representation of the protein chain; Step 4: Designing a protein torsion angle prediction network architecture based on full-dimensional dynamic 1D convolution; the full-dimensional dynamic 1D convolution uses an attention mechanism to dynamically weight the convolution kernel, thereby achieving input-dependent convolution operations; Step 5: Design a loss function for training the protein torsion angle prediction network based on mean absolute error; Step 6: Optimize the network settings and determine the final torsion angle prediction model; Step 7: Predict protein torsion angles based on the final torsion angle prediction model.

2. The protein torsion angle prediction method based on embedded features and dynamic convolutional network according to claim 1, characterized in that: In step 1, the selection criteria for protein chains is that the resolution of the protein structure is less than The maximum R-value was 1.0, the maximum protein chain length was 800, the minimum protein chain length was 50, and the threshold for sequence identity was 25%, and protein chains with more than 25% sequence identity with protein chains in the test set were removed.

3. The protein torsion angle prediction method based on embedded features and dynamic convolutional network according to claim 1, characterized in that: The step 2 includes: Obtain the corresponding structure file based on the mmCIF format according to the PDBID, then convert the structure file into the corresponding dssp file based on the DSSP program, and finally extract the torsion angle information from the dssp file based on the chain ID. For the torsion angle for which the torsion angle information cannot be obtained, its value is set to NAN.

4. The protein torsion angle prediction method based on embedded features and dynamic convolutional network according to claim 1, characterized in that: The step 4 comprises: First, basic building blocks are designed based on full-dimensional dynamic 1D convolution, and then the protein torsion angle prediction network architecture is designed based on the basic building blocks.

5. The protein torsion angle prediction method based on embedded features and dynamic convolutional networks according to claim 4, characterized in that: The processing flow within the basic building blocks includes: First, the input tensor of size N×C×L is evenly divided into two parts along the channel dimension. The left part remains unchanged, and the right part is sent to a convolution block consisting of a dynamic 1-dimensional convolution with a convolution kernel size of 3, a ReLU activation function and a normalization layer. The feature tensor of the left part and the output of the convolution block are concatenated through the channel to form a new feature tensor of size N×C×L, which is then sent to a convolution block with a convolution kernel size of 1 and a Dropout layer in turn. The output tensor size of the basic building block is still N×C×L, where N represents the batch size, C represents the channel dimension, and L represents the maximum length of the protein chain in the batch.

6. The protein torsion angle prediction method based on embedded features and dynamic convolutional networks according to claim 5, characterized in that: Skip connections are introduced into the convolutional blocks to avoid gradient vanishing and enhance training stability.

7. The protein torsion angle prediction method based on embedded features and dynamic convolutional network according to claim 1, characterized in that: The processing flow within the protein torsion angle prediction network includes: For a given protein chain, it is first fed into a pre-trained protein language model to obtain embedded features, and then a 1D convolution with a convolution kernel size of 1 is used to convert the dimension of the embedded features to C. Then, n stacked basic building blocks are used, and then processed in sequence through a normalization layer, a 1D convolution, a ReLU function, and a 1D convolution. Finally, the Hardtanh activation function is used to limit the output range of the torsion angle to between -1 and 1.

8. The protein torsion angle prediction method based on embedded features and dynamic convolutional networks according to claim 3, characterized in that: The loss function is: Where T ijk and Y ijk They represent the true value and predicted value of the torsion angle in the jth residue of the i-th protein chain in the batch, respectively. k equals 0, which means the corresponding torsion angle is Phi; k equals 1, which means the corresponding torsion angle is Psi. Represents calculation and where N is the batch size and L is the maximum length of protein chains in the batch.

Images