A deep learning-based protein sequence chemical shift prediction method

Abstract

The application discloses a protein sequence chemical shift prediction method based on deep learning, comprising the following steps: constructing a data set by using a public database; constructing a deep learning model and setting related training parameters; training the model based on the data set, testing the model performance, and obtaining a trained prediction model; and using the trained prediction model to realize protein sequence chemical shift prediction; the deep learning model comprises a protein language model part and a coding and decoding part, the protein language model part converts each amino acid in the protein sequence into a word embedding vector; the coding and decoding part comprises an encoder and a decoder, the encoder performs feature extraction on the protein sequence which has been converted into a word embedding vector, and the decoder predicts the chemical shift according to the extracted features. The protein sequence chemical shift prediction method based on deep learning has only a dependence on a protein sequence, has high accuracy and robustness, and has a high processing speed.

Description

Technical Field

[0001] This invention relates to the field of protein nuclear magnetic resonance, and more particularly to a method for predicting the chemical shift of protein sequences based on deep learning. Background Technology

[0002] Nuclear magnetic resonance (NMR) is a powerful tool for material analysis, playing a crucial role in the study of protein structure. By measuring the NMR chemical shifts of individual atoms in a protein, we can obtain information about the environment in which those atoms are located, thereby analyzing the protein's structure.

[0003] In NMR experiments on proteins, the NMR apparatus generates a uniform external magnetic field that causes the originally degenerate energy levels of the observed atomic nuclei in the protein to split. Then, radio frequency pulses of different frequencies are applied perpendicular to the uniform external magnetic field. When the energy of the radio frequency pulse exactly matches the energy difference between two adjacent energy levels resulting from the nuclear split, NMR occurs, causing energy level transitions in the atomic nuclei and creating a non-equilibrium state. After the radio frequency pulse is removed, the atomic nuclei release energy and return to equilibrium. The frequency of the free-induction decay signal received during this process is the NMR frequency. For the same type of atom, the shielding effect caused by different chemical environments will result in inconsistent NMR frequencies. Therefore, by selecting an appropriate reference material, the chemical shift of the resonant nucleus relative to the reference material nucleus can be measured.

[0004]

[0005] Where v s With v r These represent the resonance frequencies of the measured and reference nuclei in the sample, respectively, while v0 represents the Larmor precession frequency of the measured nucleus. Chemical shifts, which are directly related to the chemical environment, help researchers understand the chemical structure of substances.

[0006] Various methods have been proposed to obtain the chemical shifts of individual atoms in proteins without relying on experiments. These methods can be broadly classified into sequence alignment-based methods and structure-based methods. SPARTA is a typical sequence-based prediction program that predicts the chemical shifts of the backbone atoms in each amino acid by matching each amino acid in a protein with the surrounding amino acids in a database and then performing a weighted average of the results. Some structure-based methods, using the structure of the input protein, select or calculate typical features such as amino acid type, φ / ψ / χ1 torsion angle, etc., as input to the prediction model and output the predicted chemical shift. For example, ProShift uses 350 features as input (Meiler, J. (2003). "PROSHIFT: protein chemical shift prediction using artificial neural networks." Journal of Biomolecular NMR 26:25-37), SPARTA+ (Shen, Y. and A. Bax (2010). "SPARTA+: a modest improvement in empirical NMR chemical shift prediction by means of an artificial neural network." Journal of Biomolecular NMR 48(1):13-22.) and SHIFTX+ (Han, B., et al. (2011). "SHIFTX2: significantly improved protein chemical shift prediction." Journal of Biomolecular NMR 50(1):43-57.) The program used 113 and 97-bit feature dimensions respectively.In addition, there are ensemble models that combine sequence alignment and structural calculation methods, such as SHIFTX2 (Han, B., et al. (2011). "SHIFTX2: significantly improved protein chemical shift prediction." Journal of Biomolecular NMR 50(1):43-57.) and UCBShift (Li, J., et al. (2020). "Accurate prediction of chemical shifts for aqueous protein structure on “Real World” data." Chemical Science 11(12):3180-3191.). However, in reality, both sequence alignment-based and structural calculation-based feature extraction methods rely on the known structure of the protein. Therefore, the quality and accuracy of the prediction are also affected by the accuracy of the protein structure calculation. Summary of the Invention

[0007] The purpose of this invention is to solve the problem that existing technologies rely on known protein structures and have uncertain prediction accuracy.

[0008] The technical solution adopted by this invention to solve its technical problem is: to provide a protein sequence chemical shift prediction method based on deep learning, comprising the following steps:

[0009] Build datasets using public databases;

[0010] Build a deep learning model and set the relevant training parameters;

[0011] The model is trained based on the dataset, and its performance is tested to obtain a well-trained prediction model.

[0012] Use a trained prediction model to predict the chemical shift of protein sequences;

[0013] The deep learning model includes a protein language model and an encoding / decoding part. The protein language model converts each amino acid in the protein sequence into a word embedding vector. The encoding / decoding part includes an encoder and a decoder. The encoder extracts features from the protein sequence that has been converted into word embedding vectors, and the decoder predicts chemical shifts based on the extracted features.

[0014] Preferably, the construction of the dataset includes the following steps:

[0015] Each protein in the RefDB database is processed into a protein sequence composed of amino acid abbreviations;

[0016] Specify the backbone atom whose chemical shift is to be predicted, and form a sequence by combining the chemical shift values ​​of the backbone atom in each amino acid of the protein sequence to form a chemical shift sequence as a training label. Missing values ​​are padded with zeros.

[0017] Construct a mask sequence of the same length as the chemical shift sequence. Mark the mask sequence with 0 at the positions where missing values ​​in the corresponding chemical shift sequence are padded with zeros, and mark the other positions with 1.

[0018] The processed protein samples containing training labels were divided into 80% training set and 20% test set.

[0019] Preferably, the backbone atoms include Ca atoms, Ha atoms, HN atoms and N atoms, and the prediction model for each backbone atom needs to be trained independently.

[0020] Preferably, the encoder includes a linear layer, a multi-head attention module, a residual connection and layer normalization operation module, a feedforward module, and a residual connection and layer normalization operation module connected in sequence.

[0021] Preferably, the decoder includes a multi-head attention module, a residual connection and a layer normalization operation module, and two linear layers connected in sequence.

[0022] Preferably, the multi-head attention module is represented as

[0023] MultiHead(Q,K,V)=Concat(head1,…,head h W O

[0024] head i =Attention(QW i Q ,KW i k VW i V )

[0025] Among them W Q W K W V W O The parameter represents the linear projection matrix, where W Q W K , Where h represents the number of attention heads, W O Used to project the concatenated attention results onto the dimensions of the model.

[0026] Preferably, the feedforward module consists of a linear layer connected to another linear layer after passing through a nonlinear activation unit and a Dropout layer.

[0027] Preferably, training the model based on the dataset includes the following steps:

[0028] The training set, along with the mask sequence, is input into the deep learning model to obtain the output chemical shift prediction value.

[0029] The output chemical shift prediction value is processed by a mask matrix, combined with the training labels, and the loss value is calculated by the root mean square loss function. The gradient of the network parameters is obtained by the backpropagation algorithm, and then the network parameters are updated by the stochastic gradient descent method.

[0030] The training set, along with the mask matrix, is input into the deep learning model after the network parameters are updated to obtain the output chemical shift prediction value.

[0031] The process of repeatedly calculating the loss value and iteratively updating the network parameters continues until the loss value decreases to convergence or reaches the set number of training rounds, at which point training is terminated.

[0032] Preferably, the test model performance is specifically achieved by inputting the test set along with the mask sequence into the trained model to obtain chemical shift prediction values, comparing them with the training labels, and testing the model's performance in predicting chemical shifts for that atom type.

[0033] Preferably, the step of using a trained prediction model to predict the chemical shift of a protein sequence specifically involves inputting the protein sequence to be predicted into the model to obtain a chemical shift sequence of the same length as the original sequence.

[0034] This invention, as the only method currently available that can determine chemical shifts solely based on protein sequences, has the following advantages:

[0035] (1) The prediction results are not affected by the accuracy of protein structure calculation and have strong robustness;

[0036] (2) It has a fast prediction speed, and can predict the chemical shift of a single protein in just a few seconds;

[0037] (3) The multi-head attention mechanism was used to capture sequence-related information of amino acids, which enabled the prediction level to be comparable to that of structure-based methods when only the sequence was input.

[0038] (4) The simple model training and the provision of readily available datasets make this invention easy to reproduce.

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. Attached Figure Description

[0040] Figure 1 This is a diagram illustrating the method steps of an embodiment of the present invention;

[0041] Figure 2 This is a diagram of the deep learning network model structure according to an embodiment of the present invention;

[0042] Figure 3 This is a structural diagram of the multi-head attention module according to an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram illustrating a protein sample prediction example according to an embodiment of the present invention. Detailed Implementation

[0044] See Figure 1 The diagram shows the method steps of an embodiment of the present invention, including the following steps:

[0045] S101, Construct a dataset using a public database;

[0046] S102, Build a deep learning model and set the relevant training parameters;

[0047] S103: Train the model based on the dataset, test the model performance, and obtain a trained prediction model;

[0048] S104, using a trained prediction model to predict the chemical shift of protein sequences;

[0049] The deep learning model includes a protein language model and an encoding / decoding part. The protein language model converts each amino acid in the protein sequence into a word embedding vector. The encoding / decoding part includes an encoder and a decoder. The encoder extracts features from the protein sequence that has been converted into word embedding vectors, and the decoder predicts chemical shifts based on the extracted features.

[0050] Specifically, the S101 Reference Database (RefDB) contains 2162 proteins and their corresponding chemical shifts. This method requires each protein in the RefDB to be processed into sequence data as input, i.e., a protein sequence composed of the abbreviations of twenty common amino acids. Simultaneously, a chemical shift sequence of equal length to the protein sequence is used as the training label. The chemical shifts of each type of backbone atom (Cα, Hα, HN, N) are processed into separate sequences. The position of each chemical shift sequence corresponding to the protein sequence indicates the chemical shift value of that atom in that amino acid, and missing values ​​are padded with zeros. It is important to note that during the construction of the training data, for parallel model training, all sequences need to be padded with zeros to a length of 512. A mask sequence of equal length to the chemical shift sequence is also constructed. This mask sequence labels missing chemical shift information at corresponding amino acid positions as 0, and other positions as 1, ensuring that the sequence padding and previous zero-padding operations do not affect model training. The final input consists of protein sequences, and four mask sequence sets and chemical shift sequence sets corresponding to the four backbone atom types (Cα, Hα, HN, N), respectively. All data is then divided into an 80% training set and a 20% test set. In this example, the Cα partition contains 1321 proteins as training samples and 331 as test samples.

[0051] Specifically, the deep learning network model structure described in S102 can be found here. Figure 2 As shown, the model first includes a pre-trained ESM2 module. ESM2 is a pre-trained protein language model provided by Facebook's FAIR (Meta Fundamental AI Research Protein Team). The protein sequence data is first pre-processed by the ESM2 module, converting each amino acid in the protein sequence into a word embedding vector. This vector is then input into the encoder part of the right half of the model. The encoder consists of a linear layer, a multi-head attention module, two add connections, a layer normalization operation, and a feedforward module. The encoder output is then input into the decoder part of the left half of the model. The decoder includes a multi-head attention module, a add connection, a layer normalization operation, and two linear layers. The multi-head attention module is the core architecture of the model, and its structure is as follows: Figure 3 As shown, the input matrix is ​​multiplied by three different matrices to obtain three matrices: Q (Query), K (Key), and V (Value). These matrices are then divided into multiple blocks (i.e., multiple heads), and the outputs are calculated separately according to the attention mechanism formula. The outputs from these multiple heads are then concatenated and projected through a linear layer to obtain the final output. The mathematical model of the attention mechanism is as follows:

[0052]

[0053] The mathematical model for multi-head attention is:

[0054] MultiHead(Q,K,V)=Concat(head1,…,head h W O

[0055] head i =Attention(QW i Q ,KW I k VW i V )

[0056] The feedforward module consists of two linear layers, a dropout layer, and a nonlinear activation unit (ReLU).

[0057] In this embodiment, the linear layer parameters of the encoder are set to 1280→512, projecting the dimension of the word embedding vector to a size of 512. The number of heads in the multi-head attention module is set to 8. The two linear layer parameters in the feedforward module are set to 512→2048 and 2048→512, respectively, and Dropout is set to 0.1. The parameters of the multi-head attention module in the decoder are the same as those of the encoder, with the two linear layer parameters being 512→2048 and 2048→1, respectively.

[0058] Specifically, S103 inputs the training set into the deep learning model. The ESM2 module in the model first converts each amino acid in the sequence into a word embedding vector. In this example, taking Cα as an example, the dimension of each sample after processing is 512×1280. Here, 512 is the length of the padded sequence, 1280 is the dimension of the word embedding vector of the ESM model, and the dimensions of the mask sequence and chemical shift label are 512×1. Then, the sample along with the mask sequence is input into the model to obtain the output chemical shift prediction value. The network output is processed by the mask sequence and then the chemical shift label of Cα is used to calculate the loss value using the root mean square loss function (RMSE). The gradient of the network parameters is obtained through the backpropagation algorithm. Based on the calculated gradient, the network parameters are updated using the SGD (stochastic gradient descent) optimization algorithm. The data is input into the network with updated parameters, and the output and label are repeated to calculate a new loss value. The network parameters are iteratively updated until the loss value decreases to convergence or reaches the set number of training rounds, at which point training is terminated. The loss value represents the error between the network output and the label. The smaller the error, the closer the network output is to the label, and the better the network's prediction performance. It should be noted that each specified atom type requires an independent training of a separate prediction model. This example uses Cα to train a prediction model for Cα type atoms separately, employing the SGD optimization algorithm with a learning rate of 0.1, a batch size of 16, and a network training epoch of 5000.

[0059] Specifically, in S104, the test set and mask matrix are directly input into the trained model, and the network output is the predicted chemical shift value. When comparing and evaluating the predicted values ​​with the true values, the mask matrix is ​​used to mask the missing parts of the true values ​​in the dataset to prevent the zero-padding operation in S101 from affecting the evaluation. The root mean square error of the models for different atom types on the test set is shown in Table 1:

[0060] Table 1: Root mean square error of models with different atoms

[0061]

[0062] This example uses the predicted and actual Cα chemical shift values ​​of a single protein to evaluate the model performance, such as... Figure 4 As shown, the sample is protein with BMRB index number 6321. The horizontal axis represents the actual chemical shift value, and the vertical axis represents the predicted chemical shift value, in ppm. The results demonstrate that this example achieved relatively good prediction results using only the protein sequence as input.

[0063] In summary, the deep learning-based protein sequence chemical shift predictor proposed in this invention has high accuracy and robustness, and also has a fast processing speed.

[0064] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting the chemical shift of a protein sequence based on deep learning, characterized in that, Includes the following steps: Build datasets using public databases; Build a deep learning model and set the relevant training parameters; The model is trained based on the dataset, and its performance is tested to obtain a well-trained prediction model. Use a trained prediction model to predict the chemical shift of protein sequences; The deep learning model includes a protein language model and an encoding / decoding part. The protein language model converts each amino acid in the protein sequence into a word embedding vector. The encoding / decoding part includes an encoder and a decoder. The encoder extracts features from the protein sequence that has been converted into word embedding vectors, and the decoder predicts chemical shifts based on the extracted features. The construction of the dataset includes the following steps: Each protein in the RefDB database is processed into a protein sequence composed of amino acid abbreviations; Specify the backbone atom whose chemical shift is to be predicted, and form a sequence by combining the chemical shift values ​​of the backbone atom in each amino acid of the protein sequence to form a chemical shift sequence as a training label. Missing values ​​are padded with zeros. Construct a mask sequence of the same length as the chemical shift sequence. Mark the mask sequence with 0 at the positions where missing values ​​in the corresponding chemical shift sequence are padded with zeros, and mark the other positions with 1. The protein samples containing training labels obtained after processing are divided into 80% training set and 20% test set. The backbone atoms include Ca atoms, Ha atoms, HN atoms and N atoms, and the prediction model for each backbone atom needs to be trained independently; The encoder includes a linear layer, a multi-head attention module, a residual connection and layer normalization operation module, a feedforward module, and a residual connection and layer normalization operation module connected in sequence. The decoder includes a multi-head attention module, a residual connection and a layer normalization operation module connected in sequence, as well as two linear layers.

2. The protein sequence chemical shift prediction method based on deep learning according to claim 1, characterized in that, The multi-head attention module is represented as: ； ； in , , , The parameter represents the linear projection matrix, where , , Where h represents the number of attention heads, Used to project the concatenated attention results onto the dimensions of the model.

3. The protein sequence chemical shift prediction method based on deep learning according to claim 1, characterized in that, The feedforward module consists of a linear layer, a nonlinear activation unit, a dropout layer, and then another linear layer.

4. The protein sequence chemical shift prediction method based on deep learning according to claim 1, characterized in that, The training of the model based on the dataset includes the following steps: The training set, along with the mask sequence, is input into the deep learning model to obtain the output chemical shift prediction value. After the output chemical shift prediction value is processed by the mask sequence, combined with the training labels, the loss value is calculated by the root mean square loss function, the gradient of the network parameters is obtained by the backpropagation algorithm, and the network parameters are updated by the stochastic gradient descent method. The training set, along with the mask sequence, is input into the deep learning model after the network parameters are updated to obtain the output chemical shift prediction value. The process of repeatedly calculating the loss value and iteratively updating the network parameters continues until the loss value decreases to convergence or reaches the set number of training rounds, at which point training is terminated.

5. The protein sequence chemical shift prediction method based on deep learning according to claim 1, characterized in that, The test model performance is specifically as follows: the test set along with the mask sequence is input into the trained model to obtain the chemical shift prediction value, which is then compared with the training label to test the model's performance in predicting chemical shifts by atomic type.

6. The protein sequence chemical shift prediction method based on deep learning according to claim 1, characterized in that, The method of using a trained prediction model to predict the chemical shift of a protein sequence specifically involves inputting the protein sequence to be predicted into the model to obtain a chemical shift sequence of the same length as the original sequence.

Images