Protein melting temperature prediction method based on mixed deep learning strategy

Abstract

The invention relates to the technical field of protein prediction, and discloses a protein melting temperature prediction method based on a hybrid deep learning strategy, comprising the following steps: step 1, constructing a protein melting temperature prediction model based on the deep learning strategy, the prediction model comprising a sequence feature extraction module, a structural feature extraction module and a forward network module; step 2, inputting a primary sequence of the protein to be detected into the prediction model; and step 3, outputting the protein melting temperature and the protein category by using the prediction model. According to the prediction method provided by the invention, the prediction model comprehensively considers the structural characteristics and sequence characteristics of the protein, and particularly considers the spatial conformation of amino acid residues in the protein structure, so that the prediction model provided by the invention can more accurately predict the protein melting temperature.

Description

Technical Field

[0001] The present invention relates to the technical field of protein performance prediction, and in particular to a protein melting temperature prediction method using a hybrid deep learning strategy. Background Art

[0002] The melting temperature of a protein is an important indicator for evaluating the thermal stability of a protein. It refers to the temperature at which a protein partially or completely loses its three-dimensional structure and dissociates into amino acids under high temperature conditions. Different proteins have different melting temperatures, depending on their structure and amino acid composition. Generally speaking, the dissociation temperature of a protein is between 50°C and 90°C. The melting temperature directly reflects the folding state and thermal stability of a protein, and is of great significance for understanding the structure and function of a protein, optimizing protein design, and improving industrial production efficiency.

[0003] The use of computational methods to predict protein melting temperature can quickly obtain this indicator. However, in existing methods, most of the input data are global features at the protein level, and fail to effectively utilize local information at the level of amino acid residues that make up the protein. In particular, protein structure determines function, and protein structure is determined by the spatial conformation of its amino acid residues.

[0004] A protein is made up of multiple amino acids connected by shrinking (removing H+ and OH-). The primary structure of a protein refers to the sequence of residues that make up the protein. The protein structure refers to the three-dimensional conformation of a protein molecule in its natural folded state.

[0005] Currently, the feature representation of proteins in computational models mostly uses features extracted from the protein level, such as amino acid content, dipeptide content, physicochemical properties, structural composition, etc. The computational models are mainly based on random forests, support vector machines, gradient boosting trees, etc., and the focus is more on input feature selection or feature dimensionality reduction.

[0006] Starting from the constituent amino acids of proteins, we can make full use of sequence feature information and amino acid local structure information to better characterize proteins;

[0007] The computational methods for predicting protein melting temperatures are mainly divided into two categories: regression problems for predicting specific values ​​of melting temperatures and classification problems for identifying thermophilic proteins. In the classification problem, there are three classification problems for the temperature limits of hyperthermophilic proteins (>65°C), thermophilic proteins (55°C to 65°C), and mesophilic proteins (<55°C); and two classification problems for thermophilic and mesophilic proteins. Summary of the invention

[0008] In order to solve the technical problems raised in the background technology, the present invention provides a protein melting temperature prediction method based on a hybrid deep learning strategy.

[0009] The present invention is implemented by the following technical solution: a protein melting temperature prediction method using a hybrid deep learning strategy, comprising the following steps:

[0010] Step 1, constructing a protein melting temperature prediction model based on a deep learning strategy, wherein the prediction model includes a sequence feature extraction module, a structural feature extraction module, and a forward network module;

[0011] Step 2, inputting the primary sequence of the protein to be tested into the prediction model;

[0012] Step 3: Use the prediction model to output protein melting temperature and protein category.

[0013] Specifically, constructing the prediction model in step 1 includes the following steps:

[0014] 1.3. Obtain training data set

[0015] 1.4. Feature extraction;

[0016] 1.3. Build a deep learning model.

[0017] Specifically, the step 1.1 is specifically operated as follows: the training and test data are from the Meltome Atlas data set, proteins with a sequence length of less than 1024 are extracted from the Meltome Atlas data set, and sequences with a sequence similarity greater than 30% are removed using CD-HIT software, and then these sequences are divided into a training set, a validation set, and a test set at a ratio of 8:1:1 using a fixed random seed.

[0018] Specifically, the specific operations of step 1.2 are as follows:

[0019] 1.21. Use the protein language pre-trained large model ProtT5-XL-UniRef50 and ESMFold model for feature extraction:

[0020] 1.22. Normalize the protein melting temperature;

[0021] Specifically, the specific operations of step 1.21 are as follows:

[0022] The primary sequence of the protein is input into the model ProtT5-XL-UniRef50, and the output data is the feature representation vector X of the i-th residue in the corresponding sequence i , and X i The dimension is 1024;

[0023] ESMFold is used to predict the three-dimensional structure of proteins, obtain the three-dimensional coordinates of atoms in each amino acid residue in the sequence, and then calculate the C α -C α The Euclidean distance of atoms, if the Euclidean distance is less than Then the two amino acids are considered to be in contact, and a residue contact map matrix is ​​obtained, the size of which is L*L, where L is the sequence length;

[0024] Where: Node e in row i and column j of the residue contact map matrix ij =1, indicating that there is a contact relationship between the i-th residue and the j-th residue in the sequence, e ij =0 means that there is no contact relationship between the i-th residue and the j-th residue in the sequence;

[0025] Specifically, the specific operations of step 1.22 are as follows:

[0026] The protein melting temperature was normalized using formula (1), where Tm i is the true melting temperature of the i-th protein, Max is the highest melting temperature value in the current data set, and Min is the lowest melting temperature;

[0027] In the three-classification, the protein temperature category C is divided according to formula (2), with 0, 1, and 2 representing the numerical temperature category respectively;

[0028] In binary classification, proteins with a melting temperature ≥55°C were classified as thermophilic proteins, otherwise they were mesophilic proteins;

[0029]

[0030]

[0031] Then the feature representation vector X of the i-th residue in the corresponding sequence is i , the contact map matrix is ​​input into the sequence feature extraction module and the structural feature extraction module respectively.

[0032] Specifically, the deep learning model in step 1.3 includes a position encoding module, a sequence feature extraction module, a structural feature extraction module, and an output module.

[0033] Specifically, in the position encoding module, a sine function or a cosine function is used for encoding, and the encoding is shown in formula (3) and formula (4);

[0034]

[0035] Where i represents the position of the amino acid in the protein sequence, 0≤i<L, D represents the feature dimension D=1024, 2j and 2j+1 represent the feature component position of the residue at the current position, 0≤j<D / 2;

[0036] Then, the residue representation input to the dual-channel network module is added with position information. The specific operation is as shown in formula (5):

[0037]

[0038] Specifically, the sequence feature extraction module is composed of a multi-head attention network, a multi-scale convolutional neural network, a bidirectional long short-term memory network and a forward attention network, and the input data of the sequence feature extraction module is

[0039] The multi-head attention network uses 8 attention heads; in the multi-head attention network, its inputs each pass through a layer of forward neural network to obtain Q i , K i 、V i , as shown in formula (6), the input data is 1024-dimensional and the output data is 128-dimensional; and one attention head i Calculation is as in formula (7) and formula (8), where d k The value is 128; the "Concat" operation in formula (9) means that the data is merged in the last dimension, and the 8 attention heads are concatenated in the last dimension, and then output after passing through a layer of forward neural network. The output dimension is 1024, and the length of the protein sequence remains unchanged;

[0040]

[0041]

[0042] Head i =Score(Q i ,K i ,V i )V i (8)

[0043] MA(Q,K,V)=Concat(Head1,Head2,...,Head8)W o (9)

[0044] The multi-scale convolutional neural network uses one-dimensional convolution and multiple convolutional neural networks with different convolution kernel sizes to perform convolution operations in parallel; the input data comes from the output data of the multi-head attention network, the input data is 1024 dimensions, the output data is 128 dimensions, and the convolution kernel sizes are [3, 5, 7, 9] respectively; the default activation function of the convolution operation is ReLU; the outputs of multiple convolution devices are merged, as shown in formula (11): the output of the multi-scale convolutional neural network is 512 dimensions;

[0045] Conv i =f(W*x j:j+k-1 +b)i∈{1,2,3,4},j∈{0,1...,1023},k∈{3,5,7,9}(10)

[0046] Conv=Concat(Conv1,Conv2,Conv3,Conv4)(11)

[0047] The bidirectional long short-term memory network is used to focus on the global information of proteins and capture the long-range dependencies of protein sequences. It is composed of a bidirectional long short-term memory network. The unidirectional LSTM model is described in the form of (12).

[0048]

[0049]

[0050] Among them, σ is the activation function using the Sigmoid function; ⊙ represents the bitwise multiplication of the matrix; x t is the network input at time t; i t 、f t ,,o t 、c t and h t They represent the input gate, forget gate, output gate, internal memory unit and output at time t respectively; h t-1 is the output of the previous moment; c t-1 is the output of the internal memory unit at the previous moment; the rest are the learnable parameters of the neural network;

[0051] The unidirectional LSTM network has a 1024-dimensional input and a 256-dimensional output. When the forward LSTM and backward LSTM data converge, a merge operation is performed at the lowest dimension, as shown in formula (13), and the output is 512-dimensional.

[0052] The forward attention network is used to convert the two-dimensional matrix into a one-dimensional vector, h t is the output of the previous network layer, representing the tth amino acid residue in the sequence;

[0053] Through a layer of forward neural network as shown in formula (14), the attention weight of the current feature representation of residue t in the sequence is obtained through the formula; the feature representations of all residues in the sequence are weighted and summed, as shown in formula (16), to achieve the conversion of two-dimensional feature representation to one-dimensional; here, L takes the value of 1024; the input data dimension is L*512, and the output dimension is 1*512;

[0054] e t =h t W t (14)

[0055]

[0056] Specifically, the structural feature extraction module includes a graph attention convolution network, a graph diffusion network, and a graph average pooling network, wherein:

[0057] The spatial structure of proteins in the graph attention convolutional network is represented by a graph, where nodes are amino acid residues and the connecting edges between nodes represent the contact between two residues; the node representation is the sequence feature information of formula (5), that is, input Input1; the edge information is described by the contact map matrix, that is, input Input2;

[0058] The network node is connected to the forward network layer, as shown in formula (17), with an input dimension of 1024 and an output dimension of 256;

[0059] For the neighbor node j of the current node i, data is concatenated, then multiplied with the learnable vector a (dimension 256*2, 1), compressed into a scalar, and then passed through the LeakyReLU activation function, as shown in formula (19);

[0060] Calculate the attention score α between the current node i and the neighbor node j ij , the calculation formula uses the Softmax function, such as formula (20), where N(i) represents the set of neighbor nodes of node i, and then updates the information of node i, such as formula (21), the activation function σ uses the LeakyReLU function, and the output dimension is 256;

[0061] h' i =Wh i (17)

[0062] h' ij =Concat(h i ,h j ) (18)

[0063] e i,j =LeakyReLU(a T h' ij ) (19)

[0064]

[0065] The graph diffusion network obtains the adjacent node information of the adjacent nodes of the node through diffusion convolution;

[0066] The original protein-protein contact matrix A has a dimension of 1024*1024, I is the identity matrix with a diagonal of 1, and has the same dimension as A. D is the degree matrix of matrix A, and D -1 is the normalized matrix of D, and the matrix A is transformed by formula (22-23) to H (0) That is, the node information output by the graph attention convolutional network, with a dimension of 256; H (k) is the node feature after k iterations, α is a learnable balance parameter with an initial value of 0.9; W is the parameter matrix;

[0067] The diffusion operation of node information acquisition is implemented through formula (24) and formula (25). The operation is repeated 5 times, hoping to obtain neighboring point information at 5 levels; the output node dimension of this network module is still 256.

[0068]

[0069] H (k+1) =ReLU(WH') (25)

[0070] The output node information and the original adjacency matrix of the graph diffusion network are sent back to the graph attention convolutional network unit, and the above operation is repeated iteratively again; at the same time, the node information is sent to the graph average pooling network; the results of repeated iterations are also sent to the graph average pooling network.

[0071] The graph average pooling network is used to adopt a graph global pooling strategy for the node information in the graph. The current feature of the i-th residue in the sequence is represented as x i , take the average of all such features in the sequence, as shown in formula (26), and compress the L*256 matrix into a 1*256 vector through graph global pooling;

[0072]

[0073] The two output results of the graph average pooling unit are concatenated to obtain a vector of 1*512 dimensions.

[0074] It is necessary to perform feature concatenation on the output F1 of the sequence feature extraction network and the output F2 of the structure feature extraction network, as shown in formula (27), with an output dimension of 1*1024;

[0075] F=Concat(F1,F2)(27)

[0076] Then connect two layers of feedforward neural networks, as shown in formula (28) and formula (29), respectively. The first feedforward neural network uses tanh as the activation function, and the output dimension is 256; the second one uses Sigmoid function or Softmax function, and the output dimension is 1 or 3;

[0077] F′=tanh(W*F+b) (28)

[0078]

[0079] In the output module:

[0080] The error loss between the model prediction value and the true value is described by formula (30) or formula (31), where (30) is the prediction loss of melting temperature and formula (31) is the prediction loss of two or three classifications;

[0081]

[0082] In the above formula, N is the number of training samples and i is the i-th protein.

[0083] Compared with the prior art, the present invention has the following beneficial effects:

[0084] When using computational methods to predict protein melting temperatures in the prior art, most of the input data are global features at the protein level, and local information at the level of amino acid residues that make up the protein is not effectively utilized. Compared with the prior art, the prediction method proposed in the present invention has a prediction model that comprehensively considers the structural characteristics and sequence characteristics of the protein, especially the spatial conformation of the amino acid residues in the protein structure. Therefore, the prediction model proposed in the present invention can predict the protein melting temperature more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 A flowchart of the prediction method proposed by the present invention;

[0086] Figure 2 This is a schematic diagram of the overall structure of the prediction model proposed by the present invention;

[0087] Figure 3 A diagram of a network module for extracting sequence features in the prediction model proposed by the present invention;

[0088] Figure 4 This is a network module diagram for extracting structural features in the prediction model proposed in the present invention. DETAILED DESCRIPTION

[0089] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0090] Example:

[0091] Reference Figure 1-Figure 4 ,This scheme proposes a protein melting temperature prediction method based on a hybrid deep learning strategy,,including the following steps:

[0092] Step 1, constructing a protein melting temperature prediction model based on a deep learning strategy, wherein the prediction model includes a sequence feature extraction module, a structural feature extraction module, and a forward network module;

[0093] Step 2, inputting the primary sequence of the protein to be tested into the prediction model;

[0094] Step 3: Use the prediction model to output protein melting temperature and protein category.

[0095] Specifically, constructing the prediction model in step 1 includes the following steps:

[0096] 1.1. Obtaining training data set

[0097] 1.2. Feature extraction;

[0098] 1.3. Build a deep learning model.

[0099] Specifically, the step 1.1 is specifically operated as follows: the training and test data are from the Meltome Atlas data set, proteins with a sequence length of less than 1024 are extracted from the Meltome Atlas data set, and sequences with a sequence similarity greater than 30% are removed using CD-HIT software, and then these sequences are divided into a training set, a validation set, and a test set at a ratio of 8:1:1 using a fixed random seed.

[0100] Specifically, the specific operations of step 1.2 are as follows:

[0101] 1.21, including the use of protein language pre-training large model ProtT5-XL-UniRef50 and ESMFold model for feature extraction:

[0102] 1.22. Normalize the protein melting temperature;

[0103] Specifically, the specific operations of step 1.21 are as follows:

[0104] The primary sequence of the protein is input into the model ProtT5-XL-UniRef50, and the output is the feature representation vector X of the i-th residue in the corresponding sequence i , X i The dimension is 1024;

[0105] ESMFold was used to predict the three-dimensional structure of proteins, obtain the three-dimensional coordinates of atoms in each amino acid residue in the sequence, and calculate the C α -C α The Euclidean distance of atoms, if the Euclidean distance is less than Then the two amino acids are considered to be in contact, and a residue contact map matrix is ​​obtained, the size of which is L*L, where L is the sequence length;

[0106] Where: Node e in row i and column j of the residue contact map matrix ij =1, indicating that there is a contact relationship between the i-th residue and the j-th residue in the sequence, e ij =0 means that there is no contact relationship between the i-th residue and the j-th residue in the sequence;

[0107] Specifically, considering the convenience of data batch input during deep learning model training, all sequence lengths were aligned to 1024, those less than 1024 were padded with zeros, the residue features were 1024*1024, and the contact map matrix was 1024*1024.

[0108] Specifically, the specific operations of step 1.22 are as follows:

[0109] The protein melting temperature was normalized using formula (1), where Tm i is the true melting temperature of the ith protein, Max is the highest melting temperature value in the current data set, and Min is the lowest melting temperature. In the three-classification, the protein temperature category C is divided according to formula (2), with 0, 1, and 2 representing the numerical temperature category respectively. In the two-classification (0 and 1 represent the numerical category), the protein with a melting temperature ≥ 55°C is classified as a thermophilic protein, otherwise it is a mesophilic protein.

[0110]

[0111] The feature representation vector X corresponding to the i-th residue in the sequence i , the contact map matrix is ​​input into the sequence feature extraction module and the structural feature extraction module respectively.

[0112] Specifically, the deep learning model in step 1.3 includes a position encoding module, a sequence feature extraction module, a structural feature extraction module, and an output module;

[0113] Specifically, in the position coding, the arrangement direction of amino acids in the protein starts from the N-terminal amino group and ends at the C-terminal carboxyl group; in order to enable the model to use the order information of amino acids in the protein sequence, the sequence features are positionally coded; the coding is performed using a sine function or a cosine function, and the coding is shown in formula (3) and formula (4);

[0114]

[0115] Where i represents the position of the amino acid in the protein sequence, 0≤i<L, D represents the feature dimension D=1024, 2j and 2j+1 represent the feature component position of the residue at the current position, 0≤j<D / 2;

[0116] For the residue representation input to the dual-channel network module, the position information is added. The specific operation is as shown in formula (5):

[0117]

[0118] Specifically, in the sequence feature extraction module;

[0119] The sequence feature extraction module consists of a multi-head attention network, a multi-scale convolutional neural network, a bidirectional long short-term memory network (BLSTM), and a forward attention network. The specific structure is as follows: Figure 2 As shown. And the input of the sequence feature extraction module is the

[0120] About the multi-head attention network;

[0121] Eight attention heads are used; in the multi-head attention network, each input passes through a layer of forward neural network to obtain Q i , K i 、V i , as in formula (6), the input data is 1024-dimensional and the output data is 128-dimensional;

[0122] And an attention head i Calculate as formula (7) and formula (8), where d k value 128;

[0123] The "Concat" operation in formula (9) indicates that the data is merged in the last dimension. The meaning of this operation is consistent in the present invention. The eight attention heads are concatenated in the last dimension and then output after passing through a layer of feedforward neural network (MLP). The output dimension is 1024, and the length of the protein sequence remains unchanged.

[0124]

[0125]

[0126] Head i =Score(Q i ,K i ,V i )V i (8)

[0127] MA(Q,K,V)=Concat(Head1,Head2,...,Head8)W o (9)

[0128] Regarding the multi-scale convolutional neural network, it adopts one-dimensional convolution, and multiple convolutional neural networks with different convolution kernel sizes perform convolution operations in parallel; the input data comes from the output data of the multi-head attention network, the input data is 1024 dimensions, the output data is 128 dimensions, and the convolution kernel size (the number of residues involved in the convolution each time) is [3, 5, 7, 9] respectively, aiming to fully extract the local characteristics of the sequence and fit the characteristics of the secondary structure composed of a few continuous residues; the default activation function of the convolution operation is ReLU; the outputs of multiple convolution devices are merged, as shown in formula (11): the output of this network layer is 512 dimensions.

[0129] Conv i =f(W*x j:j+k-1 +b)i∈{1,2,3,4},j∈{0,1...,1023},k∈{3,5,7,9} (10)

[0130] Conv=Concat(Conv1,Conv2,Conv3,Conv4) (11)

[0131] About a layer of bidirectional long short-term memory (BLSTM) network; the second part focuses on the global information of proteins, captures the long-range dependencies of protein sequences, and is composed of a bidirectional long short-term memory network; the unidirectional LSTM model is formulated as shown in (12).

[0132]

[0133] Among them, σ is the activation function using the Sigmoid function; ⊙ represents the bitwise multiplication of the matrix; x t is the network input at time t; i t 、f t ,,o t 、c t and h t They represent the input gate, forget gate, output gate, internal memory unit and output at time t respectively; h t-1 is the output of the previous moment; c t-1 is the output of the internal memory unit at the previous moment; the rest are learnable parameters of the neural network.

[0134] The unidirectional LSTM network has an input of 1024 dimensions and an output of 256 dimensions. When the forward LSTM and backward LSTM data are converged, a merge operation is performed at the lowest dimension, as shown in formula (13), and the output is 512 dimensions.

[0135] The forward attention network is used to convert the two-dimensional matrix into a one-dimensional vector, h t is the output of the previous network layer, representing the tth amino acid residue in the sequence;

[0136] Through a layer of forward neural network as shown in formula (14), the attention weight of the current feature representation of residue t in the sequence is obtained through the formula; the weighted summation of all residue feature representations in the sequence is performed, as shown in formula (16), to realize the conversion of two-dimensional feature representation to one-dimensional; the input data dimension is L*512, and the output dimension is 1*512.

[0137] e t =h t W t (14)

[0138]

[0139] Specifically, the structural feature extraction module includes a graph attention convolutional network unit, a graph diffusion network unit, and a graph average pooling unit, wherein:

[0140] The spatial structure of the protein in the graph attention convolutional network unit is represented by a graph, where the nodes are amino acid residues and the connecting edges between nodes represent the contact between two residues (C α The distance between atoms is less than ); the node represents the sequence feature information of formula (5), that is, input Input1; the edge information is described by the contact graph matrix, that is, input Input2;

[0141] The network node is connected to the forward network layer, as shown in formula (17), with an input dimension of 1024 and an output dimension of 256.

[0142] For the neighbor node j of the current node i, data is concatenated, then multiplied with the learnable vector a (dimension 256*2, 1), compressed into a scalar, and then passed through the LeakyReLU activation function, as shown in formula (19);

[0143] Calculate the attention score α between the current node i and the neighbor node j ij , the calculation formula uses the Softmax function, such as formula (20), where N(i) represents the set of neighbor nodes of node i, and then updates the information of node i, such as formula (21), where the activation function σ uses the LeakyReLU function, and the output dimension is 256;

[0144] h' i =Wh i (17)

[0145] h' ij =Concat(h i ,hj ) (18)

[0146] e i,j =LeakyReLU(a T h' ij ) (19)

[0147]

[0148] In the diffusion network unit of the graph, the adjacent node information of the adjacent nodes of the node is obtained through diffusion convolution (in the graph, the information of node 1, adjacent nodes 2 and 3, and the information of adjacent nodes 2 and 4, and adjacent nodes 3 and 5 are obtained after one diffusion convolution).

[0149] The original protein-protein contact matrix A has a dimension of 1024*1024, I is the identity matrix with a diagonal of 1, and has the same dimension as A. D is the degree matrix of matrix A, and D -1 is the normalized matrix of D, and the matrix A is transformed by formula (22-23) to H (0) That is, the node information output by the graph attention convolutional network, with a dimension of 256; H (k) is the node feature after k iterations, α is a learnable balance parameter with an initial value of 0.9; W is the parameter matrix;

[0150] The diffusion operation of node information acquisition is implemented through formula (24-25). The operation is repeated 5 times, hoping to obtain neighboring point information at 5 levels; the output node dimension of this network module is still 256.

[0151]

[0152] H (k+1) =ReLU(WH') (25)

[0153] The output node information and the original adjacency matrix (i.e., ContactMap) of the graph diffusion network unit are sent back to the graph attention convolutional network unit, and the above operation is repeated again; at the same time, the node information is also sent to the graph average pooling network module. The result of the above repeated operation is also sent to the graph average pooling unit network module.

[0154] The graph average pooling unit is used to apply a graph global pooling strategy to the node information in the graph. The current feature of the i-th residue in the sequence is represented as x i , take the average of all such features in the sequence, as shown in formula (26). Through graph global pooling, the L*256 matrix is ​​compressed into a 1*256 vector;

[0155]

[0156] The output results of the two graph average pooling units are concatenated to obtain a vector of 1*512 dimensions.

[0157] It is necessary to perform feature concatenation on the output F1 of the sequence feature extraction network and the output F2 of the structure feature extraction network, as shown in formula (27), with an output dimension of 1*1024.

[0158] F=Concat(F1,F2)(27)

[0159] Then connect two layers of feedforward neural networks, as shown in formulas (28) and (29), respectively. The first feedforward neural network uses tanh as the activation function, and the output dimension is 256; the second one uses Sigmoid function (regression and binary classification) or Softmax function (three-classification), and the output dimension is 1 or 3.

[0160] F′=tanh(W*F+b) (28)

[0161]

[0162] Specifically, in the output module:

[0163] The error loss between the model prediction value and the true value is described by (30) or (31), where (30) is the melting temperature prediction loss and (31) is the binary or tri-class prediction loss.

[0164]

[0165] In the above formula, N is the number of training samples and i is the i-th protein.

[0166] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A protein melting temperature prediction method based on a hybrid deep learning strategy, characterized in that: The steps include: Step 1: construct a protein melting temperature prediction model based on a deep learning strategy, wherein the prediction model includes a sequence feature extraction module and a structural feature extraction module; Step 2, inputting the primary sequence of the protein to be tested into the prediction model; Step 3: Use the prediction model to output protein melting temperature and protein category.

2. The protein melting temperature prediction method of a hybrid deep learning strategy according to claim 1, characterized in that: The step 1 of constructing the prediction model includes the following steps: 1.

1. Obtain training data set; 1.

2. Feature extraction; 1.

3. Build a deep learning model.

3. The protein melting temperature prediction method of a hybrid deep learning strategy according to claim 1, characterized in that: The specific operation of step 1.1 is as follows: the training and test data are from the Meltome Atlas dataset, proteins with a sequence length of less than 1024 are extracted from the Meltome Atlas dataset, and sequences with a sequence similarity higher than 30% are removed using CD-HIT software, and then these sequences are divided into a training set, a validation set, and a test set at a ratio of 8:1:1 using a fixed random seed.

4. The protein melting temperature prediction method of a hybrid deep learning strategy according to claim 1, characterized in that: The specific operations of step 1.2 are as follows: 1.

21. Use the protein language pre-trained large model ProtT5-XL-UniRef50 and ESMFold model for feature extraction: 1.

22. Normalize the protein melting temperature.

5. The protein melting temperature prediction method of a hybrid deep learning strategy according to claim 1, characterized in that: The specific operation of step 1.21 is as follows: The primary sequence of the protein is input into the model ProtT5-XL-UniRef50, and the output data is the feature representation vector X of the i-th residue in the corresponding sequence i , and X i The dimension is 1024; ESMFold is used to predict the three-dimensional structure of proteins, obtain the three-dimensional coordinates of atoms in each amino acid residue in the sequence, and then calculate the C α -C α The Euclidean distance of atoms, if the Euclidean distance is less than Then the two amino acids are considered to be in contact, and a residue contact map matrix is ​​obtained, the size of which is L*L, where L is the sequence length; Where: Node e in row i and column j of the residue contact map matrix ij =1, indicating that there is a contact relationship between the i-th residue and the j-th residue in the sequence, e ij =0 means that there is no contact relationship between the i-th residue and the j-th residue in the sequence.

6. The protein melting temperature prediction method of a hybrid deep learning strategy according to claim 1, characterized in that: The specific operation of step 1.22 is as follows: The protein melting temperature was normalized using formula (1), where Tm i is the true melting temperature of the i-th protein, Max is the highest melting temperature value in the current data set, and Min is the lowest melting temperature; In the three-classification, the protein temperature category C is divided according to formula (2), with 0, 1, and 2 representing the numerical temperature category respectively; In binary classification, proteins with a melting temperature ≥55°C were classified as thermophilic proteins, otherwise they were mesophilic proteins; Then the feature representation vector X of the i-th residue in the corresponding sequence is i , the contact map matrix is ​​input into the sequence feature extraction module and the structural feature extraction module respectively.

7. The protein melting temperature prediction method of a hybrid deep learning strategy according to claim 1, characterized in that: The deep learning model in step 1.3 includes a position encoding module, a sequence feature extraction module, a structural feature extraction module, and an output module.

8. The protein melting temperature prediction method of a hybrid deep learning strategy according to claim 1, characterized in that: In the position encoding module, a sine function or a cosine function is used for encoding, and the encoding is shown in formula (3) and formula (4); Where i represents the position of the amino acid in the protein sequence, 0≤i<L, D represents the feature dimension D=1024, 2j and 2j+1 represent the feature component position of the residue at the current position, 0≤j<D / 2; Then, the residue representation input to the dual-channel network module is added with position information. The specific operation is as shown in formula (5):

9. The protein melting temperature prediction method of a hybrid deep learning strategy according to claim 1, characterized in that: The sequence feature extraction module is composed of a multi-head attention network, a multi-scale convolutional neural network, a bidirectional long short-term memory network and a forward attention network, and the input data of the sequence feature extraction module is The multi-head attention network uses 8 attention heads; in the multi-head attention network, its inputs each pass through a layer of forward neural network to obtain Q i , K i 、V i , as in formula (6), the input data is 1024-dimensional and the output data is 128-dimensional; And an attention head i Calculation is as in formula (7) and formula (8), where d k value 128; The "Concat" operation in formula (9) means that the data is merged in the last dimension, and the eight attention heads are concatenated in the last dimension, and then output after passing through a layer of forward neural network. The output dimension is 1024, and the length of the protein sequence remains unchanged; Head i =Score(Q i ,K i ,V i )V i (8) MA(Q,K,V)=Concat(Head1,Head2,...,Head8)W o (9) The multi-scale convolutional neural network uses one-dimensional convolution and multiple convolutional neural networks with different convolution kernel sizes to perform convolution operations in parallel; the input data comes from the output data of the multi-head attention network, the input data is 1024 dimensions, the output data is 128 dimensions, and the convolution kernel sizes are [3, 5, 7, 9] respectively; the default activation function of the convolution operation is ReLU; the outputs of multiple convolution devices are merged, as shown in formula (11): the output of the multi-scale convolutional neural network is 512 dimensions; Conv i =f(W*x j:j+k-1 +b)i∈{1,2,3,4},j∈{0,1...,1023},k∈{3,5,7,9}(10) Conv=Concat(Conv1,Conv2,Conv3,Conv4) (11) The bidirectional long short-term memory network is used to focus on the global information of proteins and capture the long-range dependencies of protein sequences. It is composed of a bidirectional long short-term memory network LSTM. The unidirectional LSTM model is described in the form of (12). i t =σ(W i x t +U i h t-1 +b i ) f t =σ(W f x t +U f h t-1 +b f ) the t =σ(W o x t +U o h t-1 +b o ) c t =f t ⊙c t-1 +i t ⊙tanh(W g x t +U g h t-1 +b g ) h t =o t ⊙tanh(c t ) (12) Among them, σ is the activation function using the Sigmoid function; ⊙ represents the bitwise multiplication of the matrix; x t is the network input at time t; i t 、f t ,,o t 、c t and h t They represent the input gate, forget gate, output gate, internal memory unit and output at time t respectively; h t-1 is the output of the previous moment; c t-1 is the output of the internal memory unit at the previous moment; the rest are the learnable parameters of the neural network; The unidirectional LSTM network has a 1024-dimensional input and a 256-dimensional output. When the forward LSTM and backward LSTM data converge, a merge operation is performed at the lowest dimension, as shown in formula (13), and the output is 512-dimensional. The forward attention network is used to convert the two-dimensional matrix into a one-dimensional vector, h t is the output of the previous network layer, representing the tth amino acid residue in the sequence; Through a layer of forward neural network as shown in formula (14), the attention weight of the current feature representation of residue t in the sequence is obtained through the formula; the feature representations of all residues in the sequence are weighted and summed, as shown in formula (16), to achieve the conversion of two-dimensional feature representation to one-dimensional; here, L takes the value of 1024; the input data dimension is L*512, and the output dimension is 1*512; have been t =h t W t (14) 10. The protein melting temperature prediction method of a hybrid deep learning strategy according to claim 7, characterized in that: The structural feature extraction module includes a graph attention convolution network, a graph diffusion network, and a graph average pooling network, wherein: The spatial structure of proteins in the graph attention convolutional network is represented by a graph, where nodes are amino acid residues and the connecting edges between nodes represent the contact between two residues; the node representation is the sequence feature information of formula (5), that is, input Input1; the edge information is described by the contact map matrix, that is, input Input2; The network node is connected to the forward network layer, as shown in formula (17), with an input dimension of 1024 and an output dimension of 256; For the neighbor node j of the current node i, data is concatenated, then multiplied with the learnable vector a (dimension 256*2, 1), compressed into a scalar, and then passed through the LeakyReLU activation function, as shown in formula (19); Calculate the attention score α between the current node i and the neighbor node j ij , the calculation formula uses the Softmax function, such as formula (20), where N(i) represents the set of neighbor nodes of node i, and then updates the information of node i, such as formula (21), the activation function σ uses the LeakyReLU function, and the output dimension is 256; h’ i =Wh i (17) h’ ij =Concat(h’ i ,h’ j ) (18) have been i,j =LeakyReLU(a T h' ij ) (19) The graph diffusion network obtains the adjacent node information of the adjacent nodes of the node through diffusion convolution; The original protein-protein contact matrix A has a dimension of 1024*1024, I is the identity matrix with a diagonal of 1, and has the same dimension as A. D is the degree matrix of matrix A, and D -1 is the normalized matrix of D, and the matrix A is transformed by formula (22-23) to H (0) That is, the node information output by the graph attention convolutional network, with a dimension of 256; H (k) is the node feature after k iterations, α is a learnable balance parameter with an initial value of 0.9; W is the parameter matrix; The diffusion operation of node information acquisition is implemented through formula (24) and formula (25). The operation is repeated 5 times, expecting to obtain neighboring point information at 5 levels. The output node dimension of this network module is still 256. A (k+1) =ReLU(WH′) (25) The output node information and the original adjacency matrix of the graph diffusion network are sent back to the graph attention convolutional network unit, and the above operation is repeated iteratively again; at the same time, the node information is also sent to the graph average pooling network; the results of repeated iterations are also sent to the graph average pooling network. The graph average pooling network is used to adopt a graph global pooling strategy for the node information in the graph. The current feature of the i-th residue in the sequence is represented as x i , take the average of all such features in the sequence, as shown in formula (26), and compress the L*256 matrix into a 1*256 vector through graph global pooling; Concatenate the two output results of the graph average pooling unit to obtain a vector of 1*512 dimensions; It is necessary to perform feature concatenation on the output F1 of the sequence feature extraction network and the output F2 of the structure feature extraction network, as shown in formula (27), with an output dimension of 1*1024; F=Concat(F1,F2) (27) Then connect two layers of feedforward neural networks, as shown in formula (28) and formula (29), respectively. The first feedforward neural network uses tanh as the activation function, and the output dimension is 256; the second one uses Sigmoid function or Softmax function, and the output dimension is 1 or 3; F′=tanh(W*F+b) (28) In the output module: The error loss between the model prediction value and the true value is described by formula (30) or formula (31), where (30) is the prediction loss of melting temperature and formula (31) is the prediction loss of two or three classifications; In the above formula, N is the number of training samples and i is the i-th protein.

Images