Protein attribute prediction method, computer equipment and computer storage medium
By performing multi-level feature extraction and temporal processing on protein sequences, an initial prediction model was constructed, which solved the problems of accuracy and cost in protein attribute prediction, achieving higher prediction accuracy and robustness while reducing prediction costs.
Patent Information
- Application Number
- CN202411030056.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2026-01-30
AI Technical Summary
Existing protein attribute prediction methods suffer from problems such as low prediction accuracy and high prediction costs, making it difficult to effectively capture deep information in protein sequences.
By performing multi-level feature extraction and temporal processing on protein sequences, an initial prediction model is constructed using machine learning algorithms. The feature representations of the protein sequences are extracted layer by layer, and the model parameters are adjusted using real attribute values until the convergence condition is met, thus obtaining the target prediction model.
It improves the accuracy and robustness of protein attribute prediction, reduces prediction costs, and demonstrates excellent performance on different datasets with good generalization ability.
Smart Images
Figure CN121438951A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of bioinformatics, specifically to a protein property prediction method, computer equipment, and computer storage medium. Background Technology
[0002] Predicting protein properties (such as solubility and thermal stability) has significant applications in biomedicine, drug development, and biotechnology. Accurate prediction of protein properties helps in understanding protein function, stability, and interactions, thereby advancing related research and applications.
[0003] Currently, methods for predicting protein properties can be mainly classified into the following categories:
[0004] Sequence-based statistical methods utilize statistical features in protein sequences for prediction. Common techniques include amino acid composition analysis and sequence alignment analysis. However, these methods rely on feature engineering, making it difficult to capture deeper information within protein sequences, resulting in relatively low prediction accuracy.
[0005] Structure-based simulation methods predict protein properties using their three-dimensional structure, employing techniques such as molecular dynamics simulations. However, obtaining the three-dimensional structure of proteins is typically costly and time-consuming, limiting their application.
[0006] Therefore, existing protein attribute prediction methods suffer from problems such as low prediction accuracy and high prediction costs, which hinder the widespread application of protein attribute prediction. Summary of the Invention
[0007] This application provides a protein attribute prediction method, computer device, and computer storage medium. By extracting multi-level features and performing temporal processing on protein sequences, the accuracy of protein attribute prediction is improved and the prediction cost is reduced.
[0008] A first aspect of this application provides a protein property prediction method, the method comprising:
[0009] Obtain a training set, which includes multiple sets of training samples, each set of training samples including a protein sequence and the true attribute value of the protein sequence;
[0010] Obtain an initial prediction model, which includes a multi-layered feature extraction network;
[0011] The multiple sets of training samples are input into the initial prediction model so that the feature extraction network at each level of the initial prediction model extracts the feature representation corresponding to the protein sequence in each set of training samples, thereby obtaining the feature representation of the protein sequence at multiple levels.
[0012] The initial prediction model is input with the feature representation of each protein sequence at multiple levels and the true attribute value of the protein sequence, so that the initial prediction model processes the feature representation of the protein sequence at multiple levels to obtain the predicted attribute value of the protein sequence, and adjusts the model parameters of the initial prediction model based on the true attribute value and the predicted attribute value, until the model training stops when the convergence condition is met, and the target prediction model is obtained.
[0013] The target prediction model is used to predict the attribute values of the protein sequence to be tested based on the feature representation of the protein sequence at multiple levels.
[0014] A second aspect of this application provides a protein property prediction method, the method comprising:
[0015] A target prediction model is obtained, which includes a multi-layer feature extraction network. The target prediction model is trained by a machine learning algorithm on multiple sets of training samples. Each set of training samples includes a protein sequence and the true attribute value of the protein sequence.
[0016] The target prediction model is input with the protein sequence to be tested, so that the feature extraction network at each level of the target prediction model extracts the feature representation corresponding to the protein sequence to be tested, thereby obtaining the feature representation of the protein sequence to be tested at multiple levels. The feature representation of the protein sequence to be tested at multiple levels is then processed to output the predicted attribute value of the protein sequence to be tested.
[0017] A third aspect of this application provides a computer device, the computer device comprising:
[0018] The first acquisition unit is used to acquire a training set, which includes multiple sets of training samples, and each set of training samples includes a protein sequence and the true attribute value of the protein sequence.
[0019] The second acquisition unit is used to acquire an initial prediction model, which includes a feature extraction network with multiple layers.
[0020] The feature extraction unit is used to input the multiple sets of training samples into the initial prediction model, so that the feature extraction network at each level in the initial prediction model extracts the feature representation corresponding to the protein sequence in each set of training samples, thereby obtaining the feature representation of the protein sequence at multiple levels.
[0021] The model training unit is used to input the feature representation of each protein sequence at multiple levels and the true attribute value of the protein sequence into the initial prediction model, so that the initial prediction model processes the feature representation of the protein sequence at multiple levels to obtain the predicted attribute value of the protein sequence, and adjusts the model parameters of the initial prediction model based on the true attribute value and the predicted attribute value until the convergence condition is met and the model training stops to obtain the target prediction model.
[0022] The target prediction model is used to predict the attribute values of the protein sequence to be tested based on the feature representation of the protein sequence at multiple levels.
[0023] A fourth aspect of this application provides a computer device, the computer device comprising:
[0024] An acquisition unit is used to acquire a target prediction model, the target prediction model including a multi-layer feature extraction network, the target prediction model being trained by a machine learning algorithm on multiple sets of training samples, each set of training samples including a protein sequence and the true attribute value of the protein sequence;
[0025] The prediction unit is used to input the protein sequence to be tested into the target prediction model, so that the feature extraction network at each level of the target prediction model extracts the feature representation corresponding to the protein sequence to be tested, obtains the feature representation of the protein sequence to be tested at multiple levels, processes the feature representation of the protein sequence to be tested at multiple levels, and outputs the predicted attribute value of the protein sequence to be tested.
[0026] A fifth aspect of this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods of the first and second aspects described above.
[0027] A sixth aspect of this application provides a computer storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described in the first and second aspects.
[0028] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:
[0029] In this embodiment, each level of the feature extraction network in the initial prediction model extracts the feature representation corresponding to the protein sequence in each training sample, obtaining multi-level feature representations of the protein sequence. The initial prediction model processes these multi-level feature representations to obtain the predicted attribute values of the protein sequence. Based on the true and predicted attribute values, the model parameters of the initial prediction model are adjusted until the convergence condition is met, at which point model training stops, resulting in the target prediction model. By extracting multi-level feature information from the protein sequence, richer sequence information can be captured. Temporal processing of this multi-level feature information effectively captures long-range dependencies, improving prediction accuracy and robustness. Furthermore, through multi-level feature extraction and temporal processing, the model performs excellently on different datasets, exhibiting good generalization ability and improving the accuracy of protein attribute prediction while reducing prediction costs. Attached Figure Description
[0030] Figure 1 This is a flowchart illustrating a protein property prediction method in an embodiment of this application.
[0031] Figure 2 This is another flowchart illustrating the protein attribute prediction method in the embodiments of this application;
[0032] Figure 3 This is an exemplary schematic diagram of a network structure for the time step unit of an LSTM model in the embodiments of this application;
[0033] Figure 4 This is another flowchart illustrating the protein attribute prediction method in the embodiments of this application;
[0034] Figure 5 This is a schematic diagram of the structure of a computer device in an embodiment of this application;
[0035] Figure 6 This is another schematic diagram of the structure of the computer device in the embodiments of this application;
[0036] Figure 7 This is another schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation
[0037] This application provides a protein attribute prediction method, computer device, and computer storage medium. By extracting multi-level features and performing temporal processing on protein sequences, the accuracy of protein attribute prediction is improved and the prediction cost is reduced.
[0038] The protein property prediction method in the embodiments of this application is described below:
[0039] Please see Figure 1One embodiment of the protein property prediction method in this application includes:
[0040] 101. Obtain a training set, which includes multiple sets of training samples, each set of training samples including a protein sequence and the true attribute value of the protein sequence;
[0041] The method of this embodiment can be applied to computer devices, which can be servers, terminals, or other computer devices capable of performing data processing. When the computer device is a terminal, it can be a personal computer (PC), desktop computer, or other terminal device; when the computer device is a server, it can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud databases, cloud computing, big data, and artificial intelligence platforms.
[0042] The computer equipment can acquire a training set for model training. This training set includes multiple training samples, each containing a protein sequence and its true attribute values. These true attribute values can be obtained through biochemical experiments on the protein sequence or from an existing protein sequence database. This embodiment does not limit the method of obtaining the attribute values of the protein sequences in the training samples. Specifically, protein attributes include solubility, thermal stability, etc., and the attribute values of the protein sequence can include solubility data, thermal stability data, and other attribute data.
[0043] 102. Obtain an initial prediction model, which includes a feature extraction network with multiple layers;
[0044] The computer device can deploy an initial prediction model, which includes a multi-layered feature extraction network used to extract feature representations of protein sequences.
[0045] 103. Input the multiple sets of training samples into the initial prediction model, so that the feature extraction network at each level in the initial prediction model extracts the feature representation corresponding to the protein sequence in each set of training samples, thereby obtaining the feature representation of the protein sequence at multiple levels.
[0046] The computer device can input multiple sets of training samples into the initial prediction model. Then, the feature extraction network of each layer in the initial prediction model extracts the feature representation corresponding to the protein sequence in each set of training samples. That is, the feature extraction network of each layer outputs a feature representation of the protein sequence at one level. The output of each layer of the feature extraction network represents the feature information of the protein sequence at different levels, thereby obtaining the feature representation of the protein sequence at multiple levels.
[0047] 104. Input the feature representation of each protein sequence at multiple levels and the true attribute value of the protein sequence into the initial prediction model, so that the initial prediction model processes the feature representation of the protein sequence at multiple levels to obtain the predicted attribute value of the protein sequence, and adjusts the model parameters of the initial prediction model based on the true attribute value and the predicted attribute value until the convergence condition is met and the model training is stopped to obtain the target prediction model.
[0048] The computer equipment can input multiple levels of feature representations of each protein sequence and the true attribute values of the protein sequence into the initial prediction model. The initial prediction model can then be trained based on these multiple levels of feature representations and the true attribute values. Specifically, the initial prediction model processes the multiple levels of feature representations of the protein sequence to obtain the predicted attribute values, and adjusts the model parameters based on the true and predicted attribute values until a convergence condition is met, at which point model training stops, resulting in the target prediction model. This target prediction model can be used to predict the attribute values of the protein sequence under test based on its multiple levels of feature representations.
[0049] Model training can meet the convergence condition if: the loss function value constructed based on the true and predicted attribute values of the protein sequence tends to stabilize, i.e., the change range of the loss function value is within a preset range; or, the number of iterations of model training reaches a preset number. This embodiment does not limit the circumstances under which the model reaches the convergence condition.
[0050] In this embodiment, each level of the feature extraction network in the initial prediction model extracts the feature representation corresponding to the protein sequence in each training sample, obtaining multi-level feature representations of the protein sequence. The initial prediction model processes these multi-level feature representations to obtain the predicted attribute values of the protein sequence. Based on the true and predicted attribute values, the model parameters of the initial prediction model are adjusted until the convergence condition is met, at which point model training stops, resulting in the target prediction model. By extracting multi-level feature information from the protein sequence, richer sequence information can be captured. Temporal processing of this multi-level feature information effectively captures long-range dependencies, improving prediction accuracy and robustness. Furthermore, through multi-level feature extraction and temporal processing, the model performs excellently on different datasets, exhibiting good generalization ability and improving the accuracy of protein attribute prediction while reducing prediction costs.
[0051] The following will be discussed in the preceding text. Figure 1 Based on the illustrated embodiments, embodiments of this application will be described in further detail. Please refer to [link / reference]. Figure 2 Another embodiment of the protein property prediction method in this application includes:
[0052] 201. Obtain a training set, which includes multiple sets of training samples, each set of training samples including a protein sequence and the true attribute value of the protein sequence;
[0053] In this embodiment, the computer device can acquire a dataset of protein sequences and divide it into a training set, a validation set, and a test set according to a preset ratio. This dataset can be derived from existing protein sequence databases, from which various protein sequences and their attribute values can be obtained. The attribute values of each protein sequence in the protein sequence database can be considered as the true attribute values of the protein sequences. Of course, the true attribute values of protein sequences can also be obtained through biochemical experiments.
[0054] 202. Obtain an initial prediction model, which includes a multi-layered feature extraction network;
[0055] In one optional implementation, the initial prediction model may consist of a protein language model and an initial temporal processing model. The protein language model includes a multi-layered feature extraction network for extracting multi-layered feature representations of the protein sequence. The initial temporal processing model performs temporal processing on the multi-layered feature representations of the protein sequence to extract the dependencies between these representations, thereby learning the association between the features of the protein sequence and its attributes, laying the foundation for accurately predicting the attributes of the protein sequence.
[0056] 203. Input the multiple sets of training samples into the initial prediction model, so that the feature extraction network at each level in the initial prediction model extracts the feature representation corresponding to the protein sequence in each set of training samples, thereby obtaining the feature representation of the protein sequence at multiple levels.
[0057] When the initial prediction model consists of a protein language model and an initial temporal processing model, the training set can be input into the protein language model. Each level of the feature extraction network in the protein language model extracts the feature representation corresponding to the protein sequence in each training sample, obtaining multi-level feature representations of the protein sequence. This protein language model can be an ESM (Embedding Sequence Model). ESM is a method that uses deep learning technology to predict protein structure and function. The core idea of ESM is to treat the protein sequence as a language, with each amino acid as a character, and then use an autoregressive neural network to learn the statistical regularities of this language. This neural network is the Transformer, an encoder-decoder model based on an attention mechanism, capable of effectively processing long sequence data. ESM trains a Transformer encoder on a large-scale protein sequence database to learn the evolutionary rules of proteins and the sequence-structure-function relationship.
[0058] 204. Input the feature representation of each protein sequence at multiple levels and the true attribute value of the protein sequence into the initial prediction model, so that the initial prediction model processes the feature representation of the protein sequence at multiple levels to obtain the predicted attribute value of the protein sequence, and adjusts the model parameters of the initial prediction model based on the true attribute value and the predicted attribute value until the convergence condition is met and the model training is stopped to obtain the target prediction model.
[0059] When the initial prediction model consists of a protein language model and an initial time-series processing model, after obtaining multi-level feature representations of each protein sequence using the protein language model, the multi-level feature representations of each protein sequence and the true attribute values of the protein sequence can be input into the initial time-series processing model. The initial time-series processing model then processes the multi-level feature representations of the protein sequence to obtain the predicted attribute values of the protein sequence. Based on the true attribute values and the predicted attribute values, the model parameters of the initial time-series processing model are adjusted until the convergence condition is met, at which point model training stops, and the target time-series processing model is obtained.
[0060] Among them, the protein language model is used to extract feature representations of the protein sequence to be tested at multiple levels, and the target time series processing model is used to predict the attribute values of the protein sequence to be tested based on the feature representations of the protein sequence to be tested at multiple levels.
[0061] The initial time-series processing model can be any neural network model capable of processing sequential data, such as a recurrent neural network (RNN), like the Long Short-Term Memory (LSTM) model, or other types of recurrent neural networks. The LSTM model has memory units and gating mechanisms, which can effectively capture long-term dependencies. LSTM introduces memory units and gating mechanisms such as input gates, output gates, and forget gates. Through these gating mechanisms, information can be selectively remembered, forgotten, and output, effectively handling long-term dependencies and gradient problems.
[0062] Specifically, the initial time-series processing model includes multiple time-step units. For the first time-step unit of the initial time-series processing model, the first-level feature representation of the protein sequence is used as the input of the first time-step unit. The first-level feature representation of the protein sequence is calculated according to the preset activation function and preset operators in the first time-step unit to obtain the cell state and hidden state of the first time-step unit.
[0063] Subsequently, each time step unit after the first time step unit of the initial time-series processing model is regarded as the current time step unit. The feature representation of the protein sequence at the same level as the current time step unit is used as the input of the current time step unit. At the same time, the cell state and hidden state output by the previous time step unit are used as the input of the current time step unit. The inputs are calculated according to the preset activation function and preset operators in the current time step unit to obtain the cell state and hidden state of the current time step unit.
[0064] When the output of the last time step unit of the initial time-series processing model is obtained, the predicted attribute value of the protein sequence can be calculated based on the output of the last time step unit of the initial time-series processing model. The model parameters can then be adjusted based on the predicted attribute value and the true attribute value of the protein sequence until the convergence condition is met, at which point training stops and the target time-series processing model is obtained.
[0065] For example, when the initial time series processing model is an LSTM model, one time step unit of its model structure is as follows: Figure 3 As shown, in this time step unit, σ refers to the sigmoid activation function, and tanh refers to another activation function. If it is the first time step unit, then its input x is the first level of feature representation of the protein sequence.
[0066] If it is a time step unit following the first time step unit, then its input x is a feature representation of the protein sequence at the same level as that time step unit, among multiple levels of feature representation. For example, the input x of the second time step unit is the feature representation of the protein sequence at the second level, the input x of the third time step unit is the feature representation of the protein sequence at the third level, and so on. Simultaneously, its input c is the cell state output by the previous time step unit, and its input h is the hidden state output by the previous time step unit.
[0067] The forget gate operation uses a sigmoid function to determine which information should be forgotten and which should be retained. The output of the forget gate is a value between 0 and 1, indicating how much past information should be forgotten. When the output is close to 1, the information passed through the forget gate is completely retained; when the output is close to 0, the information passed through the forget gate is completely forgotten.
[0068] The input gate operation uses a sigmoid function to determine which information should be retained. The output of the input gate is a value between 0 and 1, indicating which new inputs should be preserved. When the value is close to 1, all new inputs are fully preserved; when the value is close to 0, all new inputs are completely ignored. The input gate controls the weights of new inputs at the current time step. Through the input gate, LSTM can better handle long sequences of data, avoiding the vanishing and exploding gradient problems, thereby improving the model's training performance and stability.
[0069] The output gate operation uses a sigmoid function to determine which information should be output. The output of the gate is a value between 0 and 1, indicating which information should be output. When the value is close to 1, all information is fully preserved; when the value is close to 0, all information is completely masked. Through the output gate, LSTM can adaptively control the output of information based on the cell state and hidden state of the current time step. In this way, important information in the LSTM network can be automatically filtered out and passed to the next layer or the output layer. The output gate controls which information from the hidden state of the current time step should be output, thereby improving the accuracy and performance of the LSTM network.
[0070] The ESM model processes protein sequences to generate multi-layered feature representations, with each layer capturing different information from the protein sequence. For example, an ESM model with 34 layers can extract 34 layers of feature information. The Transformer network in the ESM model performs attention calculations on the input of each layer; later layers extract features that focus more on global semantics, while earlier layers extract features that focus more on local semantics. Therefore, the outputs of these layers can be considered as inputs to different time steps of the LSTM model.
[0071] In the protein language model, the first-level feature extraction network can extract the feature representation of the protein sequence in each training sample. Then, the feature extraction network at every other level can extract the feature representation of the protein sequence in each training sample, resulting in multiple levels of feature representation for each protein sequence.
[0072] For example, a protein language model has multiple layers of feature extraction networks. The feature representation of a protein sequence can be extracted by multiple feature extraction networks at the first, third, fifth, and seventh levels of the model. That is, the feature extraction network at each level extracts features from the protein sequence.
[0073] If the protein language model is an ESM model, then the feature extraction network at each level can be a Transformer network. The Transformer network can extract the feature information of the protein sequence, such as encoding the protein sequence and generating an embedding representation of each protein sequence.
[0074] Therefore, by extracting protein sequence features through a feature extraction network every other interval, feature redundancy in the protein sequence can be avoided. This prevents the generation of too many or overly related features with small datasets, thus avoiding feature redundancy and preventing the model from repeatedly sampling highly correlated features during training, which could degrade model training performance. Consequently, model training efficiency can be improved, and the robustness of model predictions can be enhanced. Furthermore, it avoids the excessive computational consumption during later model training caused by protein sequence feature redundancy, thus saving computational resources, improving model training efficiency, and reducing model training costs.
[0075] In addition to using a combination of protein language models and temporal processing models as a predictive model for protein sequence attributes, embodiments of this application can also use the Transformer method to perform sequence feature extraction and temporal processing, thereby replacing the combination of protein language models and temporal processing models, such as replacing the combination of ESM and LSTM. This can also achieve multiple operation steps such as extracting multiple levels of feature representations of protein sequences and performing temporal processing on multiple levels of feature representations of protein sequences.
[0076] In this embodiment, the feature extraction network can extract feature vectors of multiple amino acids from the protein sequence in each training sample, and take the average value of the feature vectors of multiple amino acids to obtain the feature representation of the protein sequence.
[0077] When extracting the feature representation of a protein sequence, the tags that are pre-added to the protein sequence to indicate the start position of the amino acid sequence can be removed. For example, if the tag is a CLS tag, the CLS tag can be ignored first. The feature vector of each amino acid in the protein sequence can be extracted, and the average value of the feature vectors of multiple amino acids can be taken to obtain the feature representation of the protein sequence.
[0078] After model training is complete, cross-validation or independent test sets can be used to evaluate model performance, assessing its predictive accuracy and robustness. When evaluating model performance, various metrics can be used to assess the accuracy and robustness of the model in predicting protein properties, such as root mean square error (RMSE), coefficient of determination (r²), and classification accuracy (acc).
[0079] The following describes various implementation methods based on the technical concepts of the above embodiments of this application, one of which includes:
[0080] S1: Data collection.
[0081] The esol dataset used in GraphSol is employed, which consists of triples of <gene name, sequence, solubility value>. This dataset has been pre-split into training and testing sets and validated using five-fold cross-validation. In this implementation, the same splitting method is used for ease of evaluating model performance.
[0082] The ESM-650M model was used to extract features from the protein sequences in the dataset. To avoid potential feature redundancy, extraction was performed every other layer. Specifically, feature vectors based on all tokens (i.e., amino acids) were extracted from each layer, excluding the CLS label, and the average of the feature vectors of all amino acids was taken as the feature representation of the protein sequence.
[0083] S2: Model training.
[0084] An initial time series processing model is built and trained using the model mentioned in the previous step. The specific hyperparameters are: number of layers in the model network num_layer = 1, learning rate lr = 5e-4, MSE as the loss function, and Adam optimizer (Adaptive Moment Estimation) for training. To avoid overfitting, an early stopping strategy can be used to guide the parameter tuning process during model training.
[0085] S3: Model Evaluation.
[0086] The comparison between the protein attribute prediction method proposed in this embodiment and other protein attribute prediction methods is shown in the table below:
[0087]
[0088] Among them, RMSE is the loss value, and the lower the value, the better the model performance; for the other indicators, the higher the value, the better the model performance. It can be seen that the model proposed based on the above-mentioned embodiments of this application outperforms other models in almost all indicators, and the prediction model proposed in the embodiments of this application has higher accuracy and robustness in protein attribute prediction.
[0089] Here, precision represents accuracy; F1 score is the harmonic mean of precision and recall, which combines the model's accuracy and recall; and recall represents the model's ability to identify positive samples.
[0090] Based on the technical concepts of the above embodiments of this application, another implementation method includes:
[0091] S1: Data collection.
[0092] The dataset used in the HotProtein paper was organized into triplets of <protein name, sequence, thermal stability value>. This dataset has been pre-split for 10-fold cross-validation; however, to facilitate comparison of performance differences between multiple models, the original splitting method is used in this implementation.
[0093] The ESM-650M model was used to extract features from the protein sequences in the dataset. To avoid potential feature redundancy, extraction was performed every other layer. Specifically, feature vectors based on all tokens (i.e., amino acids) were extracted from each layer, excluding the CLS label, and the average of the feature vectors of all amino acids was taken as the feature representation of the protein sequence.
[0094] S2: Model building and training.
[0095] An initial time series processing model is built and trained using the model mentioned in the previous step. The specific hyperparameters are: number of layers in the model network num_layer = 1, learning rate lr = 1e-2, MSE as the loss function, and the AdamW optimizer for training. To avoid overfitting, an early stopping strategy can be used to guide the parameter tuning process during model training.
[0096] S3: Model Evaluation.
[0097] The performance comparison of the protein attribute prediction method proposed in this embodiment with other protein attribute prediction methods on the Hotprotein dataset is shown in the following table:
[0098] Performance on the s2c2 dataset:
[0099]
[0100] Performance on the s2c5 dataset:
[0101]
[0102] As can be seen, the model proposed based on the above embodiments of this application outperforms other models in almost all metrics. The prediction model proposed in the embodiments of this application has higher accuracy and robustness in protein property prediction.
[0103] Early stopping is a technique used in machine learning and deep learning to prevent model overfitting. It works by monitoring the model's performance on the validation set and immediately stopping training once performance begins to decline, reverting to the parameters of the previously best-performing model. This strategy not only helps improve the model's generalization ability on new data but also saves training time and reduces unnecessary computational costs.
[0104] The principle and operational steps are as follows: during training, the model performance is periodically evaluated on a validation set. If the model performance evaluated on the validation set does not improve for several consecutive epochs, training is stopped, and the parameters of the previously best-performing model are used. This method avoids the model overfitting noise and details on the training set.
[0105] Early stopping is a regularization method that can be used in conjunction with other regularization techniques (such as L1 and L2 regularization, dropout, etc.) to help improve the model's generalization ability and prevent overfitting.
[0106] Based on the above embodiments of this application, this application also proposes another embodiment of the protein property prediction method. Please refer to [link to embodiment]. Figure 4 This embodiment includes:
[0107] 401. Obtain a target prediction model, wherein the target prediction model includes a multi-layer feature extraction network, and the target prediction model is trained by a machine learning algorithm on multiple sets of training samples, wherein each set of training samples includes a protein sequence and the true attribute value of the protein sequence.
[0108] The method of this embodiment can be applied to computer devices. The target prediction model obtained by the computer device can be a target prediction model obtained by training an initial prediction model based on the above-mentioned multiple embodiments of the protein attribute prediction method in this application. That is, the target prediction model is obtained by training multiple sets of training samples by a machine learning algorithm. Each set of training samples includes a protein sequence and the true attribute value of the protein sequence. During training, the protein sequence can be extracted from multiple layers of feature extraction network based on the initial prediction model, and the multiple layers of feature representation of the protein sequence can be processed temporally to obtain the predicted attribute value of the protein sequence. The model parameters are adjusted based on the predicted attribute value and the true attribute value of the protein sequence until the convergence condition is met, and the model training is stopped to obtain the target prediction model.
[0109] 402. Input the protein sequence to be tested into the target prediction model, so that the feature extraction network at each level of the target prediction model extracts the feature representation corresponding to the protein sequence to be tested, obtains the feature representation of the protein sequence to be tested at multiple levels, processes the feature representation of the protein sequence to be tested at multiple levels, and outputs the predicted attribute value of the protein sequence to be tested.
[0110] The target prediction model can be used to predict the attributes of a protein sequence to be tested. That is, the protein sequence to be tested is input into the target prediction model, and the feature extraction network at each level of the target prediction model extracts the feature representation corresponding to the protein sequence to be tested, thereby obtaining the feature representation of the protein sequence to be tested at multiple levels. The feature representation of the protein sequence to be tested at multiple levels is then processed to output the predicted attribute value of the protein sequence to be tested.
[0111] The target prediction model can be a combination of a protein language model and a target temporal processing model. The protein language model includes a multi-level feature extraction network to extract multi-level feature representations of the protein sequence, such as an ESM model. The target temporal processing model is used to process the dependencies between the multi-level feature representations of the protein sequence and output the predicted attribute values of the protein sequence, such as a recurrent neural network model like an LSTM model.
[0112] Therefore, in this embodiment, by extracting multi-level feature information from the protein sequence, richer sequence information can be captured. Furthermore, temporal processing of this multi-level feature information effectively captures long-range dependencies, improving prediction accuracy and robustness. Moreover, through multi-level feature extraction and temporal processing, the model performs excellently on different datasets, exhibiting good generalization ability and improving the accuracy of protein attribute prediction while reducing prediction costs.
[0113] The protein property prediction method in this application focuses on multi-level information about proteins. Since protein properties can be divided into different levels, such as secondary structure information based on local amino acid interactions, isoelectric point information based on the whole sequence of amino acids, and active site information based on interactions between structurally similar amino acids, and these different properties are dispersed across different levels of the model, in this application embodiment, combining different intermediate layers to characterize the amino acid sequence during downstream task prediction can improve the performance of the downstream task model.
[0114] Based on this technical approach, this application provides a protein attribute prediction method based on time-series processing. It leverages the multi-level output of a protein language model and the time-series processing capabilities of the model to improve the accuracy and robustness of protein attribute prediction. Specifically, it fully utilizes the multi-level feature information of the protein sequence, extracting feature embeddings and other feature information from multiple levels of the protein sequence through a protein language model to capture deep-level features in the sequence; it strengthens time-series dependency processing, using time-series processing models such as LSTM to process the time-series data output by the protein language model, effectively capturing long-range dependencies in the protein sequence; and it improves prediction accuracy and generalization ability by experimentally validating and optimizing the model to enhance its prediction performance on different datasets, thereby improving the accuracy and robustness of the model's predictions.
[0115] The protein attribute prediction method in the embodiments of this application has been described above. The computer device in the embodiments of this application is described below. Please refer to [link / reference]. Figure 5 One embodiment of the computer device in this application includes:
[0116] The first acquisition unit is used to acquire a training set, which includes multiple sets of training samples, and each set of training samples includes a protein sequence and the true attribute value of the protein sequence.
[0117] The second acquisition unit is used to acquire an initial prediction model, which includes a feature extraction network with multiple layers.
[0118] The feature extraction unit is used to input the multiple sets of training samples into the initial prediction model, so that the feature extraction network at each level in the initial prediction model extracts the feature representation corresponding to the protein sequence in each set of training samples, thereby obtaining the feature representation of the protein sequence at multiple levels.
[0119] The model training unit is used to input the feature representation of each protein sequence at multiple levels and the true attribute value of the protein sequence into the initial prediction model, so that the initial prediction model processes the feature representation of the protein sequence at multiple levels to obtain the predicted attribute value of the protein sequence, and adjusts the model parameters of the initial prediction model based on the true attribute value and the predicted attribute value until the convergence condition is met and the model training stops to obtain the target prediction model.
[0120] The target prediction model is used to predict the attribute values of the protein sequence to be tested based on the feature representation of the protein sequence at multiple levels.
[0121] In a preferred embodiment of this invention, the second acquisition unit is specifically used for:
[0122] Obtain a protein language model and an initial temporal processing model; the protein language model includes a multi-layered feature extraction network.
[0123] The feature extraction unit is specifically used for:
[0124] The training set is input into the protein language model so that the feature extraction network at each level of the protein language model extracts the feature representation corresponding to the protein sequence in each group of training samples, thereby obtaining the feature representation of the protein sequence at multiple levels.
[0125] The model training unit is specifically used for:
[0126] The initial time-series processing model is input with the feature representations of each protein sequence at multiple levels and the true attribute values of the protein sequence. The initial time-series processing model processes the feature representations of the protein sequence at multiple levels to obtain the predicted attribute values of the protein sequence. The model parameters of the initial time-series processing model are adjusted based on the true attribute values and the predicted attribute values until the convergence condition is met, at which point the model training stops, and the target time-series processing model is obtained.
[0127] The protein language model is used to extract feature representations of the protein sequence to be tested at multiple levels, and the target time series processing model is used to predict the attribute values of the protein sequence to be tested based on the feature representations of the protein sequence to be tested at multiple levels.
[0128] In a preferred embodiment of this invention, the feature extraction unit is specifically used for:
[0129] The first-level feature extraction network in the protein language model extracts the feature representation of the protein sequence in each group of training samples. Then, the feature extraction network at every other level extracts the feature representation of the protein sequence in each group of training samples, resulting in multiple levels of feature representation for each protein sequence.
[0130] In a preferred embodiment of this invention, the model training unit is specifically used for:
[0131] For the first time step unit of the initial time-series processing model, the first-level feature representation of the protein sequence is used as the input of the first time step unit. The first-level feature representation of the protein sequence is calculated according to the preset activation function and preset operator in the first time step unit to obtain the cell state and hidden state of the first time step unit.
[0132] Each time step unit after the first time step unit of the initial time-series processing model is regarded as the current time step unit. The feature representation of the protein sequence at the same level as the current time step unit is used as the input of the current time step unit. At the same time, the cell state and hidden state output by the previous time step unit are used as the input of the current time step unit. The inputs are calculated according to the preset activation function and preset operators in the current time step unit to obtain the cell state and hidden state of the current time step unit.
[0133] The predicted attribute value of the protein sequence is calculated based on the output of the last time step unit of the initial time-series processing model.
[0134] In a preferred embodiment of this example, the protein language model includes an embedded sequence model (ESM), and the initial temporal processing model includes a long short-term memory network (LSTM).
[0135] In a preferred embodiment of this invention, the feature extraction unit is specifically used for:
[0136] The feature extraction network extracts feature vectors of multiple amino acids from the protein sequence in each group of training samples, and takes the average of the feature vectors of the multiple amino acids to obtain the feature representation of the protein sequence.
[0137] In this embodiment, the operations performed by each unit in the computer device are the same as those described above. Figures 1 to 2 The embodiments shown are similar and will not be repeated here.
[0138] In this embodiment, by extracting multi-level feature information from the protein sequence, richer sequence information can be captured. Furthermore, temporal processing of this multi-level feature information effectively captures long-range dependencies, improving prediction accuracy and robustness. Moreover, through multi-level feature extraction and temporal processing, the model performs excellently on different datasets, exhibiting good generalization ability and improving the accuracy of protein attribute prediction while reducing prediction costs.
[0139] Please see Figure 6 One embodiment of the computer device in this application includes:
[0140] An acquisition unit is used to acquire a target prediction model, the target prediction model including a multi-layer feature extraction network, the target prediction model being trained by a machine learning algorithm on multiple sets of training samples, each set of training samples including a protein sequence and the true attribute value of the protein sequence;
[0141] The prediction unit is used to input the protein sequence to be tested into the target prediction model, so that the feature extraction network at each level of the target prediction model extracts the feature representation corresponding to the protein sequence to be tested, obtains the feature representation of the protein sequence to be tested at multiple levels, processes the feature representation of the protein sequence to be tested at multiple levels, and outputs the predicted attribute value of the protein sequence to be tested.
[0142] In this embodiment, the operations performed by each unit in the computer device are the same as those described above. Figure 4 The embodiments shown are similar and will not be repeated here.
[0143] The computer device in the embodiments of this application is described below. Please refer to [link / reference]. Figure 7 One embodiment of the computer device in this application includes:
[0144] The computer device 700 may include one or more central processing units (CPUs) 701 and a memory 705, in which one or more applications or data are stored.
[0145] The memory 705 can be volatile or persistent storage. The program stored in the memory 705 can include one or more modules, each module including a series of instruction operations on the computer device. Furthermore, the central processing unit 701 can be configured to communicate with the memory 705 and execute the series of instruction operations stored in the memory 705 on the computer device 700.
[0146] The computer device 700 may also include one or more power supplies 702, one or more wired or wireless network interfaces 703, one or more input / output interfaces 704, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.
[0147] The central processing unit 701 can perform the aforementioned... Figures 1 to 2 , Figure 4 The specific operations performed by the computer device in the illustrated embodiment will not be described in detail here.
[0148] This application also provides a computer storage medium, one embodiment of which includes: the computer storage medium storing instructions, which, when executed on a computer, cause the computer to perform the aforementioned... Figures 1 to 2 , Figure 4 The operations performed by the computer device in the illustrated embodiment.
[0149] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0150] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.
[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0152] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0153] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A method of protein property prediction, characterized by, The method comprises: obtaining a training set comprising a plurality of groups of training samples, each group of the training samples comprising a protein sequence and a true attribute value of the protein sequence; obtaining an initial prediction model comprising a plurality of levels of feature extraction networks; inputting the plurality of groups of training samples into the initial prediction model, so that each level of feature extraction network in the initial prediction model extracts a feature representation corresponding to the protein sequence in each group of the training samples, respectively, to obtain a plurality of levels of feature representations of the protein sequence; inputting the plurality of levels of feature representations of each of the protein sequences and the true attribute value of the protein sequence into the initial prediction model, so that the initial prediction model processes the plurality of levels of feature representations of the protein sequence to obtain a predicted attribute value of the protein sequence, and adjusts model parameters of the initial prediction model based on the true attribute value and the predicted attribute value until a convergence condition is met to stop model training, to obtain a target prediction model; wherein the target prediction model is used to predict an attribute value of a to-be-tested protein sequence based on a plurality of levels of feature representations of the to-be-tested protein sequence.
2. The method of claim 1, wherein, The method comprises: obtaining a protein language model and an initial time sequence processing model; the protein language model comprises a plurality of levels of feature extraction networks; The method comprises: inputting the training set into the protein language model, so that each level of feature extraction network in the protein language model extracts a feature representation corresponding to the protein sequence in each group of the training samples, respectively, to obtain a plurality of levels of feature representations of the protein sequence; The method comprises: inputting the plurality of levels of feature representations of each of the protein sequences and the true attribute value of the protein sequence into the initial time sequence processing model, so that the initial time sequence processing model processes the plurality of levels of feature representations of the protein sequence to obtain a predicted attribute value of the protein sequence, and adjusts model parameters of the initial time sequence processing model based on the true attribute value and the predicted attribute value until a convergence condition is met to stop model training, to obtain a target time sequence processing model; wherein the protein language model is used to extract a plurality of levels of feature representations of a to-be-tested protein sequence, and the target time sequence processing model is used to predict an attribute value of the to-be-tested protein sequence based on the plurality of levels of feature representations of the to-be-tested protein sequence.
3. The method of claim 2, wherein, Each level of feature extraction network in the protein language model extracts a feature representation corresponding to the protein sequence in each group of the training samples, respectively, comprising: The first-level feature extraction network in the protein language model extracts a feature representation of the protein sequence in each set of the training samples, and then every other-level feature extraction network extracts a feature representation of the protein sequence in each set of the training samples by the feature extraction network at the level, to obtain a plurality of level feature representations of each protein sequence.
4. The method of claim 2, wherein, The processing of the plurality of level feature representations of the protein sequence to obtain the predicted attribute value of the protein sequence comprises: For a first time step unit of the initial time series processing model, taking the first-level feature representation of the protein sequence as the input of the first time step unit, calculating the first-level feature representation of the protein sequence according to the preset activation function and the preset operation symbol in the first time step unit, to obtain the cell state and the hidden state of the first time step unit; Each time step unit after the first time step unit of the initial time series processing model is regarded as a current time step unit, and a feature representation of the plurality of level feature representations of the protein sequence at the same level as the level of the current time step unit is taken as the input of the current time step unit, while the cell state and the hidden state output by the previous time step unit are taken as the input of the current time step unit, and each input is calculated according to the preset activation function and the preset operation symbol in the current time step unit, to obtain the cell state and the hidden state of the current time step unit; The predicted attribute value of the protein sequence is calculated according to the output of the last time step unit of the initial time series processing model.
5. The method according to any one of claims 2 to 4, characterized in that, The protein language model comprises an embedded sequence model (ESM) model, and the initial time series processing model comprises a long short-term memory (LSTM) model.
6. The method according to any one of claims 1 to 5, characterized in that, The feature extraction network extracts a feature representation of the protein sequence in each set of the training samples, which comprises: The feature extraction network extracts a feature vector of each amino acid in the protein sequence in each set of the training samples, and takes an average value of the feature vectors of the plurality of amino acids to obtain the feature representation of the protein sequence.
7. A method of protein property prediction, characterized by, The method comprises: obtaining a target prediction model, wherein the target prediction model comprises a plurality of level feature extraction networks, and the target prediction model is trained by a machine learning algorithm on a plurality of sets of training samples, each set of the training samples comprising a protein sequence and a real attribute value of the protein sequence; inputting a to-be-tested protein sequence into the target prediction model, so that each level feature extraction network in the target prediction model extracts a corresponding feature representation of the to-be-tested protein sequence, respectively, to obtain a plurality of level feature representations of the to-be-tested protein sequence, and processes the plurality of level feature representations of the to-be-tested protein sequence to output a predicted attribute value of the to-be-tested protein sequence.
8. A computer device, comprising: The computer device comprises: a first obtaining unit configured to obtain a training set, wherein the training set comprises a plurality of sets of training samples, and each set of the training samples comprises a protein sequence and a real attribute value of the protein sequence; The second acquisition unit is configured to acquire an initial prediction model, wherein the initial prediction model comprises a plurality of hierarchical feature extraction networks. The feature extraction unit is configured to input the plurality of groups of training samples into the initial prediction model, so that each hierarchical feature extraction network in the initial prediction model extracts a feature representation corresponding to a protein sequence in each group of the training samples, and obtains a plurality of hierarchical feature representations of the protein sequence. The model training unit is configured to input the plurality of hierarchical feature representations of each protein sequence and a real attribute value of the protein sequence into the initial prediction model, so that the initial prediction model processes the plurality of hierarchical feature representations of the protein sequence to obtain a predicted attribute value of the protein sequence, and adjusts model parameters of the initial prediction model based on the real attribute value and the predicted attribute value until a convergence condition is met to stop the model training, and obtain a target prediction model. The target prediction model is configured to predict an attribute value of a to-be-tested protein sequence based on a plurality of hierarchical feature representations of the to-be-tested protein sequence. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the method in any one of claims 1 to 7.
10. A computer storage medium, characterized in that, The computer storage medium stores instructions, and the instructions are executed on the computer to enable the computer to perform the method in any one of claims 1 to 7.