Protein phase separation tendency prediction method based on transfer learning and attention mechanism

Abstract

The invention relates to a protein phase separation tendency prediction method based on transfer learning and an attention mechanism. The method comprises the following steps: obtaining a to-be-identified protein sequence; converting the to-be-identified protein sequence into a feature matrix by using at least two language models; and inputting the feature matrix into a trained TransPS neural network containing an attention mechanism, and selecting a maximum value of an attention vector to obtain a prediction result of whether the protein is subjected to phase separation or not and a prediction result of a sequence section for driving phase separation. Compared with the prior art, the method has the advantages that the method has remarkable advantages in the aspect of prediction accuracy, input can be simplified, and the phase separation tendency and the specific sequence section can be predicted at the same time.

Description

Technical Field

[0001] The present invention belongs to the technical field of protein phase separation characteristic detection, and in particular relates to a method for predicting protein phase separation tendency based on transfer learning and attention mechanism. Background Art

[0002] Biomacromolecular condensates are ubiquitous in eukaryotic cells through phase separation (PS) and have multiple functions across multiple scales. For example, at the molecular level, they regulate the rates of biochemical reactions, such as photosynthesis and mRNA degradation, and participate in immune signaling and macromolecular folding. At the mesoscale, condensates regulate protein density distribution in neuronal synapses and nucleoli and help coordinate DNA damage responses. At the cellular level, condensates function by mediating subcellular localization and buffering random cellular noise. Disturbance of phase separation is closely related to a variety of diseases, including cancer and neurodegenerative diseases. Phase separation is mainly driven by multivalent interactions and multiple physicochemical forces. Both structured regions and intrinsically disordered regions (IDRs) are involved in driving phase separation. Recent studies on the conformational properties of IDRs have shown that the chain compactness of IDRs is closely related to their ability to drive phase separation. Given the large amount of uncharacterized sequence data, developing reliable phase separation prediction methods and identifying corresponding sequences are of great significance for a deeper understanding of the functional roles of biomacromolecular condensates and the pathogenesis of diseases.

[0003] To predict phase separation, early algorithms focused on specific protein sequence features. Recent more advanced integrated feature models combine multiple sequence features with new attributes such as immunofluorescence imaging, post-translational modifications (PTMs), protein-protein interactions (PPIs), and functional annotations. Recent models also have the ability to predict common phase separation and distinguish between phase-separated proteins and amyloid proteins.

[0004] Despite this, many recent high-performing models still rely heavily on features extracted from various sources such as protein annotation and imaging data, which limits their generalization ability on proteins that lack detailed annotations, including variants and isoforms. In addition, current methods that provide residue-level phase separation (PS) scores typically focus on only one or two specific sequence features, making it difficult to accurately predict the regions that drive phase separation because the sequence patterns involved are complex and diverse. These limitations have hampered progress in uncovering key biological mechanisms, highlighting the need to develop more flexible and broadly applicable models. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a protein phase separation tendency prediction method based on transfer learning and attention mechanism with high accuracy and strong generalization, which can simultaneously predict whether the protein will undergo phase separation and the sequence segment that drives the phase separation.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A method for predicting protein phase separation tendency based on transfer learning and attention mechanism, comprising the following steps:

[0008] Obtaining the protein sequence to be identified;

[0009] Using at least two language models to convert the protein sequence to be identified into a feature matrix;

[0010] The feature matrix is ​​input into a trained TransPS neural network with an attention mechanism to obtain prediction results of whether the protein undergoes phase separation and the sequence segments that drive phase separation.

[0011] Furthermore, the language model includes a protein language model ESM2 and a language model trained based on molecular dynamics simulation data.

[0012] Furthermore, the transformation of the feature matrix specifically includes:

[0013] The protein sequence to be identified is converted into a first matrix using a protein language model ESM2;

[0014] Taking the protein sequence to be identified as input, obtaining the hidden layer output of the language model trained based on molecular dynamics simulation data, and obtaining a second matrix;

[0015] The first matrix and the second matrix are concatenated to obtain the characteristic matrix.

[0016] Furthermore, the language model trained based on molecular dynamics simulation data is constructed using an LSTM model.

[0017] Furthermore, the feature matrix is ​​a matrix containing language coding features and protein conformational features, and the protein conformational features include multiple ones of asphericity, end-to-end distance, and radius of gyration.

[0018] Furthermore, the TransPS neural network includes an average pooling layer and a trainable fully connected neural network, and the processing process of the trainable fully connected neural network includes:

[0019] Performing average pooling processing on the input feature matrix through an average pooling layer to obtain an averaged feature matrix;

[0020] The averaged feature matrix is ​​processed using a trainable fully connected neural network to generate a residue-level phase separation tendency score for the sequence length and a protein phase separation tendency prediction result, and the sequence segment driving the phase separation is determined based on the residue-level phase separation tendency score.

[0021] Furthermore, the trainable fully connected neural network includes an initial layer, a hidden layer, a final output layer and a maximum pooling layer connected in sequence, and the residue-level phase separation tendency score of the sequence length is obtained based on the output of the final output layer, and the protein phase separation tendency prediction result is obtained based on the output of the maximum pooling layer.

[0022] Furthermore, a LeakyReLU activation function is arranged after the initial layer, and a Sigmoid activation function is arranged after the final output layer.

[0023] The present invention also provides a computer-readable storage medium, comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the protein phase separation tendency prediction method based on transfer learning and attention mechanism as described above.

[0024] The present invention also provides an electronic device comprising one or more processors, a memory and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the protein phase separation tendency prediction method based on transfer learning and attention mechanism as described above.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. The present invention uses protein sequence as input to accurately obtain the prediction results of whether the protein undergoes phase separation and the prediction results of the sequence segment that drives phase separation, without requiring complex engineering features such as protein structure, post-translational modification, immunofluorescence images and other information that other methods rely on, effectively simplifying the input conditions and making the solution simpler.

[0027] 2. The present invention utilizes at least two language models to perform feature transformation on the input protein sequence, and fine-tunes the prediction application in the field of phase separation through transfer learning, thereby effectively improving the prediction reliability.

[0028] 3. The present invention can accurately predict the tendency of protein phase separation and the driving section of protein phase separation at the same time through a trainable fully connected neural network, and the prediction results are comprehensive.

[0029] 4. The present invention can predict spontaneous phase separation or chaperone-dependent phase separation proteins, and has strong generalizability. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the principle of the present invention;

[0031] Figure 2 The performance of the present invention in predicting different types of phase separation proteins is compared with the performance of existing phase separation prediction methods (including PhaSePred, PSPHunter, PLAAC, PSAP, DeePhase, catGRANULE, PScore, ParSe and FuzDrop) and TransPS on the phase separation independent validation set, wherein A is the AUC curve of TransPS, B is the AUC performance comparison of TransPS and existing algorithms, C is the AUPR curve of TransPS, and D is the AUPR performance comparison of TransPS and existing algorithms;

[0032] Figure 3 This is a schematic diagram of the experimental results of different protein phase separations (including spontaneous phase separation SaPS and chaperone-dependent phase separation PdPS) according to the embodiments of the present invention, demonstrating the consistency between the residue-level predictions made by TransPS and the experimentally-validated PS region. DETAILED DESCRIPTION

[0033] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0034] refer to Figure 1 As shown, this embodiment provides a method for predicting the phase separation tendency of proteins based on transfer learning and attention mechanism, including the following steps: obtaining a protein sequence to be identified; using at least two language models to convert the protein sequence to be identified into a feature matrix, the language model may include a protein language model ESM2 and a language model trained based on molecular dynamics simulation data, etc., applying the large protein language model and the learning model for predicting the simulated conformation of protein molecules in the field of phase separation and fine-tuning; inputting the feature matrix into a trained TransPS (Transfer-learning based PS prediction) neural network containing an attention mechanism to obtain the prediction results of whether the protein undergoes phase separation and the prediction results of the sequence segments driving the phase separation. Through the above method, a prediction result with high accuracy can be obtained, the input can be simplified, and the phase separation tendency and specific sequence segments can be predicted at the same time.

[0035] like Figure 1As shown, the above method as a whole can be implemented based on a TransPS machine learning model. In this embodiment, the TransPS machine learning model includes a protein language model ESM2, a language model trained based on molecular dynamics simulation data, and a TransPS neural network. The construction and training of the model include data collection, feature construction, algorithm architecture design, model training, and performance evaluation. Specifically:

[0036] (1) Data Collection

[0037] We used the datasets used for training and evaluating PhaSePred, which are from the supplementary materials of their publications. The acquired data include 201 spontaneously phase-separated proteins (SaPS), 327 chaperone-dependent phase-separated proteins (PdPS), and 60,220 background proteins (NoPS). We also used the training and testing dataset (Mix) of PSPHunter, which contains 136 human phase-separated proteins, 375 non-human phase-separated proteins, 6259 human background proteins, and 17,268 non-human background proteins. These data were previously deduplicated using CD-Hit, with a threshold of 0.4.

[0038] (2) Feature Construction

[0039] This example generates two embedding matrices for each protein. First, based on the characteristics of the large language model of proteins: the embedding calculated using the ESM-2 model, the esm2_t6_8M_UR50D model (8 million parameter version, referred to as ESM2-8M) in the Python library 'esm' is selected. The sequence is input into the ESM2-8M model, and the representation of the sixth layer is used as the output to generate a 320-dimensional vector for each sequence position, forming the first matrix of the sequence length multiplied by 320.

[0040] Secondly, based on the feature construction of ALBATROSS: The ALBATROSS model trains the LSTM-BRNN model based on the molecular dynamics simulation data of IDR to predict the conformational properties of IDR. We use three sub-models: "asphericity", "radius_of_gyration_scaled" and "end_to_end_distance_scaled". After the sequence is input into each sub-model, the hidden layer output of each time step is extracted to obtain a vector of sequence length ×110, and these values ​​range from -1 to 1. Then the vectors of the three sub-models are concatenated into an embedding vector of sequence length ×330, forming a second matrix of sequence length multiplied by 330.

[0041] Finally, the first matrix and the second matrix are concatenated to obtain a feature matrix whose size is the sequence length multiplied by 650.

[0042] (3) TransPS algorithm structure

[0043] The core structure of the TransPS algorithm consists of a TransPS neural network (TransPS-Scan model) that takes as input a feature matrix of size sequence length × 650 for each protein sequence, where the 650 dimensions include a 320-dimensional language model encoding and a 330-dimensional IDR conformation encoding. The model is implemented using the 'pytorch' library in Python. In each TransPS-Scan module: The first layer is an average pooling layer with "same" padding. For each protein position, a local window of the feature matrix is ​​averaged over the residue dimension to generate a 650-dimensional vector. To reduce the effect of "same" padding, a scaling step is added to the end of the sequence to adjust the values ​​of these positions to ensure that they reflect the average of the actual sequence elements rather than padded zero values. The averaged vector is then processed using a trainable multilayer perceptron (MLP) kernel (a trainable fully connected neural network) with shared parameters. The MLP core consists of an initial layer (20 neurons) followed by a LeakyReLU activation function; a hidden layer (5 neurons); and a final output layer (1 neuron), and uses a Sigmoid activation function to constrain the score of each residue position to between 0 and 1. The score of each residue position forms an attention vector of the sequence length. The last layer is a maximum pooling layer, which selects the maximum value of the attention vector as the prediction of phase separation tendency at the protein level, and optimizes the MLP parameters by backpropagation.

[0044] (4) Model training and performance evaluation

[0045] This example uses "kaiming_normal" to initialize network parameters, applies regularization to penalize weights, and sets the penalty coefficient α to 0.003. The optimizer uses Adam, the learning rate is set to 0.003, and no hyperparameter tuning is performed. During training, at each iteration (50 epochs in total), a subset of proteins equal in size to the positive sample set is randomly selected from the background data set as a negative sample set to solve the problem of sample imbalance.

[0046] Figure 2 It is shown that the above model exhibits excellent performance on an independent validation set for phase separation prediction. Figure 3 It was shown that the phase separation segment predicted by the model was highly consistent with the experimentally verified results, and the phase separation driving segment of different proteins could be predicted.

[0047] The experimental results show that the present invention can accurately predict the proteins that affect phase separation and can accurately predict the protein fragments involved in phase separation.

[0048] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0049] In another embodiment, an electronic device may be provided, including one or more processors, a memory, and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the prediction method as described above.

[0050] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for predicting protein phase separation tendency based on transfer learning and attention mechanism, characterized in that: The following steps are involved: Obtaining the protein sequence to be identified; Using at least two language models to convert the protein sequence to be identified into a feature matrix; The feature matrix is ​​input into a trained TransPS neural network with an attention mechanism to obtain prediction results of whether the protein undergoes phase separation and the sequence segments that drive phase separation.

2. The method for predicting protein phase separation tendency based on transfer learning and attention mechanism according to claim 1, characterized in that: The language model includes a protein language model ESM2 and a language model trained based on molecular dynamics simulation data.

3. The protein phase separation tendency prediction method based on transfer learning and attention mechanism according to claim 2, characterized in that: The transformation of the feature matrix specifically includes: The protein sequence to be identified is converted into a first matrix using a protein language model ESM2; Taking the protein sequence to be identified as input, obtaining the hidden layer output of the language model trained based on molecular dynamics simulation data, and obtaining a second matrix; The first matrix and the second matrix are concatenated to obtain the feature matrix.

4. The method for predicting protein phase separation tendency based on transfer learning and attention mechanism according to claim 2, characterized in that: The language model trained based on molecular dynamics simulation data is constructed using an LSTM model.

5. The method for predicting protein phase separation tendency based on transfer learning and attention mechanism according to claim 1, characterized in that: The feature matrix is ​​a matrix containing language coding features and protein conformational features, and the protein conformational features include multiple ones of asphericity, end-to-end distance, and radius of gyration.

6. The method for predicting protein phase separation tendency based on transfer learning and attention mechanism according to claim 1, characterized in that: The TransPS neural network includes an average pooling layer and a trainable fully connected neural network. The processing process of the trainable fully connected neural network includes: Performing average pooling processing on the input feature matrix through an average pooling layer to obtain an averaged feature matrix; The averaged feature matrix is ​​processed using a trainable fully connected neural network to generate a residue-level phase separation tendency score for the sequence length and a protein phase separation tendency prediction result, and the sequence segment driving the phase separation is determined based on the residue-level phase separation tendency score.

7. The method for predicting protein phase separation tendency based on transfer learning and attention mechanism according to claim 6, characterized in that: The trainable fully connected neural network includes an initial layer, a hidden layer, a final output layer and a maximum pooling layer connected in sequence. The residue-level phase separation tendency score of the sequence length is obtained based on the output of the final output layer, and the protein phase separation tendency prediction result is obtained based on the output of the maximum pooling layer.

8. The method for predicting protein phase separation tendency based on transfer learning and attention mechanism according to claim 7, characterized in that: A LeakyReLU activation function is set after the initial layer, and a Sigmoid activation function is set after the final output layer.

9. A computer-readable storage medium, characterized in that: It includes one or more programs for execution by one or more processors of an electronic device, and the one or more programs include instructions for executing the protein phase separation tendency prediction method based on transfer learning and attention mechanism as described in any one of claims 1-8.

10. An electronic device, characterized in that: It includes one or more processors, a memory and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the protein phase separation tendency prediction method based on transfer learning and attention mechanism as described in any one of claims 1-8.

Images