Protein function prediction method fusing multi-view information and self-supervised learning

Abstract

The invention provides a protein function prediction method fusing multi-view information and self-supervised learning. Protein sequence features are obtained. A protein contact graph is generated by using a direct generation method and a deep learning method, and protein structure features are generated from the protein contact graph through a graph convolution model. And carrying out feature extraction on the protein sequence features and the protein structure features through comparative self-supervised learning and generative self-supervised learning. And minimizing the contrast type self-supervised learning loss function and the generative self-supervised learning loss function. And splicing the protein sequence features and the protein structure features which are finally updated, and predicting the protein function through a multi-layer perceptron. According to the technical scheme, the problems that in the prior art, a part of protein molecules can only obtain sequence information but cannot obtain structure information, and the protein three-dimensional structure data volume capable of being applied to drug target affinity prediction is small are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of protein function prediction, and in particular to a protein function prediction method integrating multi-view information and self-supervised learning. Background Art

[0002] Protein function prediction is of great significance in biological and medical research. Predicting protein function through artificial intelligence methods is crucial to understanding biological processes and can better understand the complex processes occurring within cells, such as metabolism, signal transduction, and gene expression regulation. By predicting the function of proteins, researchers can identify potential drug targets and develop treatments for these targets. This plays a key role in discovering new drugs.

[0003] The key to predicting protein function is how to accurately extract protein features. The primary sequence of a protein is represented by amino acids, and there are many methods for extracting protein features based on amino acid sequences. DeepGOPlus extracts features from amino acid sequences through convolutional neural networks, and combines prediction methods based on sequence similarity to achieve protein feature extraction and function prediction. DeepAdd uses the dependency of protein classification as background information to build a deep learning model and learn protein features from amino acid sequences. MultiPredGO extracts features by fusing two different types of information, protein sequence and structure, generates a three-dimensional grid for the protein structure, and constructs four channels based on the type of protein atoms. Convolutional neural networks are used to calculate the attribute values ​​of different channel grids to generate protein multi-channel descriptors. Some scholars have also studied graph convolution models based on protein structures. However, whether it is three-dimensional grid calculation or graph convolution calculation, the premise is that the three-dimensional structure of the protein must be obtained. However, there are still a large number of proteins whose three-dimensional structures are unknown, especially for drug target affinity prediction databases, where the three-dimensional structure data is even scarcer.

[0004] With the emergence of self-supervised learning models, a large number of unlabeled data sets can be utilized through self-supervised learning methods, thereby reducing the need for labeled samples. With the development of large-scale self-supervised pre-training models in the field of natural language processing, related methods have been used to train protein language models. Based on the BERT model, ProteinBERT simultaneously adopts bidirectional language modeling and function prediction tasks of protein sequences as pre-training strategies, and provides character-level and sequence-level feature representations. Rives et al. used self-supervised learning to train contextual language models for 86 billion amino acids in 250 million protein sequences. CPCPro applies the principle of maximizing mutual information between local information and global information to protein feature extraction, divides protein sequences into fixed-size fragments, and uses autoregressive models to distinguish subsequent fragments from the same protein and fragments of random proteins. However, these pre-trained protein language models do not consider the structural information of proteins.

[0005] At present, most protein molecules can only obtain their sequence information but not their structural information, and the amount of protein three-dimensional structure data that can be used for drug target affinity prediction is even smaller. However, the three-dimensional structure of proteins is of great significance for function prediction. Therefore, how to effectively use the three-dimensional structure information and combine it with the existing large amount of protein sequence information to improve the robustness and generalization of protein characteristics is also a key scientific problem that must be solved in protein function prediction.

[0006] Therefore, there is a need for a protein function prediction method that integrates multi-view information and self-supervised learning, which can obtain protein structure information and then be applied to drug target affinity prediction. Summary of the invention

[0007] The main purpose of the present invention is to provide a protein function prediction method that integrates multi-view information and self-supervised learning to solve the problem in the prior art that only sequence information but no structural information can be obtained for some protein molecules, and the amount of protein three-dimensional structure data that can be applied to drug target affinity prediction is small.

[0008] To achieve the above object, the present invention provides a protein function prediction method integrating multi-view information and self-supervised learning, which specifically comprises the following steps:

[0009] S1, the BiLSTM model is used to extract the correlation between the amino acid sequence and the protein sequence features through a fully connected layer.

[0010] S2, uses direct generation method and deep learning method to generate protein contact map, and generates protein structural features from the protein contact map through the graph convolution model.

[0011] S3, takes protein sequence features and protein structure features as input of contrastive self-supervised learning, and performs feature extraction through contrastive self-supervised learning.

[0012] S4, takes protein sequence features and protein structure features as input of generative self-supervised learning, and performs feature extraction through generative self-supervised learning.

[0013] S5, minimizes the contrastive self-supervised learning loss function and the generative self-supervised learning loss function to achieve the goal of self-supervised learning.

[0014] S6, the final updated protein sequence features and protein structure features are spliced ​​together, and then the protein function is predicted through a multi-layer perceptron.

[0015] Furthermore, step S1 specifically includes the following steps:

[0016] S1.1, encode three consecutive amino acids as a word, and randomly initialize each word into a one-dimensional vector.

[0017] S1.2, the BiLSTM model is used to extract the correlation between the previous and next sequences.

[0018] S1.3, protein sequence features are obtained through a fully connected layer.

[0019] Furthermore, step S2 specifically includes the following steps:

[0020] S2.1, Direct generation of protein contact maps for known protein structures.

[0021] S2.2, for unknown protein structures, use deep learning methods to generate protein contact maps.

[0022] S2.3, extracting the position-specific matrix of amino acid residues for each amino acid residue in the protein contact map, inputting the position-specific matrix and the protein contact map into the graph convolution model to extract protein structural features.

[0023] Furthermore, step S3 specifically includes the following steps:

[0024] S3.1, protein sequence features and protein structure features from the same protein are considered as positive pairs, and protein sequence features and protein structure features from different proteins are considered as negative pairs.

[0025] S3.2, adopt the multi-view contrastive learning method based on noise contrast estimation NCE to solve the noise contrast estimation NCE, the loss function L of contrastive self-supervised learning 1 for:

[0026]

[0027] Among them, x i and i represent the sequence characteristics and structural characteristics of the ith protein, respectively, N is the number of proteins, sim(x i ,y i ) is x i With y i The similarity between them, τ is the temperature parameter used to control the degree of score smoothing, exp() is the exponential function, and log() is the logarithmic function.

[0028] Furthermore, step S4 specifically includes the following steps:

[0029] S4.1, when generating protein structure features from corresponding protein sequence features, a reparameterized latent variable function z is introduced x =μ x +σ x ⊙ε, where ε∈N(0,1), μ x and σ x are two learnable functions about the protein sequence feature x, ⊙ is the element product; when reconstructing the protein sequence feature from the corresponding protein structure feature, the hidden variable function z is introduced y =μ y +σ y ⊙ε. Where ε∈N(0,1), μ y and σ y They are two learnable functions about the protein structural feature y.

[0030] S4.2. Loss function L for generative self-supervised learning 2 for:

[0031]

[0032] Among them, z is a hidden variable, represents the expected value, q φ (z|x) represents mapping protein sequence feature x to latent variable z; p θ (y|z) represents the generation of protein structure feature y from latent variable z; KL represents divergence, p(z) represents the prior distribution, which is the standard normal distribution N(0,1), β represents the balance coefficient; q φ (z|y) represents mapping protein structural feature y to latent variable z; p θ (x|z) represents the generation of protein sequence feature x from latent variable z.

[0033] Further, step S5 is as follows:

[0034] L=L 1+αL 2 ;

[0035] Among them, L is the self-supervised learning objective function and α is the weight parameter.

[0036] The present invention has the following beneficial effects:

[0037] The present invention embeds protein structure information into sequence features by integrating contrastive and generative self-supervised learning methods, solves the problem of low generalization of protein features, constructs a protein feature extraction model that integrates multi-view information, and provides reliable guarantee for protein function prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the specific implementation or the prior art description. Obviously, the drawings described below are some implementations of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0039] Figure 1 The flowchart of the protein sequence feature extraction method of step S1 of the present invention is shown.

[0040] Figure 2 The flowchart of the protein structure feature extraction method in step S2 of the present invention is shown.

[0041] Figure 3 A flow chart of the protein self-supervised learning method of steps S3 and S4 of the present invention is shown. DETAILED DESCRIPTION

[0042] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] like Figure 1 A protein function prediction method integrating multi-view information and self-supervised learning is shown, which specifically includes the following steps:

[0044] S1, the BiLSTM model is used to extract the correlation between the amino acid sequence and the protein sequence features through a fully connected layer.

[0045] S2, uses direct generation method and deep learning method to generate protein contact map, and generates protein structural features from the protein contact map through the graph convolution model.

[0046] S3, takes protein sequence features and protein structure features as input of contrastive self-supervised learning, and performs feature extraction through contrastive self-supervised learning.

[0047] S4, takes protein sequence features and protein structure features as input of generative self-supervised learning, and performs feature extraction through generative self-supervised learning.

[0048] S5, minimizes the contrastive self-supervised learning loss function and the generative self-supervised learning loss function to achieve the goal of self-supervised learning.

[0049] S6, the final updated protein sequence features and protein structure features are spliced ​​together, and then the protein function is predicted through a multi-layer perceptron.

[0050] In response to the problems that existing protein features lack structural information and have low function prediction accuracy, the present invention proposes a feature representation method based on protein sequence and structure, effectively utilizes different types of protein representation methods, realizes the acquisition and representation of protein multi-view information, and solves the consistency problem of views in different dimensions; proposes a protein feature extraction method based on comparative self-supervised learning, extracts protein features by mining the differences and similarities between protein sequence and structural representation; proposes a protein feature extraction method based on generative self-supervised learning, develops a feature reconstruction method for protein sequence and structure, mines the internal information of protein data itself, and finally embeds the structural information enhancement into the sequence feature representation, thereby ultimately improving the accuracy of protein function prediction.

[0051] Specifically, since each protein is composed of an amino acid sequence, and the 20 amino acids that make up all proteins are represented by 20 letters, natural language processing methods can be used to extract features from amino acid sequences. In order to more effectively utilize the relationship between amino acids, three consecutive amino acids are generally encoded as a word, and the encoding dictionary contains 20 3 Three amino acid words are randomly initialized into a one-dimensional vector for each word, and the BiLSTM model is used to extract the correlation between the sequence before and after, and a fully connected layer is used to obtain the feature representation of the protein sequence. The extraction method is as follows Figure 1 shown.

[0052] Step S1 specifically includes the following steps:

[0053] S1.1, encode three consecutive amino acids as a word, and randomly initialize each word into a one-dimensional vector.

[0054] S1.2, the BiLSTM model is used to extract the correlation between the previous and next sequences.

[0055] S1.3, obtain protein sequence features through a fully connected layer.

[0056] Specifically, the protein itself is a long chain. Due to the interaction forces between different residues (such as hydrogen bonds, van der Waals forces, etc.), it will curl and fold to form proteins of various structures. The protein structure analysis process is complicated. There are still many proteins that have sequence information but lack structural information. However, the extraction of protein structure information is of great significance for protein function prediction. Among the protein structure representation methods, the protein contact map is a very important form, which is used to measure the probability matrix of interactions between different residue pairs. For known protein structures, contact maps can be generated directly. For unknown protein three-dimensional structures, existing deep learning methods such as PconsC4 and AlphaFold can be used to obtain them. After obtaining the protein contact map, it can be regarded as an adjacency matrix of the graph structure, and the position-specific scoring matrix (Position-Specific Scoring Matrix) is extracted for each amino acid residue as a feature. The protein structure features are extracted by the graph convolution method. The process is as follows: Figure 2 shown.

[0057] Step S2 specifically includes the following steps:

[0058] S2.1, Direct generation of protein contact maps for known protein structures.

[0059] S2.2, for unknown protein structures, use deep learning methods to generate protein contact maps.

[0060] S2.3, extracting the position-specific matrix of amino acid residues for each amino acid residue in the protein contact map, inputting the position-specific matrix and the protein contact map into the graph convolution model to extract protein structural features.

[0061] Specifically, step S3 includes the following steps:

[0062] S3.1, protein sequence features and protein structure features from the same protein are considered as positive pairs, and protein sequence features and protein structure features from different proteins are considered as negative pairs.

[0063] S3.2, adopt the multi-view contrastive learning method based on noise contrast estimation NCE to solve the noise contrast estimation NCE, the loss function L of contrastive self-supervised learning 1 for:

[0064]

[0065] Among them, xi and i represent the sequence characteristics and structural characteristics of the ith protein, respectively, N is the number of proteins, sim(x i ,y i ) is x i With y i The similarity between them, τ is the temperature parameter used to control the degree of score smoothing, exp() is the exponential function, log() is the logarithmic function, and the loss function L of the contrastive self-supervised learning is 1 By maximizing the positive sample pairs (sequence and structural features x of the same protein i ,y i ) and minimize the similarity score of negative sample pairs (feature pairs x between different proteins i ,y j ), so that the model can bring the features of positive samples closer and effectively distinguish negative samples, improving the model's ability to distinguish.

[0066] Specifically, step S4 includes the following steps:

[0067] S4.1, when generating protein structure features from corresponding protein sequence features, a reparameterized latent variable function z is introduced x =μ x +σ x ⊙ε, where ε∈N(0,1), μ x and σ x They are two learnable functions about the protein sequence feature x, which are used to calculate the mean and standard deviation of the latent variable. ⊙ is the element product. When reconstructing the protein sequence feature from the corresponding protein structure feature, the latent variable function z y =μ y +σ y ⊙ε, where ε∈N(0,1), μ y and σ y Respectively represent two learnable functions about the protein structure feature y. Through the reparameterization of the latent variable function, the input protein sequence features and sequence features are mapped to the latent space respectively, and then the decoder is used to generate the corresponding protein structure features and sequence features according to the latent variables. The whole process is achieved by minimizing the loss function of the variational autoencoder. The loss function includes the reconstruction error between the protein generated structure information and the protein true structure information, as well as the KL divergence between the latent variable distribution and the prior distribution, thereby realizing the mapping and generation of protein sequence features to protein structure features.

[0068] S4.2. Loss function L for generative self-supervised learning 2 for:

[0069]

[0070] Among them, z is a hidden variable, represents the expected value, q φ (z|x) represents mapping protein sequence feature x to latent variable z; p θ (y|z) represents the generation of protein structure feature y from latent variable z; KL represents divergence, p(z) represents the prior distribution, which is the standard normal distribution N(0,1), β represents the balance coefficient; q φ (z|y) represents mapping protein structural feature y to latent variable z; p θ (x|z) represents the generation of protein sequence feature x from latent variable z. Direct integration is not feasible, so the re-parameter introduced in S4.1 separates the sampling of z into z x =μ x +σ x ⊙ε, where ε∈N(0,1), and by sampling multiple ε, the expected value is approximated. KL represents the divergence, [-logpθ(y|z)] is the reconstruction loss, which is used to measure the difference between the protein structure sequence generated by the decoder and the true protein structure sequence y. [KL(q φ (z|c)∥p(z))] is the KL divergence, which is used to constrain the latent variable distribution q φ (z|x) is close to the predefined prior distribution p(z), p(z) is the standard normal distribution N(0,1), and β is a balancing coefficient used to adjust the relative importance of reconstruction loss and KL divergence. Minimize L 2 , which can achieve higher accuracy in generating protein structural features while ensuring good regularization of the latent space.

[0071] Specifically, step S5 is as follows:

[0072] L=L 1 +αL 2 ;

[0073] Among them, L is the self-supervised learning objective function and α is the weight parameter.

[0074] In order to more effectively extract features based on protein sequence and structure information, such as Figure 3As shown, the present invention proposes a multi-view based self-supervised training method, which takes sequence and structure as two complementary views of each protein. Protein sequence features are extracted through a BiLSTM network and used as input for contrastive self-supervised learning and generative self-supervised learning. Protein structure features are extracted through a GNN network and also used as input for contrastive self-supervised learning and generative self-supervised learning. By using self-supervised learning methods between these views, structural information is enhanced into the sequence representation, thereby better reflecting the properties of the protein. The model implements self-supervised learning through sequence and structure information in the pre-training stage, and the pre-trained sequence model is then fine-tuned in protein function prediction.

[0075] In the pre-training stage, two self-supervised learning tasks are used for feature extraction, namely contrastive and generative tasks. Contrastive self-supervised learning can learn the local features of molecules at the data level, while generative self-supervised learning can directly learn the global data density function. Therefore, combining the two can more effectively mine protein features.

[0076] First, the main task of contrastive self-supervised learning is to define positive pairs and negative pairs. For this task, the sequence and structural features of the same protein are regarded as positive pairs, and the sequences and structures from different proteins are regarded as negative pairs for contrastive learning.

[0077] Secondly, the purpose of generative self-supervised learning is to learn effective feature representation by reconstructing each data. For proteins, each protein sequence has a certain number of structural conformations, so the goal of this stage is to train an efficient sequence feature that can restore its corresponding structural features to the greatest extent. In this way, the sequence encoder contains key structural information and improves the performance of downstream tasks.

[0078] Finally, the updated protein sequence feature P seq and protein structural features P stru Perform splicing operations and use a multi-layer perceptron to predict protein functions.

[0079] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A protein function prediction method integrating multi-view information and self-supervised learning, characterized in that: The specific steps include: S1, uses the BiLSTM model to extract the correlation between the amino acid sequence and the protein sequence features through a fully connected layer; S2, using direct generation method and deep learning method to generate protein contact map, and then use graph convolution model to generate protein structural features; S3, taking protein sequence features and protein structure features as inputs of contrastive self-supervised learning, and extracting features through contrastive self-supervised learning; S4, taking protein sequence features and protein structure features as inputs of generative self-supervised learning, and extracting features through generative self-supervised learning; S5, minimizes the contrastive self-supervised learning loss function and the generative self-supervised learning loss function; S6, the final updated protein sequence features and protein structure features are spliced ​​together, and then the protein function is predicted through a multi-layer perceptron.

2. A protein function prediction method integrating multi-view information and self-supervised learning according to claim 1, characterized in that: Step S1 specifically includes the following steps: S1.1, encode the three consecutive amino acids as a word, and randomly initialize each word into a one-dimensional vector; S1.2, using BiLSTM model to extract the correlation between the previous and next sequences; S1.3, protein sequence features are obtained through a fully connected layer.

3. The protein function prediction method integrating multi-view information and self-supervised learning according to claim 1, characterized in that: Step S2 specifically includes the following steps: S2.1, direct generation of protein contact maps for known protein structures; S2.2, for unknown protein structures, generate protein contact maps using deep learning methods; S2.3, extracting the position-specific matrix of amino acid residues for each amino acid residue in the protein contact map, inputting the position-specific matrix and the protein contact map into the graph convolution model to extract protein structural features.

4. The protein function prediction method integrating multi-view information and self-supervised learning according to claim 1, characterized in that: Step S3 specifically includes the following steps: S3.1, protein sequence features and protein structure features from the same protein are considered as positive pairs, and protein sequence features and protein structure features from different proteins are considered as negative pairs; S3.2, a multi-view contrastive learning method based on noise contrast estimation NCE is used to solve the noise contrast estimation NCE. The loss function L1 of contrastive self-supervised learning is: Among them, x i and i represent the sequence characteristics and structural characteristics of the ith protein, respectively, N is the number of proteins, sim(x i ,y i ) is x i With y i The similarity between them, τ is the temperature parameter used to control the degree of score smoothing, exp() is the exponential function, and log() is the logarithmic function.

5. The protein function prediction method integrating multi-view information and self-supervised learning according to claim 1, characterized in that: Step S4 specifically includes the following steps: S4.1, when generating protein structure features from corresponding protein sequence features, a reparameterized latent variable function z is introduced x =μ x +σ x ⊙ε, where ε∈N(0,1), μ x and σ x They represent two learnable functions about the protein sequence feature x, ⊙ is the element product; when reconstructing the protein sequence feature from the corresponding protein structure feature, the latent variable function z y =μ y +σ y ⊙ε, where ε∈N(0,1), μ y and σ y Respectively represent two learnable functions about the protein structural feature y; S4.2, the loss function L2 of generative self-supervised learning is: Among them, z is a hidden variable, represents the expected value, q φ (z|x) represents mapping protein sequence feature x to latent variable z; p θ (y|z) represents the generation of protein structure feature y from latent variable z; KL represents divergence, p(z) represents the prior distribution, which is the standard normal distribution N(0,1), β represents the balance coefficient; q φ (z|y) represents mapping protein structural feature y to latent variable z; p θ (c|z) represents the generation of protein sequence features x from latent variables z.

6. The protein function prediction method integrating multi-view information and self-supervised learning according to claim 1, characterized in that: Step S5 is as follows: L = L1 + αL2; Among them, L is the self-supervised learning objective function and α is the weight parameter.

Images