Protein and RNA binding site prediction method based on contrastive learning and attention mechanism

Abstract

The present invention discloses a method, device, and medium for predicting protein and RNA binding sites based on contrastive learning and attention mechanisms. The method comprises: extracting the biophysical and chemical features and semantic features of the protein; using multi-head attention to fuse the biophysical and chemical features and semantic features to obtain an attention vector for each amino acid of the protein; inputting the attention vector into a prediction module to obtain the binding sites of the protein and RNA; wherein, the parameters of the multi-head attention and prediction modules are pre-optimized by constructing a first loss function based on the similarity between the attention vectors of RNA binding sites and non-binding sites on the protein sample, and constructing a second loss function based on the prediction accuracy of the protein sample; and then optimizing the parameters of the multi-head attention and prediction modules based on the two loss functions. The present invention has good classification effect, strong applicability, and strong generalization ability, and has achieved significant results in the problem of predicting protein and RNA binding sites.

Description

Technical Field

[0001] The present invention relates to the field of bioinformatics, and in particular to a method, device and medium for predicting protein and RNA binding sites based on contrastive learning and attention mechanism. Background Art

[0002] Protein-RNA interactions are important for many cellular processes, including gene expression, RNA splicing and regulation, protein synthesis, and post-transcriptional regulation. Misregulation of these proteins contributes to numerous diseases, including cancer, diabetes, cardiovascular, and neurodegenerative disorders. Molecular-level details of protein-RNA interactions can be obtained from databases such as the Protein Data Bank (PDB), which covers atomic-level details, and BioLiP and DisProt, which provide amino acid-level annotations. However, these databases are only available for a small subset of RNA-binding proteins, necessitating the development of accurate prediction methods.

[0003] While several methods have been used to predict RNA binding in proteins, they generally suffer from several issues. First, the prediction accuracy is low and the number of false positives is high, particularly when amino acids that do not bind to any other molecule are predicted as RNA-binding. Second, they cannot effectively distinguish between RNA-binding amino acids and amino acids that bind to other molecules, for example, predicting amino acids that bind to DNA or proteins as RNA-binding. Summary of the Invention

[0004] In response to the current problems of low prediction accuracy of protein and RNA binding sites, high false positives, and poor generalization ability on different types of proteins, the present invention proposes a protein and RNA binding site prediction method, device and medium based on contrastive learning and attention mechanism to improve the accuracy of protein and RNA interaction site prediction.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0006] A method for predicting protein and RNA binding sites based on contrastive learning and attention mechanism, including:

[0007] Extract the biophysical and chemical characteristics of proteins and the semantic features of their sequences;

[0008] Use multi-head attention to fuse the physical and chemical characteristics of proteins and the semantic features of their sequences to obtain the attention vector of each amino acid in the protein;

[0009] The attention vector is input into the prediction module, and the binding site between protein and RNA is output;

[0010] Among them, the parameter pre-optimization method of the multi-head attention and prediction module is: constructing a first loss function based on the similarity between the attention vectors of RNA binding sites and non-binding sites on the protein sample, and constructing a second loss function based on the prediction accuracy of the protein sample, and then optimizing the parameters of the multi-head attention and prediction module based on the sum of the first and second loss function values ​​of the protein sample.

[0011] Furthermore, the biophysical and chemical characteristics of proteins include the relative solvent accessibility, disorder tendency, evolutionary information and binding ability tendency information of each amino acid in the protein.

[0012] Furthermore, the pre-trained language model is used to extract the semantic features of the protein sequence. The process is as follows:

[0013] Use a pre-trained language model to extract semantic features of protein sequences, where each amino acid in the sequence is represented by a 1024-dimensional word vector;

[0014] A multi-channel convolutional layer is then used to compress the 1024-dimensional word vector of each amino acid to generate a low-dimensional feature vector, which is recorded as the semantic feature of the extracted protein sequence.

[0015] Furthermore, we use multi-head attention to fuse the biological, physical, and chemical features of proteins and the semantic features of their sequences. The process is as follows:

[0016] The biophysical and chemical features and semantic features of each amino acid in the protein sequence are spliced ​​together, and the splicing vector of the i-th amino acid is recorded as X i ;

[0017] For the i-th amino acid A in the protein sequence i , use a window of length W to take amino acid A i Context information [X i-W / 2 ,…,X i ,…,X i+W / 2 ], with X i As query, [X i-W / 2 ,…,X i ,…,X i+W / 2 ] is the key and value, and the multi-head self-attention mechanism is used to extract amino acid A i The attention vector K i .

[0018] Furthermore, the prediction module includes two MLP layers.

[0019] Furthermore, the first loss function is constructed as follows:

[0020] First, the attention vector K of the amino acid iInput to the two-layer MLP, and output the latent feature vector z of the amino acid i ; A ReLU function is added between the two layers of MLP;

[0021] Then, based on the latent feature vectors of all amino acids in all protein samples, the first loss function is constructed as follows:

[0022]

[0023] Where, L c is the first loss function value, I is the sample set consisting of all amino acids in all protein samples, i is the amino acid sample in sample set I, P(i) is the positive sample set, A(i) is the set of other samples except sample i, z i is the latent feature vector of sample i, z p is the latent feature vector of the amino acid that belongs to the same category as sample i, z a is the latent feature vector of amino acids that do not belong to the same class as sample i; amino acids are classified into two categories: binding sites and non-binding sites. τ is the temperature coefficient, which adjusts the degree of attention paid to difficult negative samples.

[0024] Furthermore, the second loss function method is constructed as follows:

[0025]

[0026] Where N is the number of amino acid samples in the sample set I consisting of all amino acids in all protein samples, and y i is the label of amino acid sample i, 1 indicates a binding site, and 0 indicates a non-binding site; p i Is the prediction model based on the input attention vector K i The predicted binding tendency has a value range of [0,1]; α is a factor for adjusting sample imbalance, and r is a factor for adjusting difficult and easy classification samples.

[0027] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor implements any of the above-mentioned methods for predicting protein and RNA binding sites based on contrastive learning and attention mechanism.

[0028] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for predicting protein and RNA binding sites based on contrastive learning and attention mechanism.

[0029] Beneficial effects

[0030] The present invention, based on the integration of biological physicochemical features and sequence semantic features, predicts protein and RNA binding sites through an end-to-end model that adopts a multi-head attention mechanism and comparative learning. Among them, the multi-head attention mechanism makes the model pay more attention to and extracts features that can effectively distinguish binding sites and non-binding sites, which can improve the model accuracy and reduce false positives to a certain extent. In addition, the loss function of the model training is constructed based on similarity in the feature space, so that the same type of amino acids (i.e. binding sites or non-binding sites) of different types of proteins are closer, and the distance between different types of amino acids is farther, further ensuring that the learned binding site and non-binding site features have obvious differences, alleviating the poor generalization problem caused by different types of proteins. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a framework diagram of the method described in the embodiment of the present application. DETAILED DESCRIPTION

[0032] The following is a detailed description of an embodiment of the present invention. This embodiment is based on the technical solution of the present invention, provides a detailed implementation method and a specific operation process, and further explains the technical solution of the present invention.

[0033] This example provides a method for predicting protein and RNA binding sites based on contrastive learning and attention mechanism. Figure 1 As shown, the following steps are included.

[0034] Step 1: Extract the biophysical and chemical characteristics of the protein and the semantic features of its sequence.

[0035] Previous studies have shown that there is a certain relationship between the interaction sites of proteins and some biophysical and chemical properties, such as the solvent accessibility of amino acids, protein secondary structure, and the conservation of protein evolutionary information. In order to ensure that the model can quickly output prediction results, the embodiment of the present invention uses some fast tools and methods to obtain the biophysical and chemical characteristics of proteins. First, the protein sequence is used as input, and the relative solvent accessibility of protein amino acids is extracted through ASAquick. The relative solvent accessibility of each amino acid is expressed as a floating point value a. j The protein sequence feature vector is represented by A Lx1 Secondly, ANCHOR is used to extract the binding tendency of amino acids in disordered regions of proteins. The disorder tendency of each amino acid is represented by a vector of length 2, where one value is a floating point number between 0 and 1, indicating the probability of disorder, and the other value is 0 or 1. 0 indicates non-disordered, and 1 indicates that the amino acid is disordered. The protein sequence is represented by a D Lx2Then use hhblits to extract protein evolution information, using a P Lx20 The matrix represents the evolutionary conservation characteristics of each protein sequence. In addition, the binding tendencies of proteins and proteins, proteins and DNA, proteins and RNA, proteins and proteins and DNA, and proteins and proteins and RNA were collected from the literature and a vector B with a dimension of 5 was used. Lx5 Finally, the obtained feature matrix A i , D i , P i , B i Splicing is performed in the amino acid dimension to obtain the protein biophysical and chemical characteristic matrix X1.

[0036] There are 20 common amino acids in protein sequences (L, S, A, G, E, V, K, R, P, T, D, I, N, Q, F, Y, H, M, C, W). Taking the protein sequence as a sentence and each amino acid as a word, a semantic feature matrix M of each protein sequence is generated based on the Bert language model. Each row in the feature matrix M corresponds to an amino acid in the sequence, and each amino acid is represented by an N-dimensional word vector. Since the amino acid word vector generated by the semantic model has a large dimension, in order to reduce the complexity and overfitting of subsequent training tasks, this embodiment uses a convolutional layer to reduce its dimensionality. The dimension of the amino acid word vector is used as the number of input channels of the convolutional layer, and a 1-dimensional convolution kernel is used to slide along the direction of the protein sequence for dimensionality reduction. Finally, the amino acid word vector is compressed into a 12-dimensional feature vector X2.

[0037] Step 2: Use multi-head attention to fuse the biophysical and chemical features of the protein and the semantic features of its sequence to obtain the attention vector of each amino acid in the protein.

[0038] How to integrate different types of protein features and effectively extract amino acid binding site representation features directly affects the model's final prediction results. This paper uses a self-attention mechanism to integrate protein biophysical and chemical features with sequence semantic features and extract amino acid representation features for subsequent classification and prediction of bound and unbound amino acids.

[0039] In the present invention, amino acids are used as prediction samples, and the predicted amino acid A i As the center, take window W to supplement the context information of the predicted amino acid. In window W, only the center position A is calculated i First, combine the biological, physical, and chemical features X1 and the sequence semantic features X2 to obtain the feature X3, and take A i The window context feature information H = [X3 i-w / 2 ,…X3 i ,…X3 i+w / 2]. Then set X3 i Mapping to Q space yields q i , H is mapped to the key space to obtain K=[k i-w / 2 ,…k i ,…,k i+w / 2 ], H is mapped to the value space to obtain V=[v i-w / 2 ,…v i ,…,v i+w / 2 ],K=[X3 i-w / 2 ,…X3 i ,…X3 i+w / 2 ],V=[X3 i-w / 2 ,…X3 i ,…X3 i+w / 2 Finally, the attention distribution is calculated according to the formula and the corresponding results are weighted summed to obtain amino acid A i The attention vector K i .

[0040] Step 3: Input the attention vector into the prediction module, and output the binding site between protein and RNA;

[0041] Protein amino acids A i The attention vector K i Input to the prediction module, which contains two MLP layers to obtain the prediction result p i .

[0042] Among them, the parameters of the multi-head attention and prediction modules used to integrate biological, physical, and chemical features with semantic features in the model of the present invention are pre-trained based on protein samples. Specifically, a first loss function is constructed based on the similarity between the attention vectors of RNA binding sites and non-binding sites on the protein samples, and a second loss function is constructed based on the prediction accuracy of the protein samples. Then, the parameters of the multi-head attention and prediction modules are optimized based on the sum of the first and second loss function values ​​of the protein samples.

[0043] The first loss function is constructed by first transforming each amino acid A of the protein into i The attention vector K i Input to the MLP with two layers to get the latent feature vector Z i , add a ReLU function between the two layers; then according to the latent feature vectors of all amino acids in all protein samples, construct the first loss function as:

[0044]

[0045] Where, L cis the first loss function value, I is the sample set consisting of all amino acids in all protein samples, i is the amino acid sample in sample set I, P(i) is the positive sample set, A(i) is the set of other samples except sample i, z i is the latent feature vector of sample i, z p is the latent feature vector of the amino acid that belongs to the same category as sample i, z a is the latent feature vector of amino acids that do not belong to the same category as sample i, τ is the temperature coefficient, which adjusts the degree of attention to difficult negative samples; amino acids are divided into two categories according to whether they are binding sites with RNA. Amino acids that bind to RNA are positive samples, and amino acids that do not bind to RNA are negative samples.

[0046] Training model parameters based on the first loss function is a supervised contrastive learning method, which makes the latent feature vectors of amino acids of the same category have high similarity and the difference between the latent feature vectors of amino acids of different categories becomes larger, thereby further enhancing the feature vectors of RNA binding sites and non-RNA binding sites, while alleviating the difference between the binding site feature vectors caused by the two types of stable and disordered structures of proteins.

[0047] Taking into account the imbalance of samples, the focal loss function is used to construct the second loss function. The formula is as follows:

[0048]

[0049] Where N is the number of residue samples in the sample set I consisting of all amino acids in all protein samples, and y i is the label of residue sample i, 1 indicates a binding site, and 0 indicates a non-binding site; p i Is the prediction model based on the input attention vector K i The predicted binding tendency has a value range of [0,1]. The closer the value is to 1, the greater the probability of being a binding site. Conversely, the closer the value is to 0, the more likely it is a non-binding site. α is a factor for adjusting sample imbalance. In the present invention, the α value is set to 0.25. r is a factor for adjusting the difficulty of classifying samples. In the present invention, the r value is set to 2.

[0050] Experimental verification

[0051] In order to verify the effectiveness of the binding site identification method of this embodiment and its performance superiority compared to other methods, this experiment collected proteins with stable structures from the PDB database and disordered proteins from the DisProt database, and then divided the data sets into training sets, validation sets and test sets. The present invention uses the area under the receiver operating characteristic curve (AUC), the area under the precision-recall curve (AUPRC), the area under the cross-prediction curve (AUCPC) and the area under the over-prediction curve (AUOPC) to evaluate the prediction results. The ROC curve plots a comparison of TPR and FPR, and the Precision-recall curve plots a comparison of precision and recall. The thresholds used in the calculation take the tendency values ​​generated by the given predictor in turn. Wherein TPR is the proportion of true positive samples in the predicted positive samples, and FPR is the proportion of true positive samples predicted as negative samples. The cross-prediction curve plots the comparison of CPR and TPR, and the over-prediction curve plots the comparison of OPR and TPR, all of which are calculated using the same set of thresholds. Where CPR is the amino acid predicted to interact with other molecules, and OPR is the amino acid predicted to interact with RNA. The resulting AUCPC value assesses whether amino acids that interact with non-RNA molecules are predicted to bind to RNA, while AUOPC assesses whether amino acids that do not interact with any molecules are predicted to bind to RNA. Higher AUC and AUPRC values ​​indicate higher prediction quality, while lower AUCPC and AUOPC values ​​mean that the corresponding predictor produces fewer cross-predictions and over-predictions.

[0052] Table 1 Comparative analysis of prediction results on protein datasets with stable structures

[0053]

[0054] Table 2 Comparative analysis of prediction results on disordered protein datasets

[0055]

[0056] Table 3 Comparative analysis of prediction results on all test sets

[0057]

[0058] As shown in Tables 1-3, in order to demonstrate the effectiveness of the method of the embodiment of the present invention, this experiment selected 8 sequence-based prediction methods, including 2 methods for disordered protein prediction (DisoRDPbind and DeepDISOBind) and 6 methods for structural proteins (Pprint, BindN+, DRNApred, NCBRPred, ProNA2020 and MTDsite).

[0059] The method of the present invention was empirically compared with existing methods. The method of the present invention produced the most accurate results across the entire test dataset, with an AUC of 0.844 and an AUPRC of 0.181. Furthermore, the method of the present invention outperformed the best method based on disordered proteins, DeepDISObind (AUC = 0.816 vs. 0.716; AUPRC = 0.103 vs. 0.058), and also outperformed the best structure-based prediction method, MTDsite (AUC = 0.918 vs. 0.762; AUPRC = 0.389 vs. 0.142), on proteins annotated with structure.

[0060] In addition, the cross-prediction and over-prediction performance of different methods were analyzed. The results in Tables 1-3 demonstrate that the proposed method significantly outperforms other comparison methods. For structurally annotated proteins, the AUCPC and AUOPC of the method of the present invention are both less than 0.1, while the AUCPC and AUOPC of the other comparison methods are both greater than 0.25. Furthermore, the method of the present invention also achieved the lowest AUCPC for unannotated proteins (AUCPC = 0.132 vs. 0.341 for the second-best method, DeepDISObind) and the entire test dataset (0.125 vs. 0.355 for the second-best DeepDISObind). The same conclusion was observed based on the AUOPC evaluation. It can be seen that the method of the present invention has a relatively low cross-prediction rate and over-prediction rate. Overall, the method of the present invention significantly outperformed other methods on the test dataset and provided a relatively low level of cross-prediction.

[0061] The above embodiments are preferred embodiments of the present application. Ordinary technicians in this field can also make various changes or improvements on this basis. Without departing from the overall concept of the present application, these changes or improvements should fall within the scope of protection required by the present application.

Claims

1. A method for predicting protein and RNA binding sites based on contrastive learning and attention mechanism, characterized in that: include: Extract the biophysical and chemical characteristics of proteins and the semantic features of their sequences; Use multi-head attention to fuse the physical and chemical characteristics of proteins and the semantic features of their sequences to obtain the attention vector of each amino acid in the protein; The attention vector is input into the prediction module, and the binding site between protein and RNA is output; The parameters of the multi-head attention and prediction modules are pre-optimized by constructing a first loss function based on the similarity between the attention vectors of RNA binding sites and non-binding sites on the protein sample, and a second loss function based on the prediction accuracy of the protein sample. The parameters of the multi-head attention and prediction modules are then optimized based on the sum of the first and second loss function values ​​of the protein sample. The construction method of the first loss function is: First, the attention vector Ki of the amino acid is input into the two-layer MLP, and the hidden feature vector z of the amino acid is output. i ; A ReLU function is added between the two layers of MLP; Then, based on the latent feature vectors of all amino acids in all protein samples, the first loss function is constructed as follows: Where, L c is the first loss function value, I is the sample set consisting of all amino acids in all protein samples, i is the amino acid sample in sample set I, P(i) is the positive sample set, A(i) is the set of other samples except sample i, z i is the latent feature vector of sample i, z p is the latent feature vector of the amino acid that belongs to the same category as sample i, z a is the latent feature vector of amino acids that do not belong to the same class as sample i; τ is the temperature coefficient, which adjusts the degree of attention to difficult negative samples; among them, amino acids are divided into two categories according to whether they are binding sites with RNA. Amino acids that bind to RNA are positive samples, and amino acids that do not bind to RNA are negative samples; The second loss function method is constructed as follows: Where N is the number of amino acid samples in the sample set I consisting of all amino acids in all protein samples, and y i is the label of amino acid sample i, 1 indicates a binding site, and 0 indicates a non-binding site; p i Is the prediction model based on the input attention vector K i The predicted binding tendency has a value range of [0,1]; α is a factor for adjusting sample imbalance, and r is a factor for adjusting difficult and easy classification samples.

2. The method according to claim 1, characterized in that The biophysical and chemical characteristics of proteins include the relative solvent accessibility, disorder tendency, evolutionary information and binding ability tendency information of each amino acid in the protein.

3. The method according to claim 1, characterized in that The pre-trained language model is used to extract the semantic features of the protein sequence. The process is as follows: Use a pre-trained language model to extract semantic features of protein sequences, where each amino acid in the sequence is represented by a 1024-dimensional word vector; A multi-channel convolutional layer is then used to compress the 1024-dimensional word vector of each amino acid to generate a low-dimensional feature vector, which is recorded as the semantic feature of the extracted protein sequence.

4. The method according to claim 1, wherein Multi-head attention is used to fuse the biological, physical, and chemical features of proteins and the semantic features of their sequences. The process is as follows: The biophysical and chemical characteristics and semantic features of each amino acid in the protein sequence are spliced ​​together, and the splicing vector of the i-th amino acid is recorded as X i ; For the i-th amino acid A in the protein sequence i , use a window of length W to take amino acid A i Context information [X i-W / 2 ,…,X i ,…,X i+W / 2 ], with X i As query, [X i-W / 2 ,…,X i ,…,X i+W / 2 ] is the key and value, and the multi-head self-attention mechanism is used to extract amino acid A i The attention vector K i .

5. The method according to claim 1, characterized in that The prediction module includes a two-layer perceptron.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the computer program is executed by the processor, the processor is caused to implement the method according to any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images