A general protein-RNA binding prediction method based on multi-molecular modality fusion

Abstract

A general protein-RNA binding prediction method based on multi-molecular modality fusion is proposed. This method uses a generalized biological model to initially characterize pre-processed protein and RNA sequences, which are then used to train a neural network model based on an improved cross-attention mechanism. In the online phase, the trained model outputs a predicted value for the binding probability of any input protein and RNA sequences. This invention uses a generalized biological model to initially characterize both proteins and RNA molecules, and employs an improved cross-attention mechanism to effectively extract and fuse information from these characterizations. The artificial neural network trained based on this effectively fused information can more accurately predict protein-RNA binding outcomes and has stronger generalization capabilities, enabling it to handle binding prediction scenarios and tasks involving multiple protein types.

Description

Technical Field

[0001] This invention relates to a technology in the field of genetic engineering, specifically a universal protein-RNA binding prediction method based on multi-molecular modality fusion. Background Technology

[0002] Predicting protein-RNA binding is crucial for exploring RNA function and treating diseases. Current prediction methods are mostly based on traditional machine learning, which relies on single-protein data and cannot effectively predict the binding of unknown proteins to RNA, lacking generalization ability. Summary of the Invention

[0003] This invention addresses the limitation of existing technologies in predicting the binding outcome of any given protein and RNA. It proposes a universal protein-RNA binding prediction method based on multi-molecular modality fusion. This method uses a general biological model to initially characterize both proteins and RNA molecules. An improved cross-attention mechanism is used to extract and fuse effective information from the characterizations of the two molecules. The artificial neural network trained based on the effectively fused information can more accurately predict the protein-RNA binding outcome and has a stronger generalization ability, enabling it to handle binding prediction scenarios and tasks involving multiple types of proteins.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a general protein-RNA binding prediction method based on multi-molecular modality fusion. After initial characterization of pre-processed protein and RNA sequences using a generalized biological basic model, the pre-processed protein and RNA sequences are used to train a neural network model based on an improved cross-attention mechanism. In the online phase, the trained model outputs a predicted value for the binding probability of the two molecular sequences based on any input protein and RNA sequences.

[0006] The preprocessing refers to: constructing binding signal spectra of various proteins on full-length RNA, using a sliding window method to traverse the full-length signal spectra, obtaining positive samples of RNA sequences bound by each protein and setting the label to 1 according to the pre-set window signal value division criteria, and setting the label to 0 for the same number of negative samples that do not bind. Finally, the samples are divided into different training sets, validation sets, and test sets according to the chromosome.

[0007] The generalized biological fundamental model mentioned above adopts, but is not limited to, the general biological fundamental model.

[0008] LucaOne (https: / / github.com / LucaOne) is a pre-trained basic model for unified learning of nucleic acid and protein languages, which performs initial characterization of preprocessed protein and RNA molecular sequences, respectively.

[0009] The neural network model based on the improved cross-attention mechanism includes a multi-molecule representation fusion module and a multilayer perceptron prediction module. The multi-molecule representation fusion module, based on the pre-characterized protein and RNA molecular sequences as initial representations, uses an improved cross-attention mechanism based on these two molecular features. Specifically, each attention head calculates the attention coefficient between the two modal features according to the following formula. Q r The feature matrix representing the initial characterization of RNA. This represents the transpose of the feature matrix representing the initial characterization of the protein. This is used for attention scaling. It enables the model to dynamically weight information between the representations of RNA and protein molecular modalities, capturing the interaction patterns between RNA and protein sequences, and providing fusion features that fuse the two molecular features at a fine-grained level. This helps the model enhance its understanding of protein-RNA binding tasks. The multilayer perceptron prediction module, based on the fused multimodal representations and utilizing the previously fused multimodal information, deeply mines the nonlinear features in the data through its multilayer structure, outputting predicted values ​​for protein-RNA binding.

[0010] The multilayer perceptron prediction module includes: an average pooling layer, a linear layer, and a GELU activation function layer.

[0011] Technical effect

[0012] This invention utilizes a cross-attention module improved based on the molecular characteristics of both proteins and RNA. This module can capture crucial molecular information influencing the binding process of various proteins and RNA, providing more comprehensive prior knowledge for prediction tasks. The neural network model built upon this module can accurately predict the binding outcomes of various unknown proteins and RNA. Compared to existing technologies, this invention effectively completes the prediction task of binding various unknown proteins and RNA, providing more accurate and reliable prediction results. It eliminates the limitations of existing methods based on protein specificity and possesses strong generalization capabilities. Attached Figure Description

[0013] Figure 1 This is a flowchart of the present invention;

[0014] Figure 2 This is a schematic diagram of the neural network model of the present invention;

[0015] Figure 3This is a comparison chart of data from the present invention. Detailed Implementation

[0016] like Figure 1 As shown, this embodiment relates to a general protein-RNA binding prediction method based on multi-molecular modality fusion, specifically including:

[0017] Step 1) Data collection and preprocessing, specifically including:

[0018] Step 1.1) Obtain the alignment results file of eCLIP-seq for 168 human RNA-binding proteins and the identified binding site file from the ENCODE project website.

[0019] Step 1.2) Construct the binding signal spectrum of each protein on the full-length RNA based on the above documents.

[0020] Step 1.3) Use a fixed-length sliding window to traverse the full-length RNA binding signal spectrum of each protein, obtain a fixed-length fragment at each signal peak, and construct a positive RNA sample set with each protein tagged as 1.

[0021] Step 1.4) Based on the number of positive samples for each protein, use the same sliding window to obtain the same number of RNA fragments without signal peaks on their respective full-length RNA binding signal spectra, and construct a set of negative RNA samples with each protein labeled as 0.

[0022] Step 2) Perform pre-characterization on all protein sequences and RNA positive and negative samples after data preprocessing. Use a general biological basic model to give the initial characterization of the important biological information contained in each protein sequence and RNA sequence. Save the initial characterization of each protein and all RNA samples so that they can be directly called in the subsequent model training process, which greatly speeds up the model training.

[0023] Step 3) After data preparation is complete, a neural network model is constructed using an improved cross-attention mechanism, such as... Figure 2 As shown, it specifically includes:

[0024] Step 3.1) Input the initial characterizations of the protein and RNA sequences into the improved multi-head cross-attention module. The calculated attention coefficients are weighted and summed with the protein feature matrix. The sums are then added to the initial RNA characterization to obtain a multimodal characterization that integrates both protein and RNA molecular information.

[0025] Step 3.2) The fused multimodal representation input is a multilayer perceptron prediction module composed of an average pooling layer, a linear layer and a GELU activation function, which predicts the binding value of the input protein and RNA.

[0026] Step 4) Train the model on the preprocessed data. To verify the generalization ability of the model's prediction capabilities, we divided all the data into the following parts: 10 proteins and their corresponding RNA samples were randomly selected as the test set, and the remaining proteins and their corresponding RNA samples were used as the training set. This was to verify the model's ability to predict the binding of unknown proteins to RNA. All samples in the training set corresponding to chromosome 1 were separated to verify the model's ability to predict the binding of RNA samples to proteins corresponding to unknown chromosome categories. The AUROC of each protein test set was used to measure the model's generalization performance and prediction accuracy.

[0027] Step 5) After adjusting the model parameters and training convergence, use the model to predict all samples of each protein in the test set.

[0028] Through specific practical experiments, using a Linux environment with Python 3.9, and training the model with 807,362 RNA samples of 134 proteins on an A100 graphics card, a general RNA binding prediction model for various proteins was obtained. Predictions were then performed on chromosome 1 test sets and other chromosome test sets for 10 unknown proteins. The experimental data showed that the average AUROC for predictions on chromosome 1 test sets was above 0.77 for all proteins, and the average AUROC for predictions on other chromosome test sets was above 0.8. This indicates that the method can accurately predict the binding results of various unknown proteins with RNA and has strong generalization ability.

[0029] The prediction results are shown in Table 1. The model achieved AUROC values ​​above 0.8 for various chromosome types in predicting the binding of many unknown proteins to RNA, demonstrating good generalization ability and prediction accuracy.

[0030] Table 1. Model performance on different protein test sets

[0031]

[0032]

[0033] Compared to existing technologies, this method utilizes a cross-attention module improved based on the molecular characteristics of both proteins and RNA, effectively capturing crucial molecular information influencing the binding process. It not only possesses strong versatility but also significantly improves the accuracy and precision of predicting the binding of various unknown proteins to RNA. To date, this is the first universal method capable of handling various protein-RNA binding prediction tasks.

[0034] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A universal protein-RNA binding prediction method based on multi-molecular modality fusion, characterized in that, After initial characterization of preprocessed protein and RNA sequences using a generalized biological basic model, a neural network model based on an improved cross-attention mechanism is trained. In the online phase, the trained model outputs a predicted value of the binding probability of two molecular sequences based on any input protein and RNA molecular sequences. The neural network model based on the improved cross-attention mechanism includes a multi-molecule representation fusion module and a multilayer perceptron prediction module. The multi-molecule representation fusion module, based on the pre-characterized protein and RNA molecular sequences as initial representations, uses an improved cross-attention mechanism based on these two molecular features. Specifically, each attention head calculates the attention coefficient between the two modal features according to the following formula. in The feature matrix representing the initial characterization of RNA. This represents the transpose of the feature matrix representing the initial characterization of the protein. The scaling of attention size enables the model to dynamically allocate information weights between the representations of RNA and protein molecular modalities, capture the interaction patterns between RNA and protein sequences, and provide fusion features for fine-grained information fusion of the two molecular features, thereby helping the model enhance its understanding of protein-RNA binding tasks. The multilayer perceptron prediction module, based on the fused multimodal representations, utilizes the multimodal information fused in the previous stage and deeply mines the nonlinear features in the data through its own multilayer structure to output the predicted value of protein-RNA binding. The multilayer perceptron prediction module includes: an average pooling layer, a linear layer, and a GELU activation function layer; The neural network model based on the improved cross-attention mechanism is obtained in the following way: Step 3.1) Input the initial characterizations of the protein and RNA sequences into the improved multi-head cross attention module. The calculated attention coefficients are weighted and summed with the protein feature matrix. The sums are then added to the initial RNA characterization to obtain a multimodal characterization that integrates both protein and RNA molecular information. Step 3.2) The fused multimodal representation input is a multilayer perceptron prediction module composed of an average pooling layer, a linear layer and a GELU activation function, which predicts the binding value of the input protein and RNA.

2. The universal protein-RNA binding prediction method based on multi-molecular modality fusion according to claim 1, characterized in that, The preprocessing refers to: constructing binding signal spectra of various proteins on full-length RNA, using a sliding window method to traverse the full-length signal spectra, obtaining positive samples of RNA sequences bound by each protein and setting the label to 1 according to the pre-set window signal value division criteria, and setting the label to 0 for the same number of negative samples that do not bind. Finally, the samples are divided into different training sets, validation sets, and test sets according to the chromosome.

3. The universal protein-RNA binding prediction method based on multi-molecular modality fusion according to claim 1 or 2, characterized in that, The protein and RNA sequences were obtained in the following manner: Step 1.1) Obtain the alignment results file of eCLIP-seq for 168 human RNA-binding proteins and the identified binding site file from the ENCODE project website; Step 1.2) Construct the binding signal spectrum of each protein on the full-length RNA based on the above documents; Step 1.3) Use a fixed-length sliding window to traverse the full-length RNA binding signal spectrum of each protein, obtain a fixed-length fragment at each signal peak, and construct a positive RNA sample set with each protein tagged as 1. Step 1.4) Based on the number of positive samples for each protein, use the same sliding window to obtain the same number of RNA fragments without signal peaks on their respective full-length RNA binding signal spectra, and construct a set of negative RNA samples with each protein labeled as 0.

4. The universal protein-RNA binding prediction method based on multi-molecular modality fusion according to claim 1 or 2, characterized in that, The initial characterization refers to the pre-characterization of all protein sequences and positive and negative RNA samples after data preprocessing. A general biological basic model is used to give the initial characterization of the important biological information contained in each protein sequence and RNA sequence. The initial characterization of each protein and all RNA samples after pre-characterization is saved so that they can be directly called in the subsequent model training process, which greatly speeds up the model training.

5. The universal protein-RNA binding prediction method based on multi-molecular modality fusion according to claim 1, characterized in that, The training was conducted on preprocessed data: to verify the generalization ability of the model's prediction capabilities, all data were divided into two sets: 10 proteins and their corresponding RNA samples were randomly selected as the test set, and the remaining proteins and their corresponding RNA samples were used as the training set. This was to verify the model's ability to predict the binding of unknown proteins to RNA. All samples in the training set corresponding to chromosome 1 were separated to verify the model's ability to predict the binding of RNA samples to proteins corresponding to unknown chromosome categories. The AUROC of each protein test set was used to measure the model's generalization performance and prediction accuracy.

Images