A multi-type RNA methylation modification site prediction method

Abstract

The application provides a multi-type RNA methylation modification site prediction method, which comprises the following steps: obtaining multi-species multi-type RNA methylation modification site data to determine a positive sample site set, pre-processing sequence data of a modification type to determine a training data set; constructing a biological language model based on BERT, processing the training data set data into a sentence composed of words, training the biological language model to master the biological language information extraction capability; based on the biological language model, a prediction model is constructed by using a twin network, the prediction model is classified by using a linear classifier, the prediction model is trained based on the training data set, and an optimal prediction model is obtained; and the data is input into the optimal prediction model to obtain a central site methylation type prediction. The application can effectively extract sequence features, enhance the robustness of the model, and avoid repeated training from scratch. The model is easy to expand and migrate to a new data set, and has strong generalization ability.

Description

Technical Field

[0001] This invention relates to the field of biological sequence modification prediction, and more specifically to a method for predicting multiple types of RNA methylation modification sites. Background Technology

[0002] RNA modification has been shown to play an important role in a variety of biological processes, so accurately identifying RNA modifications in the transcriptome is crucial for a deeper understanding of biological functions and mechanisms.

[0003] Many tools have been developed for predicting RNA modifications at single-base resolution. Traditional feature engineering-based methods focus on feature design and selection, which requires extensive biological expertise and may introduce redundant information. With the rapid development of artificial intelligence, end-to-end methods have gained favor among researchers, as they avoid tedious manual feature extraction. However, both traditional machine learning and deep learning methods have the following limitations: (1) most computational methods are limited to one type of methylation; in other words, a good training model is only applicable to a specific type of modification in a particular species; (2) there is a lack of integration of data for multiple modification types; and (3) deep learning-based prediction methods still need further exploration. Convolutional neural networks and recurrent neural networks are currently the mainstream, but biological sequences can be understood as biological language, and there are currently no language models applied to this task. Summary of the Invention

[0004] To address the shortcomings of existing methods, this invention proposes a method for predicting multiple types of RNA methylation modification sites, which can predict multiple species and multiple modification types, thus solving the problem that existing models can only predict a single type.

[0005] The technical solution to achieve the purpose of this invention is: a method for predicting multiple types of RNA methylation modification sites, comprising the following steps:

[0006] Data on multiple RNA methylation modification sites from multiple species were obtained to determine the positive sample site set. Based on the location and number of positive sample sites, negative sample sites were sampled. Sequences of fixed length were truncated with positive and negative sample sites as the center. Sequence data of each modification type were preprocessed to determine the training dataset.

[0007] A biological language model is built based on BERT. The training dataset is processed and transformed into sentences composed of words. This corpus is used to train the biological language model to master the ability to extract biological language information and obtain the biological language model.

[0008] Based on a biological language model, a Siamese network is used to construct a prediction model. The prediction model uses a linear classifier for classification. The prediction model is trained on the training dataset. Each type of modification is treated as a separate downstream task. For each downstream task, starting from the trained parameters, the prediction model is adjusted using task-specific data to adapt it to the classification of the corresponding data, thus obtaining the optimal prediction model.

[0009] Input the data into the optimal prediction model to obtain the predicted methylation type at the center site.

[0010] Furthermore, the data on multiple RNA methylation modification sites in the multi-species data include four RNA modifications in mice: ψ, m6A, m5C, and m1A; three RNA modifications in Arabidopsis thaliana: ψ, m5C, and m6A; and three RNA modifications in yeast: ψ, m6A, and m1A.

[0011] Furthermore, the positive sample site set is obtained by collecting positive sample location information, integrating the locations, and mapping the location information to the reference genome of the corresponding organism.

[0012] Furthermore, during the preprocessing of the sequence data, a redundancy removal operation is performed on the sequence data, eliminating sequences with a similarity greater than 80%.

[0013] Furthermore, the process of processing the training dataset into sentences composed of words includes:

[0014] The sequence data in the training dataset is segmented into words. The original equal-length sequence is segmented into words through a sliding window of fixed width. The sliding window width is set to k and the step size is s. The sequence is divided into l-k+1 words.

[0015] Two special tokens, CLS and SEP, are added to the beginning and end of each sequence, respectively. CLS represents the classification token and SEP represents the separation token. Six consecutive words of length are masked and labeled MASK. The masking rate for all samples is 15%. Then, all words are converted into token embeddings. After adding the special tokens, the sentence length is adjusted to l-k+3, where l is the initial length of the sequence.

[0016] Furthermore, the sampling of the negative sample sites is based on the positive sample modification sites of the positive sample site set, the distance between any negative sample site and any positive sample is greater than 200bp, and the fixed-length sequence is a 41bp sequence.

[0017] Furthermore, k = 6 and s = 1.

[0018] Furthermore, the biological language model employs a layer count L=12, a hidden vector size H=768, and an attention count A=12.

[0019] Furthermore, the loss function used in the biological language model is:

[0020]

[0021] Where v1 and v2 represent the vectors of the two input sentences in the new space. If the two sentences have the same label, then parameter Y = 0; otherwise, Y takes the value 1. D(v1, v2) represents the Euclidean distance between the two specified vectors, and m is the threshold for adjusting the distance.

[0022] Furthermore, the linear classifier employs the cross-entropy loss function, which is:

[0023] loss(v,y)=-ylog(f(v))-(1-y)log(1-f(v))

[0024] Where v represents the vector of the input sentence in the new space, y is the sentence label, and f(·) represents the probability of predicting a certain type of sample.

[0025] Compared with the prior art, the significant advantages of this invention are: (1) The model of this invention is fine-tuned based on the biological language model, avoiding repeated training from scratch; (2) The model of this invention also introduces a Siamese network, and the double loss K-style restriction makes the model robust and has high prediction accuracy; (3) The model of this invention has high generalization ability and is easy to extend to a variety of multi-modification types, promoting cross-species / modification exploration. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the data collection and preprocessing process of the present invention.

[0027] Figure 2 This is a flowchart of a method for predicting multiple types of RNA methylation modification sites according to the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so as to better understand the technical concept claimed by the present invention.

[0029] A method for predicting multiple types of RNA methylation modification sites specifically includes the following steps:

[0030] Combination Figure 1 Step 1: Good data is the foundation for training a model. For multi-modification type prediction research, simply merging the datasets used by existing models is not feasible, as this would lead to sampling bias in negative samples. Therefore, this invention constructs a database for multiple RNA modification types, specifically as follows:

[0031] We selected three common model organisms—mice, Arabidopsis thaliana, and yeast—and collected high-quality RNA modification data from two main sources: publicly available RNA modification databases and publicly available wet experimental data from GEO. The final results included four RNA modifications in mice (ψ, m6A, m5C, m1A), three in Arabidopsis thaliana (ψ, m5C, m6A), and three in yeast (ψ, m6A, m1A).

[0032] After collecting comprehensive positive sample location information, the locations are integrated and mapped to the corresponding organism's reference genome, resulting in an integrated set of positive sample sites. When generating the negative sample site set, the distance between a negative sample and all positive samples is limited to 200 bp, ensuring that no negative sample overlaps with any positive sample. Then, a specific length (41 bp) sequence is extracted from the reference genome, centered on the positive or negative sample, to serve as the positive and negative samples for each modification type.

[0033] During preprocessing, redundancy removal was performed on the obtained positive sample data, removing sequences with a similarity greater than 80%, and the preprocessed data was divided into training and test sets in a 9:1 ratio.

[0034] Combination Figure 2 Step 2: Pre-train the bio-language model on a large-scale dataset, i.e., learn the semantic information of the sequence using unsupervised learning. The bio-language model has the following characteristics: First, the word length is 6; second, word masking is continuous rather than random, in order to avoid easily inferring the masked word from its neighbors; third, only a masked language model is used, and no next-sentence prediction model is used, because bio-language differs from human natural language, and there is no definition of preceding and following sentences in bio-language. The specific operation is as follows:

[0035] 2.1: Segment the sentence (sequence) into words:

[0036] Training data was generated from the human genome, with the sequence length labeled l (l between 5 and 510). All sequences were randomly sampled and non-overlapping. The original equal-length sequences were segmented into words using a fixed-width sliding window, with a window width of k and a step size of 1. After this step, the sequence was divided into l-k+1 words, where k was set to 6.

[0037] 2.2: Sentence Processing

[0038] Two special tokens, [CLS] and [SEP], are added to the beginning and end of each sentence, respectively. Here, [CLS] represents the classification token, and [SEP] represents the separation token. Six consecutive words of length are masked, labeled [MASK], with a masking rate of 15% for all samples. Then, all words are converted into token embeddings. After adding the special tokens, the sentence length is adjusted to l-k+3.

[0039] 2.3: Unsupervised BERT model pre-training:

[0040] The biological language model uses the basic BERT (layers L=12, hidden vector size H=768, attention count A=12). The BERT model (masked language model) is pre-trained on biological corpora to give the model good prediction [MASK] ability. The model is then trained on the masked prediction task, i.e., the BERT model is pre-trained on biological corpora to give the model good prediction [MASK] ability.

[0041] Step 3: Perform supervised fine-tuning on the pre-trained model described above. The specific steps are as follows:

[0042] 3.1: Data Preparation

[0043] Data preparation is similar to pre-training, requiring the conversion of task-specific sequences collected in this invention into words and sentence processing. Specifically, a 41bp sequence is divided into 36 (=41-6+1) words. After adding [CLS] and [SEP], the sentence length is adjusted to 38, and random masking of sentences is not required during the fine-tuning stage.

[0044] 3.2: Predictive model construction. The model is divided into a contrastive learning module and a classification module, as detailed below:

[0045] The contrastive learning module in Figure 2 This is represented by two BERT biolinguistic models. After the sequence passes through the contrastive learning module, it is encoded into a vector. These two BERTs share parameters and are also known as Siamese networks. The goal is to make similar samples as close as possible and dissimilar samples as far apart as possible when mapping samples to a new space. Therefore, the loss function used is as follows:

[0046]

[0047] Where v1 and v2 represent the vectors of the two input sentences in the new space. If the two sentences have the same label, then Y = 0; otherwise, Y takes the value 1. D(v1, v2) represents the Euclidean distance between the two specified vectors, and m is the threshold for adjusting the distance.

[0048] The classification module pools the [CLS] encoding of the last encoder and then connects it to a linear classifier. The classifier has an input dimension of 768 and an output dimension of 2. The classifier uses the cross-entropy loss function, which is expressed as follows:

[0049] loss(v,y)=-ylog(f(v))-(1-y)log(1-f(v))

[0050] Where v represents the vector of the input sentence in the new space, y is the sentence label, and f(·) represents the probability of predicting a certain type of sample.

[0051] Finally, considering both the comparison learning module and the classification module, the function that needs optimization is as follows:

[0052] loss=loss(v1,v2,Y)+loss(v1,y1)+loss(v2,y2)

[0053] That is, for every two samples, we must ensure that each sample is classified as correctly as possible, and that the mapping vectors of the sample pairs in the new space are close within the same class and far apart between classes.

[0054] The predictive model has two constraints: first, the use of Siamese networks ensures that the vectors of samples of the same class are close to each other after being mapped to the new space by the language model, while the vectors of samples of different classes are far apart; second, the [CLS] encoded sentences, after being processed by a linear classifier, can be classified as correctly as possible. The combination of these two constraints makes the model robust.

[0055] Step 4: The pre-trained model described in Step 2 is shared. Based on Step 3, the parameters are fine-tuned for each modification type in each group, making the model applicable to specific methylation modification classifications. The test dataset for each modification type is input into the optimal improved prediction model to obtain the methylation type prediction at the center site.

[0056] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and the model can be used for other sequence-level prediction tasks. All of these changes and modifications should fall within the protection scope of the claims of this invention.

Claims

1. A method for predicting multiple types of RNA methylation modification sites, characterized in that, Including the following steps: Data on multiple RNA methylation modification sites from multiple species were obtained to determine the positive sample site set. Based on the location and number of positive sample sites, negative sample sites were sampled. Sequences of fixed length were truncated with positive and negative sample sites as the center. Sequence data of each modification type were preprocessed to determine the training dataset. Based on BERT, a biological language model is built. The training dataset is processed and transformed into sentences composed of words. The biological language model is then trained to master the ability to extract biological language information and obtain the biological language model. Based on a biological language model, a Siamese network is used to construct a prediction model. The prediction model uses a linear classifier for classification. The prediction model is trained on the training dataset. Each type of modification is treated as a separate downstream task. For each downstream task, starting from the trained parameters, the prediction model is adjusted using task-specific data to adapt it to the classification of the corresponding data, thus obtaining the optimal prediction model. Input the data into the optimal prediction model to obtain the predicted methylation type at the central site; The multi-species data on various RNA methylation modification sites includes four RNA modifications in mice: m6A, m5C, and m1A are three RNA modifications in Arabidopsis thaliana. m5C and m6A, three RNA modifications in yeast: m6A, m1A; The positive sample site set is obtained by collecting positive sample location information, integrating the locations, and mapping the location information to the reference genome of the corresponding organism.

2. The method for predicting multiple types of RNA methylation modification sites according to claim 1, characterized in that, During the preprocessing of the sequence data, a redundancy removal operation is performed on the sequence data, and sequences with a similarity greater than 80% are removed.

3. The method for predicting multiple types of RNA methylation modification sites according to claim 1, characterized in that, The process of processing the training dataset into sentences composed of words includes: The sequence data in the training dataset is segmented into words. This is done by using a fixed-width sliding window (k width, s step size) to segment the original equal-length sequence into words. One word; Two special tokens, CLS and SEP, are added to the beginning and end of each sequence, respectively. CLS represents the classification token, and SEP represents the separation token. Six consecutive words of length are masked and labeled MASK. The masking rate for all samples is 15%. Then, all words are converted into tokenized embeddings. After adding the special tokens, the sentence length is adjusted to... , This is the initial length of the sequence.

4. The method for predicting multiple types of RNA methylation modification sites according to claim 3, characterized in that, The sampling of the negative sample sites is based on the positive sample modification sites of the positive sample site set. The distance between any negative sample site and any positive sample site is greater than 200 bp. The fixed-length sequence is a 41 bp sequence.

5. The method for predicting multiple types of RNA methylation modification sites according to claim 4, characterized in that, The values ​​are k=6 and s=1.

6. The method for predicting multiple types of RNA methylation modification sites according to claim 1, characterized in that, The biological language model uses 12 layers (L=12), a hidden vector size (H=768), and 12 attention points (A=12).

7. The method for predicting multiple types of RNA methylation modification sites according to claim 1, characterized in that, The loss function used in the bio-language model is: in, Let each represent a vector of the two input sentences in the new space. If the two sentences have the same label, then the parameters... ,otherwise, The value is 1. It is represented as the Euclidean distance between two specified vectors. This is a threshold for adjusting the distance.

8. The method for predicting multiple types of RNA methylation modification sites according to claim 1, characterized in that, The linear classifier uses the cross-entropy loss function, which is: in, This represents the vector of the input sentence in the new space. For sentence tags, This indicates the probability of predicting a certain type of sample.

Images