Training method and prediction method of RNA sequence modification prediction model and related device

Abstract

The application relates to a training method and a prediction method of an RNA sequence modification prediction model and related devices. The method comprises the following steps: obtaining modification site sequencing data of an RNA sequence; based on sites with target chemical modification and sites without target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths and a plurality of negative sample sequences with different bases added at a central position are obtained respectively, so as to construct a multi-level classification database; each sample sequence in the multi-level classification database is input into a to-be-trained modification prediction model respectively, and a prediction result of a modification site of each sample sequence is obtained; based on the prediction result and an actual situation of each sample sequence, model parameters of the to-be-trained modification prediction model are updated until a trained modification prediction model is obtained. Compared with the prior art, the application can improve the training efficiency and accuracy of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a training method and a prediction method of an RNA sequence modification prediction model and related devices. BACKGROUND

[0002] In the process of RNA processing and nuclear export, in addition to splicing of introns and exons, reversible chemical modifications will also be added to the RNA sequence itself to regulate genes in various cellular processes.

[0003] In recent years, the development of epigenomics technology has promoted the identification of chemical modifications at the whole-genome level, but due to the time-consuming and expensive nature of omics-based experimental methods, and the high threshold of such sequencing techniques for detecting modifications, it has been a great challenge to quickly and accurately identify chemical modification sites based on sequences. In order to make up for the bottleneck of experimental technology, using computational methods to identify chemical modification sites has become an important research direction in this field.

[0004] Currently, the use of computational methods to identify chemical modification sites usually uses One-hot encoding to represent RNA sequences, and trains a prediction model based on the encoded RNA sequences, which cannot directly capture the characteristics of the RNA sequence, resulting in low prediction accuracy of chemical modification sites. SUMMARY

[0005] Therefore, it is necessary to provide a training method and a prediction method of an RNA sequence modification prediction model capable of improving prediction accuracy and related devices to solve the above technical problems.

[0006] In a first aspect, the present application provides a training method of an RNA sequence modification prediction model, comprising:

[0007] obtaining modification site sequencing data of an RNA sequence; the modification site sequencing data is obtained by sequencing at least two types of cell lines respectively;

[0008] based on the sites with target chemical modifications in the modification site sequencing data, obtaining a plurality of positive sample sequences of different lengths, and based on the sites without the target chemical modifications in the modification site sequencing data, obtaining a plurality of negative sample sequences with different bases added at the central position; the positive sample sequences and the negative sample sequences do not overlap in genomic positions;

[0009] obtaining a multi-level classification database based on the positive sample sequences and the negative sample sequences;

[0010] inputting each sample sequence in the multi-level classification database into a modification prediction model to be trained to obtain a prediction result of the modification site of each sample sequence;

[0011] updating the model parameters of the to-be-trained modification prediction model based on the prediction result and the actual situation of each sample sequence until a trained modification prediction model is obtained.

[0012] In one of the embodiments, the sites with the target chemical modification in the modification site sequencing data are used to obtain a plurality of positive sample sequences with different lengths, including:

[0013] The sites with the target chemical modification in the modification site sequencing data obtained by sequencing the at least two types of cell lines respectively are extracted;

[0014] Based on the sites with the target chemical modification, extension is performed to different lengths to obtain a plurality of the positive sample sequences.

[0015] In one of the embodiments, the extension based on the sites with the target chemical modification to different lengths includes:

[0016] A plurality of extension lengths are determined, and the sites with the target chemical modification are used as center sites, and the center sites are respectively extended to the determined extension lengths on both sides.

[0017] In one of the embodiments, the sites without the target chemical modification in the modification site sequencing data are used to obtain a plurality of negative sample sequences with different bases added at the center positions, including:

[0018] The sites without the target chemical modification in the modification site sequencing data obtained by sequencing the at least two types of cell lines respectively are extracted, and based on the sites without the target chemical modification, different bases are added at the center positions to obtain a plurality of the negative sample sequences.

[0019] In one of the embodiments, the addition of different bases at the center positions based on the sites without the target chemical modification includes:

[0020] The similarity between the sites without the target chemical modification and the sites with the target chemical modification is determined.

[0021] Based on the similarity, the sites without the target chemical modification are divided into a plurality of negative sample sets, and different base restrictions are respectively determined for the plurality of negative sample sets based on the similarity.

[0022] Corresponding base restrictions are respectively added to each negative sample set.

[0023] In one of the embodiments, the base restriction at least includes no restriction on the central position of the site without the target chemical modification, limiting the base at the central position of the site without the target chemical modification to adenine, and / or limiting the base at the central position of the site without the target chemical modification to a base arrangement satisfying DRACH.

[0024] In one of the embodiments, the multi-level classification database based on the positive sample sequences and the negative sample sequences comprises:

[0025] The plurality of positive sample sequences and the plurality of negative sample sequences are respectively subjected to deduplication processing;

[0026] Based on the plurality of positive sample sequences and the plurality of negative sample sequences after deduplication, positive sample sequences and negative sample sequences are selected according to a preset ratio to form the multi-level classification database; the multi-level classification database comprises a plurality of sample sequences, and at least the lengths or the bases at the central positions of different sample sequences are different.

[0027] In one of the embodiments, the input of each sample sequence in the multi-level classification database into the modification prediction model to be trained to obtain the prediction result of the modification site of each sample sequence comprises:

[0028] Any sample sequence in the multi-level classification database is determined as a current sample sequence;

[0029] An initial modification prediction model is obtained; the initial modification prediction model is obtained based on the joint construction of a Bert module and a multi-head attention module;

[0030] The current sample sequence is encoded based on the Bert module to obtain an embedding representation of the current sample sequence;

[0031] The embedding representation of the current sample sequence is subjected to attention representation extraction based on the multi-head attention module to obtain an attention representation of the current sample sequence;

[0032] The attention representation of the current sample sequence is subjected to classification prediction to obtain a modification site prediction result of the current sample sequence.

[0033] In one of the embodiments, the encoding of the current sample sequence based on the Bert module to obtain the embedding representation of the current sample sequence comprises:

[0034] The current sample sequence is input into the Bert module; the Bert module has a plurality of BERT attention layers, and each BERT attention layer has a plurality of first attention heads;

[0035] vectorize the current sample sequence through an embedding layer of the Bert module to obtain a vector sequence of the current sample sequence;

[0036] split the vector sequence of the current sample sequence into a plurality of first attention heads through the multi-layer BERT attention layer and determine a first attention weight of each first attention head, and determine embedding information of the current sample sequence based on the first attention weights; the input of the first-layer BERT attention layer is the vector sequence of the current sample sequence output by the embedding layer, and the input of each subsequent-layer BERT attention layer is the embedding information of the current sample sequence output by the previous-layer BERT attention layer;

[0037] process the embedding information of the current sample sequence output by the last-layer BERT attention layer to obtain embedding representation of the current sample sequence.

[0038] In one of the embodiments, after the classification prediction based on the attention representation of the current sample sequence to obtain the modification site prediction result of the current sample sequence, the method further comprises:

[0039] extract second attention weights of each second attention head of the multi-head attention module;

[0040] determine true positive samples in the modification site prediction result of the current sample sequence, traverse the true positive samples through a preset sliding window, determine a weight sum of the true positive samples covered by each preset sliding window according to the second attention weights, extract true positive samples whose weight sum meets a preset condition to form a motif set;

[0041] cluster the motif set to determine a key motif;

[0042] match the key motif with existing motifs of a public database, extract the key motif that matches successfully and visualize the key motif.

[0043] In one of the embodiments, the clustering of the motif set to determine a key motif comprises:

[0044] search for a motif in the motif fragments of the motif set by using a MEME algorithm, and cluster the searched motifs according to a preset clustering number to obtain a plurality of motif groups;

[0045] determine a motif that meets a preset characteristic condition in each motif group as a key motif.

[0046] In one of the embodiments, the matching of the key motif with existing motifs of a public database, the extraction of the key motif that matches successfully and the visualization of the key motif comprise:

[0047] The similarity score between the key motif and each reference motif is determined by using the TOMTOM algorithm, the reference motif corresponding to the minimum similarity score is extracted, and the extracted reference motif is visualized.

[0048] In one embodiment, the model parameters of the to-be-trained modification prediction model are updated based on the prediction result and the actual situation of each sample sequence until a trained modification prediction model is obtained, comprising:

[0049] The model parameters of the to-be-trained modification prediction model are updated based on the prediction result and the actual position information of the site with the target chemical modification in the positive sample sequence and the actual position information of the site without the target chemical modification in the negative sample sequence until a trained modification prediction model is obtained.

[0050] In a second aspect, the present application also provides a modification prediction method of RNA sequence, comprising:

[0051] Obtaining the RNA sequence to be predicted;

[0052] Inputting the RNA sequence to be predicted into the trained modification prediction model; the modification prediction model is trained based on the training method of the above RNA sequence modification prediction model;

[0053] Encoding the RNA sequence based on the Bert module of the modification prediction model to obtain the embedding representation of the RNA sequence;

[0054] Extracting the attention representation of the embedding representation of the RNA sequence based on the multi-head attention module of the modification prediction model to obtain the attention representation of the RNA sequence;

[0055] Classifying and predicting based on the attention representation of the RNA sequence to obtain the prediction result of the site with the target chemical modification in the RNA sequence.

[0056] In a third aspect, the present application also provides a training device of RNA sequence modification prediction model, comprising:

[0057] A data acquisition module for acquiring modification site sequencing data of RNA sequence; the modification site sequencing data is obtained by sequencing at least two types of cell lines respectively;

[0058] a sample extraction module configured to obtain a plurality of positive sample sequences with different lengths based on sites with a target chemical modification in the modification site sequencing data, and obtain a plurality of negative sample sequences with different bases added at a central position based on sites without the target chemical modification in the modification site sequencing data; the positive sample sequences and the negative sample sequences do not overlap in genomic positions;

[0059] a database construction module configured to obtain a multi-level classification database based on the positive sample sequences and the negative sample sequences;

[0060] a site prediction module configured to input each sample sequence in the multi-level classification database into a trained modification prediction model to obtain a prediction result of a modification site of the each sample sequence;

[0061] a model updating module configured to update model parameters of the trained modification prediction model based on the prediction result and an actual situation of the each sample sequence until a trained modification prediction model is obtained.

[0062] In a fourth aspect, the present application also provides a modification prediction device for RNA sequences, and the device comprises:

[0063] a sequence acquisition module configured to acquire an RNA sequence to be predicted;

[0064] a sequence input module configured to input the RNA sequence to be predicted into a trained modification prediction model;

[0065] an embedding representation extraction module configured to encode the RNA sequence based on a BERT module of the modification prediction model to obtain an embedding representation of the RNA sequence;

[0066] an attention representation extraction module configured to extract an attention representation of the embedding representation of the RNA sequence based on a multi-head attention module of the modification prediction model to obtain an attention representation of the RNA sequence;

[0067] a classification prediction module configured to perform classification prediction based on the attention representation of the RNA sequence to obtain a prediction result of a modification site of the RNA sequence.

[0068] In a fifth aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements steps of the model training method and / or the modification prediction method when executing the computer program.

[0069] In a sixth aspect, the present application also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of the model training method and / or the modification prediction method.

[0070] In a seventh aspect, the present application also provides a computer program product, which comprises a computer program. The computer program is executed by a processor to implement the steps of the model training method and / or the modification prediction method.

[0071] The training method of the RNA sequence modification prediction model, the prediction method and the related device described above, by obtaining modification site sequencing data of the RNA sequence, based on the sites with target chemical modification and the sites without target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths and a plurality of negative sample sequences with different bases added at the center position are obtained respectively, a multi-level classification database is obtained based on the positive sample sequences and the negative sample sequences, then each sample sequence in the multi-level classification database is input into the modification prediction model to be trained respectively to obtain the prediction result of the modification site of each sample sequence, the model parameters of the modification prediction model to be trained are updated based on the prediction result and the actual situation of each sample sequence until the trained modification prediction model is obtained. Through simple processing of the extracted positive and negative samples, a multi-level classification database with different lengths and different classification difficulties can be constructed, and the data preprocessing efficiency is high. At the same time, the multi-level classification database is used as a standardized database to train the constructed end-to-end modification prediction model, which can improve the generalization ability of the model, and the model is trained by different sample sequences, which is conducive to the horizontal comparison between different model algorithms. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0073] Figure 1 The flowchart of the training method of the RNA sequence modification prediction model in an embodiment;

[0074] Figure 2 The flowchart of obtaining the multi-level classification database in an embodiment;

[0075] Figure 3 The flowchart of obtaining the prediction result of the modification site in an embodiment;

[0076] Figure 4 A flowchart of the encoding process of the Bert module in one embodiment;

[0077] Figure 5 A flowchart of the weight visualization of the modification prediction model in one embodiment;

[0078] Figure 6 A flowchart of the modification prediction method of the RNA sequence in one embodiment;

[0079] Figure 7 A schematic diagram of positive sample sequences and negative sample sequences in one application example;

[0080] Figure 8 A structural block diagram of the modification prediction model in one application example;

[0081] Figure 9 A schematic diagram of the relationship between the motif position of each sample extracted by each head of the modification prediction model and the corresponding attention weight in one application example;

[0082] Figure 10 A schematic diagram of the motif after weight visualization of the modification prediction model in one application example;

[0083] Figure 11 A comparison diagram of the prediction effect of the modification prediction model and other models in one application example;

[0084] Figure 12 A structural block diagram of the training device of the RNA sequence modification prediction model in one embodiment;

[0085] Figure 13 A structural block diagram of the modification prediction device of the RNA sequence in one embodiment. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0087] In one embodiment, as Figure 1As shown, a method for training an RNA sequence modification prediction model is provided. In this embodiment, the method is applied to a terminal. It should be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and can be implemented through the interaction of the terminal and the server. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, portable wearable devices, and the like. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services, and the like.

[0088] At present, there are more than one hundred RNA modifications (chemical modifications) identified in cells, but there are not many modifications widely existing in mRNA. Among them, m6A modification (N6-methyladenosine modification) is a very common type of chemical modification on mRNA, and there are extensive reports that m6A modification is involved in the post-transcriptional level regulation of cells and affects gene expression in multiple physiological processes. m6A modification is reversible, which is modified by a "writer" and can be removed by an "eraser", and more importantly, m6A modification can be recognized by RNA binding proteins YTHDF1 / 2 / 3, YTHDC1 / 2, etc. The m6A recognition protein can recognize and bind to the m6A modified transcript, and regulate gene expression by regulating various processes, such as regulating mRNA stability, mRNA splicing, mRNA structure, translation efficiency, and miRNA biosynthesis, etc. Studies have shown that m6A modification plays an important role when cells face various physiological or pathological conditions, especially in the occurrence and development of some diseases or cancers.

[0089] Chemical modifications in RNA sequences play a crucial role in regulating RNA expression and function. Among them, m6A modification is one of the most abundant modification types in eukaryotic cells, and m6A modification affects RNA structure, RNA stability, and RNA translation, and regulates key cellular processes such as disease occurrence and stress response, so efficient identification of m6A modification sites is one of the urgent needs to study the translation regulation response to environmental changes.

[0090] Therefore, the embodiment proposes a method for training an RNA sequence modification prediction model, which includes the following steps S101 to S105. Among them:

[0091] Step S101, obtaining modification site sequencing data of the RNA sequence.

[0092] The RNA sequence is usually represented in the form of a string and contains four bases A, U, C, and G. The modification site sequencing data of the RNA sequence refers to data containing modification site feature information, and the feature information includes the position, modification type, modification degree, and additional information of the modification site. Various chemical modifications are usually included on the modification site, such as m6A, m1A, pseudo-uracil, m5C, and the like. In this embodiment, the modification site sequencing data is obtained based on the latest published single nucleotide precision absolute quantification sequencing data.

[0093] In this embodiment, the modification site sequencing data is obtained based on at least two types of cell lines.

[0094] For example, the RNA sequences of two different types of cell lines are obtained respectively, and the modification site sequencing data of the RNA sequences is obtained based on the latest published single nucleotide precision absolute quantification sequencing data.

[0095] In this embodiment, the modification site sequencing data includes data of the sequencing samples, and the sequencing samples contain single nucleotide precision absolute quantification information. Based on the sequencing samples, the modification sites in the corresponding cell lines can be identified, and by absolutely quantifying each modification site, the modification degree of different sites can be understood.

[0096] In step S102, a plurality of positive sample sequences with different lengths are obtained based on the sites with the target chemical modification in the modification site sequencing data, and a plurality of negative sample sequences with different bases added at the center position are obtained based on the sites without the target chemical modification in the modification site sequencing data.

[0097] The target chemical modification refers to the prediction target of the modification prediction model, that is, the type of chemical modification that needs to be determined on the modification site sequencing data. The site with the target chemical modification refers to the site on the modification site sequencing data that has the chemical modification intended by this embodiment. The site without the target chemical modification refers to the site on the modification site sequencing data that does not have the chemical modification intended by this embodiment. It can be understood that the site without the target chemical modification only does not have the chemical modification intended by this embodiment, but may have other types of chemical modifications.

[0098] The positive sample sequence refers to a sequence composed of sites with target chemical modification on the RNA sequence. Different lengths refer to different lengths of each positive sample sequence, but the length of each positive sample in each positive sample sequence is the same. For example, the length of each positive sample in a positive sample sequence with a length of 51 nt is 51 nt, and for example, the length of each positive sample in a positive sample sequence with a length of 101 nt is 101 nt, and so on. According to the actual prediction needs, a plurality of positive sample sequences with different lengths can be constructed.

[0099] The negative sample sequence refers to a sequence composed of sites or fragments without target chemical modification on the RNA sequence, so that the positive sample sequence and the negative sample sequence are strictly non-overlapping on the genome. Different bases added to the center position refer to limiting the base type of the center position of each negative sample in the negative sample sequence, so as to construct different classification difficulties on the negative sample, thereby obtaining a plurality of negative sample sequences.

[0100] For example, the obtained modification site sequencing data is detected for target chemical modification, that is, four sequencing samples in the above two cell lines are detected, it is determined that the four sequencing samples all have sites with target chemical modification, and the positive samples are determined, and the length of the positive samples is extended to obtain a plurality of positive sample sequences with different lengths; then, it is determined that the four sequencing samples all do not have sites with target chemical modification, and the negative samples are determined, and different base restrictions are imposed on the center position of the negative samples to obtain a plurality of negative sample sequences with different restrictions.

[0101] Step S103, obtaining a multi-level classification database based on the positive sample sequence and the negative sample sequence.

[0102] The multi-level classification database includes a plurality of sample sequences at different levels. Different levels refer to that the samples in the sample sequences have different lengths and / or different classification difficulties (i.e., different base restrictions).

[0103] For example, each positive sample sequence is sampled with each negative sample sequence to obtain a plurality of sample sequences containing positive samples and negative samples. The length of each positive sample in each sample sequence is the same, and the base restriction type of the negative sample is the same. Different sample sequences are different at least in the length of the positive sample or at least in the base restriction type of the negative sample, so as to construct a multi-level classification database.

[0104] Step S104, inputting each sample sequence in the multi-level classification database into the modification prediction model to be trained to obtain a prediction result of the modification site of each sample sequence.

[0105] Exemplarily, according to the training requirement, each sample sequence in the multi-level classification database is input into the modification prediction model to be trained, feature information of the target chemical modification on the sample sequence is extracted based on the forward transmission of the modification prediction model, and a prediction result of the modification site on the corresponding sample sequence is output based on the feature information, so as to determine which positions on the sample sequence have the target chemical modification.

[0106] The modification prediction model is implemented based on one or more models including but not limited to a convolution network, a BLSTM model, a random forest, an attention mechanism, and a Bert model.

[0107] In step S105, the model parameters of the modification prediction model to be trained are updated based on the prediction result and the actual situation of each sample sequence until a trained modification prediction model is obtained.

[0108] The actual situation of each sample sequence refers to actual position information of the site with the target chemical modification in the positive sample sequence and / or actual position information of the site without the target chemical modification in the negative sample sequence, which is used to determine the accuracy of the prediction result.

[0109] Exemplarily, based on the prediction result and the actual position information of the site with the target chemical modification in the positive sample sequence, a deviation between the prediction result and the actual position information of the site with the target chemical modification is determined, and the model parameters are updated based on the deviation until the trained modification prediction model is obtained. Since this embodiment mainly performs prediction of the target chemical modification site, when updating the model parameters, the prediction result and the actual position information of the site with the target chemical modification in the positive sample sequence are mainly matched. It can be understood that in some other embodiments, the model parameters can also be updated based on the prediction result and the actual position information of the site without the target chemical modification in the negative sample sequence, or the model parameters can be updated based on the comprehensive matching of the prediction result, the actual position information of the site with the target chemical modification in the positive sample sequence, and the actual position information of the site without the target chemical modification in the negative sample sequence.

[0110] In the training method of the RNA sequence modification prediction model, modification site sequencing data of the RNA sequence is obtained; based on the sites with the target chemical modification and the sites without the target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths and a plurality of negative sample sequences with different bases added at the central positions are obtained respectively, and a multi-level classification database is obtained based on the positive sample sequences and the negative sample sequences; then, each sample sequence in the multi-level classification database is input into the modification prediction model to be trained respectively to obtain the prediction result of the modification site of each sample sequence; based on the prediction result and the actual situation of each sample sequence, the model parameters of the modification prediction model to be trained are updated until the trained modification prediction model is obtained. The multi-level classification database with different lengths and different classification difficulties can be constructed by simply processing the extracted positive and negative samples, and the data preprocessing efficiency is high; at the same time, the multi-level classification database is used as a standardized database to train the constructed end-to-end modification prediction model, which can improve the generalization ability of the model, and the model is trained by different sample sequences, which is beneficial to realize the horizontal comparison between different model algorithms.

[0111] In one embodiment, in step S102, based on the sites with the target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths are obtained, including:

[0112] The sites with the target chemical modification in the modification site sequencing data respectively sequenced by at least two types of cell lines are extracted; based on the sites with the target chemical modification, a plurality of positive sample sequences with different lengths are obtained by extension.

[0113] For example, the sites with the target chemical modification in the modification site sequencing data respectively sequenced by at least two types of cell lines are extracted as positive samples; the positive samples are extended to obtain a plurality of positive sample sequences with different lengths.

[0114] In this embodiment, by extracting the sites with the target chemical modification as positive samples and extending the positive samples with different lengths, a plurality of positive sample sequences with different lengths can be obtained to expand the training samples of the model from the length level and improve the diversity of the training samples.

[0115] In one embodiment, the specific method of extending the positive samples includes:

[0116] A plurality of extension lengths are determined, and the sites with the target chemical modification are taken as the central sites, and the central sites are extended to the determined extension lengths on both sides of the central sites.

[0117] For example, based on the training requirement, different extension lengths are determined, and then, the positive sample is taken as the center site, and the center site is extended to the determined extension length on both sides of the center site. For example, based on the training requirement, the extension lengths of 25 nt, 50 nt, and 100 nt are determined; for the positive sample with the extension length of 25 nt, the positive sample is taken as the center site, and the positive sample is extended to 25 nt on both sides of the center site to obtain a positive sample with the length of 51 nt (the length of the positive sample itself is 1 nt, and the length of each of the 25 nt extended on both sides is 25 nt, so that the total length of the positive sample is 51 nt); for the positive sample with the extension length of 50 nt and 100 nt, the foregoing process can be referred to to obtain positive samples with the lengths of 101 nt and 201 nt, respectively; it can be understood that, in some other embodiments, the extension lengths can also be 75 nt, 150 nt, and the like, so as to correspondingly obtain positive samples with the lengths of 151 nt and 301 nt, and the specific values are not limited in the embodiment.

[0118] In the embodiment, by extending the different extension lengths on both sides of the positive sample with the positive sample as the center, a plurality of positive samples with different lengths can be obtained, so as to form a positive sample sequence, and data processing is efficient and simple.

[0119] In one embodiment, in step S102, based on the sites without the target chemical modification in the modification site sequencing data, a plurality of negative sample sequences with different bases added at the center position are obtained.

[0120] The sites without the target chemical modification in the modification site sequencing data obtained by sequencing at least two types of cell lines are extracted, and based on the sites without the target chemical modification, different bases are added at the center position to obtain a plurality of negative sample sequences.

[0121] For example, the sites without the target chemical modification in the modification site sequencing data obtained by sequencing at least two types of cell lines are extracted as negative samples; based on the negative samples, different bases are added at the center position of each negative sample to obtain a plurality of negative sample sequences.

[0122] In the embodiment, by extracting the sites without the target chemical modification as negative samples and adding different bases at the center position of the negative samples, a plurality of different negative sample sequences can be obtained, so as to expand the training samples of the model from the classification difficulty level and further improve the diversity of the training samples.

[0123] In one embodiment, the specific method of adding different bases at the center position of the negative sample to limit includes:

[0124] determining the similarity between the sites without the target chemical modification and the sites with the target chemical modification, dividing the sites without the target chemical modification into a plurality of negative sample sets based on the similarity, and determining different base restrictions for the plurality of negative sample sets based on the similarity.

[0125] wherein the similarity between the sites without the target chemical modification and the sites with the target chemical modification refers to the similarity of the feature information at the genomic level between the negative sample and the positive sample. The negative sample set refers to a set of negative samples with a similarity to the positive sample within the same threshold range.

[0126] wherein the base restriction refers to adding different bases to the center position of the negative sample. In this embodiment, the base restriction at least includes no restriction to the center position of the site without the target chemical modification, limiting the base at the center position of the site without the target chemical modification to adenine, and / or limiting the base at the center position of the site without the target chemical modification to a base arrangement satisfying DRACH.

[0127] wherein no restriction to the center position of the site without the target chemical modification refers to no addition of any base to the center position of the negative sample, i.e. no any restriction processing; limiting the base at the center position of the site without the target chemical modification to adenine (i.e. Adenine, also known as “A”) refers to limiting the nucleotide base pair at the center position of the negative sample to adenine; and limiting the base at the center position of the site without the target chemical modification to a base arrangement satisfying DRACH refers to limiting the arrangement of the nucleotide base pair at the center position of the negative sample to DRACH, wherein “D” represents that the nucleotide base pair at the site is “A”, “G” or “U”, “R” represents that the nucleotide base pair at the site is “A” or “G”, “A” is adenine “A”, “C” represents that the nucleotide base pair at the site is cytosine “C”, and “H” represents that the nucleotide base pair at the site is “A”, “C” or “U”.

[0128] For example, first, the feature similarity at the genomic level between the negative sample and each positive sample is determined; then, based on the feature similarity, the negative sample is divided into a plurality of negative sample sets according to the size of the feature similarity, and the feature similarity of each negative sample in each negative sample set is comparable to that of the positive sample; finally, based on the feature similarity, different base restrictions are determined for the plurality of negative sample sets, and corresponding base restrictions are added to each negative sample set.

[0129] In the division of the negative sample set, the negative sample is compared with the positive sample to determine the feature similarity between the negative sample and the positive sample. The feature similarity is divided into multiple levels based on the size of the feature similarity. To facilitate subsequent sampling of the positive and negative samples, the number of levels is the same as the number of positive sample sequences. For example, the first level of the feature similarity is determined to be less than or equal to 25% or other proportion, the second level of the feature similarity is determined to be within a range of 50% ± 25% or other proportion range, and the third level of the feature similarity is determined to be greater than or equal to 75% or other proportion. In this way, according to the feature similarity of the positive and negative samples, the negative samples with the first level, the second level and the third level of the feature similarity are formed into multiple negative sample sets as a separate set, and the base restriction is added according to different feature similarity levels.

[0130] In a specific implementation, for the first level of the negative sample set, the center position of the negative sample is not restricted, that is, for the negative sample with low similarity or completely different from the positive sample, the simplest restriction (zero restriction or no restriction) is performed; for the second level of the negative sample set, the nucleotide base pair of the center position of the negative sample is restricted to adenine, that is, for the negative sample with moderate similarity or a little similar to the positive sample, the moderate difficulty restriction is performed; and for the third level of the negative sample set, the arrangement of the nucleotide base pair of the center position of the negative sample is limited to the arrangement mode satisfying DRACH, that is, for the negative sample with high similarity or almost the same as the positive sample, the most difficult or most complex restriction is performed. In this way, three negative sample sequences with different classification difficulties can be constructed to expand the level of training data. It can be understood that in some other embodiments, when the similarity of the negative sample is divided, more levels can be further divided to correspond to define different base restrictions, so as to obtain more levels of sample sequences.

[0131] In the embodiment, by dividing the negative sample into multiple sets based on the similarity with the positive sample and adding different base restrictions to each set, the variables of the same sample sequence can be controlled, and negative sample sequences with multiple classification difficulties can be constructed.

[0132] In one embodiment, as shown in Figure 2 Fig. 3, step S103 includes steps S201 to S202 based on the positive sample sequence and the negative sample sequence to obtain the multi-level classification database. Wherein:

[0133] Step S201, the plurality of positive sample sequences and the plurality of negative sample sequences are respectively subjected to a de-duplication process.

[0134] The de-duplication processing refers to de-duplicating samples in the plurality of positive sample sequences and the plurality of negative sample sequences respectively, which satisfy a certain condition in similarity.

[0135] For example, a de-duplication threshold is set, and for each sample in each positive sample sequence and each negative sample sequence, similarity calculation is performed between each sample and other samples in the sequence at the genome level, and samples with similarity greater than the de-duplication threshold are removed to obtain de-duplicated positive sample sequences and negative sample sequences.

[0136] In step S202, based on the de-duplicated plurality of positive sample sequences and the de-duplicated plurality of negative sample sequences, positive sample sequences and negative sample sequences are selected according to a preset proportion to form a multi-level classification database.

[0137] The multi-level classification database includes a plurality of sample sequences, and different sample sequences are different in length or different in base at the center position.

[0138] The selection of the positive sample sequences and the negative sample sequences according to the preset proportion refers to selecting a corresponding number of positive samples and negative samples from each positive sample sequence and different negative sample sequences according to a set selection proportion to form a sample sequence.

[0139] For example, the number of samples in each sample sequence is set, one de-duplicated positive sample sequence and any de-duplicated negative sample sequence are selected, and the samples in the positive sample sequence and the negative sample sequence are selected according to a preset proportion, until the set number of samples is reached, and the selected samples are divided into a training set and a test set according to a set proportion to form a sample sequence. Subsequently, another de-duplicated negative sample sequence is determined, and the selected positive sample sequence and the determined negative sample sequence are selected and collected according to the foregoing process to form another sample sequence, until the selected positive sample sequence and all negative sample sequences are cross-selected, another positive sample sequence is selected to repeat the foregoing process until all positive sample sequences and all negative sample sequences are cross-selected, and the plurality of sample sequences obtained are integrated to form a multi-level classification database.

[0140] Wherein, the cross selection of the samples in the positive sample sequences and the negative sample sequences refers to selecting all the positive sample sequences with each negative sample sequence to obtain sample sequences with positive and negative samples. For example, for the positive sample sequences with a length of 51 nt, cross selection is respectively performed with the negative sample sequences without any restriction, the negative sample sequences with the nucleotide base pair at the center position being adenine, and the negative sample sequences with the nucleotide base pair at the center position satisfying the DRACH arrangement, to form three sample sequences with the same length but different base restrictions. For example, for the positive sample sequences with a length of 101 nt, cross selection is respectively performed with the negative sample sequences without any restriction, the negative sample sequences with the nucleotide base pair at the center position being adenine, and the negative sample sequences with the nucleotide base pair at the center position satisfying the DRACH arrangement, to form three sample sequences with the same length but different base restrictions. Similarly, a plurality of sample sequences with different lengths and different classification difficulties are formed.

[0141] In a specific implementation, the public data set (such as DeepM6ASeq, MASS, etc.) can also be integrated into the multi-level classification database to better expand the sample types of the data set.

[0142] In this embodiment, the plurality of positive sample sequences and the plurality of negative sample sequences are respectively subjected to deduplication processing to reduce redundant data and improve analysis efficiency. Based on the plurality of positive sample sequences after deduplication and the plurality of negative sample sequences after deduplication, the positive sample sequences and the negative sample sequences are selected according to a preset ratio to form a multi-level classification database. The sample sequence types in the database are diverse, which is used for subsequent model training and is conducive to the horizontal evaluation of the performance of the model.

[0143] In one embodiment, as shown in FIG. 10, in step S104, each sample sequence in the multi-level classification database is input into the modification prediction model to be trained to obtain the prediction result of the modification site of each sample sequence, including the following steps S301 to S305. Wherein: Figure 3

[0144] Step S301, determining any sample sequence in the multi-level classification database as a current sample sequence.

[0145] For example, any sample sequence in the multi-level classification database is selected and determined as a current sample sequence, which is input into the modification prediction model for model training.

[0146] Step S302, obtaining an initial modification prediction model.

[0147] ​The initial modification prediction model is constructed based on a Bert module and a multi-head attention module. The initial modification prediction model refers to an initialized modification prediction model, and the model parameters are initial parameters that have not been trained. The model parameters of the initial modification prediction model refer to parameters that can be updated and optimized to make the prediction results of the initial modification prediction model infinitely close to the true value during model training. The model parameters of the embodiment include, but are not limited to, the attention layer, the attention head, and the hidden layer dimension of the Bert module and the multi-head attention module.

[0148] In step S303, the current sample sequence is encoded based on the Bert module to obtain the embedding representation of the current sample sequence.

[0149] The Bert module refers to a module constructed based on the Bert algorithm and capable of extracting context information from input data. The Bert module usually has an embedding layer, an encoding layer, and a Pooler layer connected in sequence. The embedding layer is used to convert the input sample sequence into a token format that can be understood by the Bert algorithm. During the conversion, each base in the sample sequence needs to be mapped to the vocabulary table of the Bert module using a specific vocabulary table. Since the standard Bert module is designed for natural language processing, the vocabulary table of the Bert module can be customized according to the bases in the sample sequence to adapt the vocabulary table of the Bert module to the sample sequence. The encoding layer is used to convert the token format sample sequence into a high-dimensional vector representation, i.e., embedding information. The Pooler layer is used to obtain the embedding information of each token to obtain the context information of the sample sequence, i.e., the embedding representation.

[0150] For example, the current sample sequence is input into the initial modification prediction model. The embedding layer of the Bert module converts the input current sample sequence into corresponding tokens. The encoding layer of the Bert module converts the tokens into high-dimensional vector representations to obtain the embedding information corresponding to each token. Finally, the Pooler layer of the Bert module obtains the embedding information of each token to obtain the context information of the RNA sequence, and the embedding representation is input into the multi-head attention module.

[0151] In step S304, the embedding representation of the current sample sequence is extracted based on the multi-head attention module to obtain the attention representation of the current sample sequence.

[0152] The multi-head attention module is a module based on a multi-head attention mechanism, which is capable of calculating attention weights of input embedding representations, determining final attention representations, and identifying sites with target chemical modifications in sample sequences through subsequent processing. The multi-head attention module has a linear layer, a head matrix, and a merging layer connected in sequence. The linear layer is used to perform linear transformation on input data (i.e., embedding representations). The head matrix is used to divide the linearly transformed embedding representations according to a set number of attention heads and calculate the attention weights of each attention head, based on which the output of each attention head is obtained. The merging layer is used to connect the outputs of each attention head end to end to form a new data structure and obtain the attention representation of the sample sequence.

[0153] The attention weight refers to the degree of association or importance between different positions in the input embedding representation.

[0154] The attention representation refers to the attention weight and the corresponding output vector obtained by running multiple independent attention heads in parallel to determine the attention distribution of different subspaces of the divided embedding representation.

[0155] For example, the embedding representation output by the BERT module is input into the multi-head attention module. The linear layer of the multi-head attention module performs linear transformation on the input embedding representation. The head matrix of the multi-head attention module divides the linearly transformed embedding representation into multiple attention heads according to a set number of attention heads, calculates the query, key, and value matrices of each attention head, uses the query matrix and the key matrix to calculate the attention weight of each attention head, and uses the attention weight to perform weighted summation on the value matrix to obtain the output of each attention head. Finally, the merging layer of the multi-head attention module connects the outputs of each attention head end to end to form a new data structure and obtain the attention representation of the current sample sequence.

[0156] In step S305, classification prediction is performed based on the attention representation of the current sample sequence to obtain the modification site prediction result of the current sample sequence.

[0157] For example, the attention representation output by the multi-head attention module is input into the pooling layer for feature extraction to reduce the dimension of the attention representation and ensure that the extracted features contain important features of the attention representation. The extracted features are then input into the fully connected layer together with the global features (or all features) of the sample sequence, and then input into the MLP layer. The MLP layer performs classification prediction on the attention representation to obtain the modification site prediction result of the current sample sequence.

[0158] In the embodiment, the end-to-end modification prediction model is constructed by fusing the Bert module and the multi-head attention module, and the end-to-end training of the modification prediction model is realized through the constructed multi-level classification database, so that the training precision and the prediction precision of the model can be effectively improved.

[0159] In one embodiment, as shown in FIG. 3, Figure 4 In step S303, the current sample sequence is encoded based on the Bert module to obtain the embedding representation of the current sample sequence, including the following steps S401 to S404. Wherein:

[0160] Step S401, input the current sample sequence into the Bert module.

[0161] Wherein, the Bert module has a plurality of BERT attention layers, and each BERT attention layer has a plurality of first attention heads, and each layer of BERT attention layer can sequentially process the input current sample sequence.

[0162] Step S402, the current sample sequence is vectorized by the embedding layer of the Bert module to obtain the vector sequence of the current sample sequence.

[0163] Wherein, the vectorization of the current sample sequence refers to converting the tokenized and labeled input current sample sequence into digital information that can be recognized by the Bert module.

[0164] Step S403, the vector sequence of the current sample sequence is divided into a plurality of first attention heads by the multi-layer BERT attention layer, and the first attention weight of each first attention head is determined, and the embedding information of the current sample sequence is determined based on the first attention weight.

[0165] Wherein, the input of the first layer BERT attention layer is the vector sequence of the current sample sequence output by the embedding layer, and the input of each subsequent layer BERT attention layer is the embedding information of the current sample sequence output by the previous layer BERT attention layer.

[0166] Exemplarily, the vector sequence of the current sample sequence is input into the first BERT attention layer. Based on the number of first attention heads in the first BERT attention layer, the vector sequence is divided into a corresponding number of first attention heads. The first attention weight of each first attention head is calculated, and the output of each first attention head is determined based on the first attention weight of each first attention head. The outputs of each first attention head are concatenated to obtain the embedding information of the current sample sequence in the first BERT attention layer. This embedding information is then used as the input to the second BERT attention layer. The new embedding information obtained based on the second BERT attention layer is then sequentially input into the third BERT attention layer, until the last BERT attention layer outputs the final embedding information of the current sample sequence.

[0167] Step S404: Process the embedding information of the current sample sequence output by the last BERT attention layer to obtain an embedding representation of the current sample sequence.

[0168] In this embodiment, the current sample sequence is encoded by the Bert module, and the current sample sequence is processed multiple times using the multi-layer BERT attention layer of the Bert module, so that more embedded information in the current sample sequence can be obtained to ensure the accuracy of subsequent prediction results.

[0169] In one embodiment, Figure 5 As shown, after step S104, the training method of the RNA sequence modification prediction model further includes the following steps S501 to S504 to visualize the attention weights to increase the interpretability of the modification prediction model.

[0170] Step S501: extract the second attention weight of each second attention head of the multi-head attention module.

[0171] Among them, the head matrix of the multi-head attention module has multiple second attention heads.

[0172] Exemplarily, the embedded representation is input into the head matrix of the linear layer of the multi-head attention module, and based on the number of second attention heads, the embedded representation is divided into a corresponding number of second attention heads, and the second attention weight of each second attention head is calculated respectively.

[0173] Step S502: determine the true positive samples in the modification site prediction results of the current sample sequence, traverse the true positive samples through the preset sliding window, determine the weight sum of the true positive samples covered by each preset sliding window according to the second attention weight, and extract the true positive samples whose weight sum meets the preset conditions to form a motif set.

[0174] The true positive sample refers to a sample in the modification site prediction result determined as having the target chemical modification and also having the target chemical modification in the input current sample sequence corresponding to the sample.

[0175] For example, the sample in the modification site prediction result determined as having the target chemical modification is determined based on the position of the sample to determine whether the sample at the corresponding position on the current sample sequence input into the modification prediction model also has the target chemical modification. If yes, the sample is determined as a true positive sample. This is repeated to determine all true positive samples in the modification site prediction result. Then, the window size and sliding step of the preset sliding window are defined according to the extraction accuracy, the preset sliding window with the window size and sliding step is used to traverse all true positive samples, the weight sum of the true positive samples covered by each preset sliding window is calculated, it is determined whether the weight sum is greater than a preset weight threshold, if yes, the true positive samples covered by the sliding window are extracted as high-weighted motif fragments, otherwise, the true positive samples are discarded. After all true positive samples are determined, all high-weighted motif fragments are extracted, the high-weighted motif fragments are sorted according to the weight sum, and the top high-weighted motif fragments are extracted according to a preset number to form a motif set.

[0176] After the extracted high-weighted motif fragments are sorted, the top high-weighted motif fragments are extracted to obtain the motif set. In this way, the motif fragments in the motif set are all typical motif fragments, which is beneficial to clustering of the motif fragments in the subsequent step, and can improve the accuracy and efficiency of clustering.

[0177] In step S503, the motif set is clustered to determine a key motif.

[0178] The key motif refers to a motif fragment in the motif set that can reflect the significant characteristics of the target chemical modification.

[0179] For example, the motif fragments in the motif set are searched and clustered according to a preset clustering number, and the motif fragments with significant characteristics are determined as the key motif according to the clustering result.

[0180] In an optional embodiment, the motif set is clustered, and the MEME algorithm is used to determine the key motif, including:

[0181] The MEME algorithm is used to search for motifs in the motif fragments in the motif set, and the searched motifs are clustered according to a preset clustering number to obtain a plurality of motif groups. The motifs in each motif group that meet the preset characteristic condition are determined as the key motif.

[0182] Wherein, the motif refers to a local conserved region in a gene sequence, or a small sequence pattern shared in a group of gene sequences.

[0183] For example, the MEME algorithm is used to search for motifs in the motif fragments of the motif set, and the searched motifs are clustered according to the set number of clusters, that is, the motifs are divided into different groups, and the motifs in each group have similar characteristics or functions. The motifs with significant characteristics in the clustered groups are determined as key motifs.

[0184] Step S504, matching the key motifs with existing motifs in the public database, extracting the matched key motifs and visualizing them.

[0185] Wherein, the public database uses an existing human CIS-BP database, which contains a large number of consistent RNA binding protein binding motifs, that is, RBP motifs.

[0186] For example, the existing motifs in the human CIS-BP database are used as reference motifs, and the key motifs are compared with the reference motifs, and the similarity scores of the key motifs and the reference motifs are calculated respectively. According to the similarity scores, the reference motifs are sorted and the reference motifs corresponding to the similarity scores meeting the set conditions are output, and the RNA binding protein binding motifs are visualized.

[0187] Wherein, the similarity score is the p value, which reflects the degree of similarity between the key motifs and the reference motifs. The smaller the p value, the higher the similarity, that is, the more similar the key motifs and the reference motifs.

[0188] In an optional embodiment, when the similarity comparison between the key motifs and the reference motifs is performed, the TOMTOM algorithm is used for matching, including:

[0189] The TOMTOM algorithm is used to determine the similarity scores between the key motifs and each reference motif, and the reference motif corresponding to the smallest similarity score is extracted, and the extracted reference motif is visualized.

[0190] For example, the TOMTOM algorithm is used to compare the motifs with the reference motifs in the human CIS-BP database, and the similarity scores (i.e. p values) between the motifs determined by the MEME algorithm and the reference motifs are calculated respectively. The reference motif with the smallest p value (i.e. the RNA binding protein binding motif) is extracted as the final result for visualization.

[0191] In this embodiment, by directly obtaining the second attention weight determined by the multi-head attention module, and by clustering the motif fragments, the key motif with significant characteristics can be determined. By comparing the similarity between the motif and the reference motif, the RNA binding protein binding motif with significant characteristics can be determined, thereby visualizing it to intuitively evaluate the performance of the model and intuitively output the interpretability of the model.

[0192] In one embodiment, in step S105, it is determined whether the model is trained based on a loss function, specifically including:

[0193] The loss function is determined, the deviation between the prediction result and the actual situation of each sample sequence is determined based on the loss function, and the model parameters of the modified prediction model are updated according to the deviation until the loss function converges or reaches the maximum iteration number, thereby obtaining the trained modified prediction model.

[0194] For example, the loss function of the modified prediction model is determined, the loss function is solved according to the prediction result and the actual situation of each sample sequence, the value of the loss function (i.e. the deviation between the prediction result and the actual situation of each sample sequence) is obtained, it is judged whether the deviation converges or whether the maximum iteration number is reached, if so, the training is ended, and the trained modified prediction model is obtained, otherwise, the model parameters of the modified prediction model are updated and optimized according to the deviation, and the modified prediction model is trained for a new round, until the deviation converges or the maximum iteration number is reached, thereby obtaining the trained modified prediction model.

[0195] The loss function can include but is not limited to a logarithmic loss function, a mean square error, a mean absolute error, etc. When determining the loss function, a loss function can be defined for the Bert module and the multi-head attention module respectively, and then a whole loss function can be defined for the modified prediction model. After solving the loss functions of the Bert module and the multi-head attention module respectively, the whole loss function of the modified prediction model is calculated based on the results of the loss functions of the two modules, to determine the deviation between the prediction result and the actual situation of each sample sequence.

[0196] In this embodiment, the loss functions of the Bert module and the multi-head attention module are determined respectively, and then a whole loss function of the modified prediction model is determined based on the loss functions, from part to whole, which is conducive to improving the training efficiency and accuracy of the model.

[0197] In one embodiment, as shown in Figure 6 An RNA sequence modification prediction method is provided. The method of this embodiment can be applied to a terminal or a server, and can also be applied to a system including a terminal and a server, and can be realized through the interaction of the terminal and the server.

[0198] The modification prediction method of the RNA sequence of the embodiment comprises the following steps S601 to S605. Among them:

[0199] Step S601, obtaining the RNA sequence to be predicted.

[0200] Step S602, inputting the RNA sequence to be predicted into the trained modification prediction model.

[0201] Among them, the modification prediction model of the embodiment is trained based on the training method of the RNA sequence modification prediction model of the above-mentioned embodiment.

[0202] Step S603, encoding the RNA sequence based on the Bert module of the modification prediction model to obtain the embedding representation of the RNA sequence.

[0203] Step S604, extracting attention representation of the embedding representation of the RNA sequence based on the multi-head attention module of the modification prediction model to obtain the attention representation of the RNA sequence.

[0204] Step S605, classification prediction based on the attention representation of the RNA sequence to obtain the site prediction result of the target chemical modification in the RNA sequence.

[0205] Among them, the specific process of encoding the RNA sequence by the Bert module and obtaining the embedding representation of the RNA sequence, the specific process of extracting the attention representation of the embedding representation of the RNA sequence by the multi-head attention module to obtain the attention representation of the RNA sequence, and the specific process of classification prediction based on the attention representation to obtain the final site prediction result of the target chemical modification can be referred to the description in the above-mentioned embodiment, which will not be repeated here.

[0206] In the embodiment, by obtaining the RNA sequence to be predicted, inputting the RNA sequence to be predicted into the trained modification prediction model, encoding the RNA sequence based on the Bert module of the modification prediction model to obtain the embedding representation of the RNA sequence, extracting the attention representation of the embedding representation of the RNA sequence based on the multi-head attention module of the modification prediction model to obtain the attention representation of the RNA sequence, and classification prediction based on the attention representation of the RNA sequence to obtain the site prediction result of the target chemical modification in the RNA sequence, a variety of RNA binding proteins closely related to the target chemical modification can be efficiently and accurately identified. The binding site, so that the prediction result can be effectively applied to identify cell line-specific binding motifs, and improve the identification accuracy.

[0207] In one specific application example, taking the target chemical modification of m6A modification as an example, the single nucleotide precision m6A modification site data quantified by GLORI (Glyoxaland nitrite-mediated deamination of unmethylated adenosine) technology adopts 293T and HeLa cell lines, and both of the two cell lines contain two sequencing samples. The m6A modification site data of RNA sequence refers to N6-methyladenosine modification, including splicing, stability, transportation and translation of mRNA, and m6A modification is usually realized by “methyltransferase complex”, including “writing” enzyme (such as METTL3, METTL14 and WTAP) and “erasing” enzyme (such as FTO and ALKBH5).

[0208] Based on this, in the prediction method of the RNA sequence modification model, the construction process of the multi-level classification database is as follows:

[0209] Firstly, the sites modified by m6A in the four sequencing samples are extracted as positive samples, and 25 nt is diffused from the center site of each positive sample to both sides to form positive samples with a length of 51 nt, 50 nt is diffused to both sides to form positive samples with a length of 101 nt, and 100 nt is diffused to both sides to form positive samples with a length of 201 nt, so as to obtain a plurality of positive sample sequences with different lengths. Then, the samples or motif fragments of non-m6A modification sites in the four sequencing samples are extracted as negative samples, and the negative samples are strictly non-overlapping with the positive samples on the genome. Considering the enrichment of m6A modification at the DRACH sequence motif, three different base restrictions are added to the negative samples to form three different classification difficulty negative sample sequences, namely the first one is not to add any restriction to the negative sample, the second one is to restrict the center position of the negative sample to be adenine (Adenine, abbreviated as A), and the third one is to restrict the center position of the negative sample to meet the base arrangement of DRACH (D represents A, G or U, R represents A or G, and H represents A, C or U). The genomic structure of the positive and negative samples is shown in Figure 7 Finally, the constructed positive sample sequences and negative sample sequences are de-duplicated using CDHIT software with a sequence similarity of 0.9 as the threshold, and finally a plurality of training sets of 48000 samples and test sets of 12000 samples are obtained by one-to-one sampling of positive and negative samples, so as to construct a multi-level classification database.

[0210] In the present example, when constructing a multi-level classification database, based on the latest published single nucleotide precision m6A absolute quantitative sequencing data, an m6A classification dataset (i.e. positive sample sequence) containing three length samples is constructed from scratch, and by adding different restrictions to the negative samples, multiple datasets with different classification difficulties (i.e. negative sample sequence) can be constructed.

[0211] In the present example, based on the deep learning framework Bert module of the self-supervised task, an end-to-end m6A modification prediction algorithm is built from scratch by combining the multi-head attention mechanism to form the m6ABert prediction model (i.e. modification prediction model). In the specific implementation, when encoding, the BERT attention layer number of the Bert module in the m6ABert prediction model is 4 layers, each BERT attention layer contains 4 first attention heads, and the hidden layer dimension is 128. When compared with other algorithms, the multi-head attention module uses 8 second attention heads, and the hidden layer dimension is 128. In the subsequent weight visualization part, first, the sequence length dimension is compressed to 1 using max pooling, and then two fully connected layers are used for classification. The learning rate during training is 1e-3, the weight decay is 1e-3, the batch size is 256, the dropout parameter is 0.1, and the model is trained on 4 Nvidia 3090 graphics cards.

[0212] As shown in Figure 8 As shown in the structure diagram of the m6ABert prediction model, the m6ABert prediction model first encodes the input sample sequence using the Bert module to obtain the embedding representation of the sample sequence. This process needs to be trained from scratch for each sample sequence in the multi-level classification database. Subsequently, the embedding representation of the sample sequence is transmitted to the multi-head attention module, and the second attention weight is determined through the multi-head attention module. Finally, after the pooling layer, the classification is performed through the fully connected layer to form an end-to-end m6A prediction process.

[0213] In addition, after the Bert module, the multi-head attention module can also visualize the model weights based on the attention mechanism, then search for important motif fragments through sliding window, extract key motifs to form a motif set, and further use the MEME algorithm to cluster the key motifs (i.e. key motifs). For the found key motifs, the TOMTOM algorithm is used to search and compare them with existing motifs in the publicly released motif database, and finally the matched motifs are visualized and displayed.

[0214] Specifically, when visualizing the second attention weight, the second attention weight is extracted for each second attention head. In the present example, the extraction formula of the second attention weight is as follows:

[0215]

[0216] wherein, Attention(Q, K, V) is the calculation process of attention mechanism, softmax() is the operation of normalized exponential function, Q, K, V are respectively the data corresponding to query, key, value matrix, d k is the length of the extracted vector.

[0217] For the extracted second attention weight, a sliding window with a length of 18 nt is used to traverse all true positive samples, and high weight base sequence fragments of each sample are extracted with a threshold of weight sum greater than 2, and about the top 200 base sequence fragments are extracted through sorting. The 200 typical base sequence fragments extracted are searched for motifs and clustered using the MEME algorithm, and when clustering, the number of motif clusters is set to 3-5. For the motifs with significance, the TOMTOM algorithm is used to compare with the existing motifs in the human CIS-BP database, and the RNA binding protein binding motif with the smallest p value is further extracted for visualization.

[0218] Please refer to Figure 9 , Figure 9 The relationship curve between the position of m6A modification on the m6A modification site prediction data sample sequence obtained based on the m6ABert prediction model and the attention weight of the position is shown, and the value of the attention weight corresponding to the position with m6A modification is greater. It can be seen that the m6ABert prediction model of the embodiment can accurately predict the m6A modification site. At the same time, please refer to Figure 10 , Figure 10 The most common motif in m6A modification is shown. It can be seen that the m6ABert prediction model of the embodiment successfully identifies the key motif and visualizes it, realizing intuitive evaluation of model performance and interpretability of output model.

[0219] In an application example, in order to further illustrate that the method of the embodiment has more accurate prediction effect compared with other models (including UniRNA, MASS, One-hot, DeepM6ASeq-RNN, DeepM6ASeq-CNN), the prediction effects of the m6ABert prediction model and the aforementioned other models are compared based on the constructed multi-level classification database, and at the same time, in order to ensure the fairness of the comparison, all models are retrained on each sample sequence respectively. In addition, One-hot encoding is added to access a simple convolutional neural network as a benchmark for horizontal comparison. The prediction effect is as follows Figure 11As shown, the m6ABert prediction model achieves better prediction accuracy on three different classification difficulty datasets, and the advantage is more obvious as the classification difficulty increases. However, even a simple One-hot encoding plus classifier can achieve high accuracy on the two datasets with no restrictions on negative samples and A-site restrictions, indicating that the differences between m6A modification site sequences and random sequences are relatively easy to distinguish.

[0220] It should be understood that although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0221] Based on the same inventive concept, the embodiments of the present application also provide a RNA sequence modification prediction model training device for implementing the RNA sequence modification prediction model training method described above. The problem-solving implementation scheme provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more RNA sequence modification prediction model training device embodiments provided below can refer to the limitations of the RNA sequence modification prediction model training method described above, which will not be repeated here.

[0222] In one embodiment, as Figure 12 shown, a RNA sequence modification prediction model training device is provided, comprising: a data acquisition module 1201, a sample extraction module 1202, a database construction module 1203, a site prediction module 1204 and a model updating module 1205, wherein:

[0223] The data acquisition module 1204 is configured to acquire modification site sequencing data of RNA sequences; the modification site sequencing data is obtained by sequencing at least two types of cell lines respectively.

[0224] The sample extraction module 1202 is configured to obtain a plurality of positive sample sequences with different lengths based on the sites with target chemical modification in the modification site sequencing data, and obtain a plurality of negative sample sequences with different bases added at the central position based on the sites without target chemical modification in the modification site sequencing data; the positive sample sequences and the negative sample sequences do not overlap in genomic positions.

[0225] The database construction module 1203 is configured to obtain a multi-level classification database based on the positive sample sequences and the negative sample sequences.

[0226] The site prediction module 1204 is configured to input each sample sequence in the multi-level classification database into a modification prediction model to be trained to obtain a prediction result of a modification site of each sample sequence.

[0227] The model updating module 1205 is configured to update model parameters of the modification prediction model to be trained based on the prediction result and an actual situation of each sample sequence until a trained modification prediction model is obtained.

[0228] In an optional embodiment, the training device of the RNA sequence modification prediction model further includes:

[0229] The weight extraction module is configured to extract second attention weights of each second attention head of the multi-head attention module.

[0230] The motif extraction module is configured to determine true positive samples in the prediction result of the modification site of the current sample sequence, traverse the true positive samples by using a preset sliding window, determine a weight sum of the true positive samples covered by each preset sliding window according to the second attention weights, and extract true positive samples with a weight sum satisfying a preset condition to form a motif set.

[0231] The clustering module is configured to cluster the motif set to determine a key motif.

[0232] The matching module is configured to match the key motif with existing motifs in a public database, extract the key motif that is successfully matched, and visualize the key motif.

[0233] Based on the same inventive concept, the embodiments of the present application also provide a modification prediction device of an RNA sequence for implementing the modification prediction method of the RNA sequence as described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more modification prediction devices of the RNA sequence provided below can refer to the limitations of the modification prediction method of the RNA sequence in the above text, which will not be described here again.

[0234] In one embodiment, as shown in Figure 13 An RNA sequence modification prediction device is provided, which includes a sequence acquisition module 1301, a sequence input module 1302, an embedded representation extraction module 1303, an attention representation extraction module 1304, and a classification prediction module 1305, wherein:

[0235] The sequence acquisition module 1301 is configured to acquire an RNA sequence to be predicted.

[0236] The sequence input module 1302 is configured to input an RNA sequence to be predicted into the trained modification prediction model.

[0237] The embedding representation extraction module 1303 is configured to encode the RNA sequence based on a Bert module of the modification prediction model to obtain an embedding representation of the RNA sequence.

[0238] The attention representation extraction module 1304 is configured to perform attention representation extraction on the embedding representation of the RNA sequence based on a multi-head attention module of the modification prediction model to obtain an attention representation of the RNA sequence.

[0239] The classification prediction module 1305 is configured to perform classification prediction based on the attention representation of the RNA sequence to obtain a modification site prediction result of the RNA sequence.

[0240] The above modules in the training device and the modification prediction device of the RNA sequence can be realized by software, hardware, or a combination thereof. The above modules can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to the above modules.

[0241] In an exemplary embodiment, a computer device, which can be a terminal, is provided. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus. The communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, near field communication (NFC), or other technologies. The computer program is executed by the processor to implement a training method of an RNA sequence modification prediction model and / or a modification prediction method of an RNA sequence. The display unit of the computer device is configured to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball, or a touchpad arranged on the shell of the computer device. The input device can also be an external keyboard, a touchpad, or a mouse, etc.

[0242] It can be understood that in some other embodiments, the computer device can also be a server, which includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus. The communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store RNA sequence and modification site sequencing data, positive sample sequences, negative sample sequences, model parameters, attention weights, and other data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals through a network connection. The computer program is executed by the processor to implement a training method of an RNA sequence modification prediction model and / or a modification prediction method of an RNA sequence.

[0243] Those skilled in the art can understand that the structure of the computer device described above is only part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those described, or combine certain components, or have a different arrangement of components.

[0244] In one exemplary embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the training method of the RNA sequence modification prediction model and / or the modification prediction method of the RNA sequence of the above-mentioned embodiments when executing the computer program.

[0245] In one embodiment, a computer readable storage medium is provided, storing a computer program thereon, and the computer program implements the training method of the RNA sequence modification prediction model and / or the modification prediction method of the RNA sequence of the above-mentioned embodiments when executed by a processor.

[0246] In one embodiment, a computer program product is provided, comprising a computer program, and the computer program implements the training method of the RNA sequence modification prediction model and / or the modification prediction method of the RNA sequence of the above-mentioned embodiments when executed by a processor.

[0247] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0248] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0249] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0250] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for training an RNA sequence modification prediction model, characterized in that: The method comprises: obtaining modification site sequencing data of RNA sequences; the modification site sequencing data is obtained based on at least two types of cell lines respectively; based on the sites with target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths are obtained, and based on the sites without the target chemical modification in the modification site sequencing data, a plurality of negative sample sequences with different bases added at the central position are obtained; the positive sample sequences and the negative sample sequences do not overlap in genomic position; obtaining a multi-level classification database based on the positive sample sequences and negative sample sequences; inputting each sample sequence in the multi-level classification database into a modification prediction model to be trained respectively to obtain a prediction result of the modification site of each sample sequence; the modification prediction model to be trained is obtained based on the joint construction of a Bert module and a multi-head attention module; updating the model parameters of the modification prediction model to be trained based on the prediction result and the actual situation of each sample sequence until a trained modification prediction model is obtained; the method comprises: extracting the sites with target chemical modification from the modification site sequencing data obtained based on the at least two types of cell lines respectively; determining a plurality of extension lengths, and extending to the determined extension lengths from both sides of the center site with the sites with target chemical modification as the center site to obtain a plurality of the positive sample sequences; the method comprises: extracting the sites without target chemical modification from the modification site sequencing data obtained based on the at least two types of cell lines respectively, determining the similarity between the sites without target chemical modification and the sites with target chemical modification; based on the similarity, the sites without target chemical modification are divided into a plurality of negative sample sets, and different base restrictions are determined for the plurality of negative sample sets based on the similarity; the base restriction at least includes no addition of the center position of the site without target chemical modification, limiting the base of the center position of the site without target chemical modification to adenine, and limiting the base of the center position of the site without target chemical modification to the base arrangement satisfying DRACH; the corresponding base restriction is added to each negative sample set to obtain a plurality of the negative sample sequences.

2. The method of claim 1, wherein, the method comprises: determining any sample sequence in the multi-level classification database as a current sample sequence; obtaining an initial modification prediction model; the initial modification prediction model is obtained based on the joint construction of a Bert module and a multi-head attention module; encoding the current sample sequence based on the Bert module to obtain the embedding representation of the current sample sequence; The multi-head attention module is used for attention representation extraction on the embedding representation of the current sample sequence, to obtain an attention representation of the current sample sequence; The attention representation of the current sample sequence is used for classification prediction, to obtain a modification site prediction result of the current sample sequence.

3. The method of claim 2, wherein, The embedding representation of the current sample sequence is obtained by encoding the current sample sequence based on the Bert module, including: The current sample sequence is input into the Bert module; the Bert module has multiple BERT attention layers, and each BERT attention layer has multiple first attention heads; The current sample sequence is vectorized by the embedding layer of the Bert module, to obtain a vector sequence of the current sample sequence; The vector sequence of the current sample sequence is divided into multiple first attention heads by the multi-layer BERT attention layer, and a first attention weight of each first attention head is determined; the embedding information of the current sample sequence is determined based on the first attention weights; the input of the first layer of the BERT attention layer is the vector sequence of the current sample sequence output by the embedding layer, and the input of each subsequent layer of the BERT attention layer is the embedding information of the current sample sequence output by the previous layer of the BERT attention layer; The embedding information of the current sample sequence output by the last layer of the BERT attention layer is processed, to obtain the embedding representation of the current sample sequence.

4. The method of claim 2, wherein, After the classification prediction based on the attention representation of the current sample sequence is performed to obtain the modification site prediction result of the current sample sequence, the method further includes: Second attention weights of each second attention head of the multi-head attention module are extracted; True positive samples in the modification site prediction result of the current sample sequence are determined, the true positive samples are traversed by a preset sliding window, a weight sum of the true positive samples covered by each preset sliding window is determined according to the second attention weights, and true positive samples whose weight sum satisfies a preset condition are extracted to form a motif set; The motif set is clustered to determine a key motif; The key motif is matched with existing motifs in a public database, and the key motif that is successfully matched is extracted and visualized.

5. The method of claim 4, wherein, The clustering of the motif set to determine the key motif includes: A MEME algorithm is used to search for motifs in motif fragments of the motif set, and the searched motifs are clustered according to a set number of clusters, to obtain multiple motif groups; Motifs in each motif group that meet a preset characteristic condition are determined as key motifs.

6. The method of claim 4, wherein, The matching of the key motif with existing motifs in a public database to extract the key motif that is successfully matched and visualize the key motif includes: A TOMTOM algorithm is used to determine similarity scores between the key motif and each reference motif, and a reference motif corresponding to the minimum similarity score is extracted, and the extracted reference motif is visualized.

7. The method of claim 1, wherein, The model parameters of the to-be-trained modification prediction model are updated based on the prediction result and the actual situation of each sample sequence until a trained modification prediction model is obtained, comprising: The model parameters of the to-be-trained modification prediction model are updated based on the prediction result and the actual position information of the site with the target chemical modification in the positive sample sequence and the actual position information of the site without the target chemical modification in the negative sample sequence until a trained modification prediction model is obtained.

8. A method for predicting RNA sequence modifications, characterized in that: The method comprises: Obtaining an RNA sequence to be predicted; Inputting the RNA sequence to be predicted into the trained modification prediction model; the modification prediction model is trained based on the method of any one of claims 1-7; Encoding the RNA sequence based on the Bert module of the modification prediction model to obtain the embedded representation of the RNA sequence; Extracting the attention representation of the embedded representation of the RNA sequence based on the multi-head attention module of the modification prediction model to obtain the attention representation of the RNA sequence; Performing classification prediction based on the attention representation of the RNA sequence to obtain the site prediction result of the target chemical modification in the RNA sequence.

Images