RNA sequence modification prediction model training method, prediction method and related device

Abstract

The invention relates to a training method of an RNA sequence modification prediction model, a prediction method and a related device. The method comprises the following steps: acquiring modification site sequencing data of an RNA sequence; on the basis of sites with target chemical modification and sites without target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths and a plurality of negative sample sequences with different basic groups added at the central position are obtained respectively, so that a multi-level classification database is constructed; inputting each sample sequence in the multi-level classification database into a to-be-trained modification prediction model to obtain a prediction result of a modification site of each sample sequence; and on the basis of the prediction result and the actual condition of each sample sequence, updating model parameters of the to-be-trained modification prediction model until the trained modification prediction model is obtained. Compared with the prior art, the method can improve the training efficiency and precision of the model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method, a prediction method and related devices for an RNA sequence modification prediction model. Background Art

[0002] During RNA processing and export from the nucleus, in addition to the splicing of introns and exons, reversible chemical modifications are added to the RNA sequence itself to regulate genes in various cellular processes.

[0003] In recent years, the development of epigenomics technology has promoted the identification of chemical modifications at the whole genome level. However, since omics-based experimental methods are time-consuming and expensive, and the sequencing technology for detecting modifications has a high threshold, it has always been difficult to quickly and accurately identify chemical modification sites based on sequences. In order to make up for the bottleneck of experimental technology, the use of computational methods to identify chemical modification sites has become an important research direction in this field.

[0004] Currently, computational methods for identifying chemical modification sites usually use one-hot encoding to represent RNA sequences, and train prediction models based on the encoded RNA sequences. This cannot directly capture the characteristics of RNA sequences, resulting in low prediction accuracy of chemical modification sites. Summary of the invention

[0005] Based on this, it is necessary to provide a training method, prediction method and related device for an RNA sequence modification prediction model that can improve prediction accuracy in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for training an RNA sequence modification prediction model, the method comprising:

[0007] Obtaining modification site sequencing data of RNA sequences; wherein the modification site sequencing data is obtained based on sequencing of at least two types of cell lines respectively;

[0008] Based on the sites with target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths are obtained, and based on the sites without the target chemical modification in the modification site sequencing data, a plurality of negative sample sequences with different bases added at the center position are obtained; the positive sample sequences and the negative sample sequences do not overlap in genomic position;

[0009] Obtain a multi-level classification database based on the positive sample sequence and the negative sample sequence;

[0010] Inputting each sample sequence in the multi-level classification database into the modification prediction model to be trained to obtain the prediction result of the modification site of each sample sequence;

[0011] Based on the prediction results and the actual situation of each sample sequence, the model parameters of the modification prediction model to be trained are updated until a trained modification prediction model is obtained.

[0012] In one embodiment, the sites with target chemical modification in the modification site sequencing data are used to obtain multiple positive sample sequences of different lengths, including:

[0013] Extracting sites with target chemical modifications from the modification site sequencing data obtained by sequencing the at least two types of cell lines respectively;

[0014] Based on the sites with target chemical modifications, extensions of different lengths are performed to obtain a variety of positive sample sequences.

[0015] In one embodiment, the extension of different lengths based on the site with the target chemical modification includes:

[0016] A plurality of extension lengths are determined, and the site with the target chemical modification is taken as the central site, and the extensions are respectively extended to the determined extension lengths on both sides of the central site.

[0017] In one embodiment, the sites without the target chemical modification in the modification site sequencing data are used to obtain multiple negative sample sequences with different bases added at the center position, including:

[0018] The sites without target chemical modification are extracted from the modification site sequencing data obtained by sequencing the at least two types of cell lines respectively. Based on the sites without target chemical modification, different bases are added at the center position for restriction to obtain a plurality of negative sample sequences.

[0019] In one embodiment, the restriction is performed by adding different bases at the central position based on the site without the target chemical modification, including:

[0020] determining a similarity between the site without the target chemical modification and the site with the target chemical modification;

[0021] Based on the similarity, the sites without target chemical modification are divided into a plurality of negative sample sets, and different base restrictions are determined for the plurality of negative sample sets based on the similarity;

[0022] Add corresponding base restrictions to each negative sample set respectively.

[0023] In one embodiment, the base restriction includes at least adding no restriction to the central position of the site without the target chemical modification, restricting the base at the central position of the site without the target chemical modification to adenine and / or restricting the base at the central position of the site without the target chemical modification to a base arrangement that satisfies DRACH.

[0024] In one embodiment, obtaining a multi-level classification database based on the positive sample sequence and the negative sample sequence includes:

[0025] Performing deduplication processing on the plurality of positive sample sequences and the plurality of negative sample sequences respectively;

[0026] Based on the multiple positive sample sequences after deduplication and the multiple negative sample sequences after deduplication, positive sample sequences and negative sample sequences are selected according to a preset ratio to form the multi-level classification database; the multi-level classification database includes multiple sample sequences, and different sample sequences have at least different lengths or different bases at the center position.

[0027] In one embodiment, the step of inputting each sample sequence in the multi-level classification database into the modification prediction model to be trained to obtain the prediction result of the modification site of each sample sequence comprises:

[0028] Determine any sample sequence in the multi-level classification database as a current sample sequence;

[0029] Obtaining an initial modification prediction model; the initial modification prediction model is constructed based on a Bert module and a multi-head attention module;

[0030] Encoding the current sample sequence based on the Bert module to obtain an embedded representation of the current sample sequence;

[0031] Extracting attention representation from the embedded representation of the current sample sequence based on the multi-head attention module to obtain the attention representation of the current sample sequence;

[0032] Classification prediction is performed based on the attention representation of the current sample sequence to obtain a modification site prediction result of the current sample sequence.

[0033] In one embodiment, encoding the current sample sequence based on the Bert module to obtain an embedded representation of the current sample sequence includes:

[0034] Input the current sample sequence into the Bert module; the Bert module has multiple BERT attention layers, and each BERT attention layer has multiple first attention heads;

[0035] Vectorize the current sample sequence through the embedding layer of the Bert module to obtain a vector sequence of the current sample sequence;

[0036] The vector sequence of the current sample sequence is divided into multiple first attention heads through the multi-layer BERT attention layer and the first attention weight of each first attention head is determined, and the embedding information of the current sample sequence is determined based on each first attention weight; the input of the first layer of the BERT attention layer is the vector sequence of the current sample sequence output by the embedding layer, and the input of each subsequent layer of the BERT attention layer is the embedding information of the current sample sequence output by the previous layer of the BERT attention layer;

[0037] The embedding information of the current sample sequence output by the last BERT attention layer is processed to obtain the embedding representation of the current sample sequence.

[0038] In one embodiment, after performing classification prediction based on the attention representation of the current sample sequence to obtain the modification site prediction result of the current sample sequence, the method further includes:

[0039] Extracting the second attention weight of each second attention head of the multi-head attention module;

[0040] Determine true positive samples in the modification site prediction result of the current sample sequence, traverse the true positive samples through a preset sliding window, determine the weight sum of the true positive samples covered by each preset sliding window according to the second attention weight, and extract the true positive samples whose weight sum meets the preset condition to form a motif set;

[0041] Clustering the motif set to determine key motifs;

[0042] The key motifs are matched with existing motifs in public databases, and the successfully matched key motifs are extracted and visualized.

[0043] In one embodiment, clustering the motif set to determine the key motifs comprises:

[0044] The MEME algorithm is used to search for motifs in the motif fragments of the motif set, and the searched motifs are clustered according to a set number of clusters to obtain multiple motif groups;

[0045] The motifs in each motif group that meet the preset characteristic conditions are identified as key motifs.

[0046] In one embodiment, matching the key motif with existing motifs in a public database, extracting the successfully matched key motifs and visualizing them, comprises:

[0047] The TOMTOM algorithm is used to determine the similarity scores between the key motif and each reference motif, extract the reference motif corresponding to the minimum similarity score, and visualize the extracted reference motif.

[0048] In one embodiment, the updating of the model parameters of the modification prediction model to be trained based on the prediction result and the actual situation of each sample sequence until a trained modification prediction model is obtained comprises:

[0049] Based on the prediction results and the actual position information of the sites with the target chemical modification in the positive sample sequence and the actual position information of the sites without the target chemical modification in the negative sample sequence, the model parameters of the modification prediction model to be trained are updated until a trained modification prediction model is obtained.

[0050] In a second aspect, the present application also provides a method for predicting modification of an RNA sequence, the method comprising:

[0051] Obtaining the RNA sequence to be predicted;

[0052] Inputting the RNA sequence to be predicted into a trained modification prediction model; the modification prediction model is trained based on the training method of the RNA sequence modification prediction model;

[0053] Encoding the RNA sequence based on the Bert module of the modification prediction model to obtain an embedded representation of the RNA sequence;

[0054] Extracting attention representation from the embedded representation of the RNA sequence using a multi-head attention module based on the modification prediction model to obtain an attention representation of the RNA sequence;

[0055] Classification prediction is performed based on the attention representation of the RNA sequence to obtain a prediction result of the site of the target chemical modification in the RNA sequence.

[0056] In a third aspect, the present application also provides a training device for an RNA sequence modification prediction model, the device comprising:

[0057] A data acquisition module, used to obtain modification site sequencing data of RNA sequences; the modification site sequencing data is obtained by sequencing at least two types of cell lines respectively;

[0058] A sample extraction module, for obtaining a plurality of positive sample sequences of different lengths based on the sites with target chemical modification in the modification site sequencing data, and for obtaining a plurality of negative sample sequences with different bases added at the center position based on the sites without the target chemical modification in the modification site sequencing data; the positive sample sequences and the negative sample sequences do not overlap in genomic position;

[0059] A database construction module, used to obtain a multi-level classification database based on the positive sample sequence and the negative sample sequence;

[0060] A site prediction module, used to input each sample sequence in the multi-level classification database into the modification prediction model to be trained, and obtain the prediction result of the modification site of each sample sequence;

[0061] The model updating module is used to update the model parameters of the modification prediction model to be trained based on the prediction result and the actual situation of each sample sequence until a trained modification prediction model is obtained.

[0062] In a fourth aspect, the present application further provides a device for predicting modification of an RNA sequence, the device comprising:

[0063] A sequence acquisition module is used to obtain the RNA sequence to be predicted;

[0064] A sequence input module, used to input the RNA sequence to be predicted into a trained modification prediction model;

[0065] An embedding representation extraction module, used for encoding the RNA sequence based on the Bert module of the modification prediction model to obtain an embedding representation of the RNA sequence;

[0066] An attention representation extraction module, used for extracting the attention representation of the embedded representation of the RNA sequence based on the multi-head attention module of the modification prediction model to obtain the attention representation of the RNA sequence;

[0067] A classification prediction module is used to perform classification prediction based on the attention representation of the RNA sequence to obtain a modification site prediction result of the RNA sequence.

[0068] In a fifth aspect, the present application also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the model training method and / or modified prediction method as described above when executing the computer program.

[0069] In a sixth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the model training method and / or modified prediction method as described above.

[0070] In a seventh aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the model training method and / or modified prediction method as described above.

[0071] The training method, prediction method and related device of the RNA sequence modification prediction model are as follows: by obtaining the modification site sequencing data of the RNA sequence; based on the sites with target chemical modification and the sites without target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths and a plurality of negative sample sequences with different bases added at the center are obtained respectively, and a multi-level classification database is obtained based on the positive sample sequences and the negative sample sequences; then, each sample sequence in the multi-level classification database is input into the modification prediction model to be trained respectively to obtain the prediction result of the modification site of each sample sequence; based on the prediction result and the actual situation of each sample sequence, the model parameters of the modification prediction model to be trained are updated until the modified prediction model after training is obtained. A multi-level classification database with different lengths and different classification difficulties can be constructed by simply processing the extracted positive and negative samples, and the data preprocessing efficiency is high; at the same time, the multi-level classification database is used as a standardized database to train the constructed end-to-end modification prediction model, which can improve the generalization ability of the model, and the model is trained by different sample sequences, which is conducive to the horizontal comparison between different model algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0073] Figure 1 A schematic diagram of a process for training a RNA sequence modification prediction model in one embodiment;

[0074] Figure 2 A schematic diagram of a process for obtaining a multi-level classification database in one embodiment;

[0075] Figure 3 A schematic diagram of a process for obtaining a prediction result of a modification site in one embodiment;

[0076] Figure 4 A schematic diagram of a process flow of a Bert module encoding process in one embodiment;

[0077] Figure 5 A schematic diagram of a process for weight visualization of a modification prediction model in one embodiment;

[0078] Figure 6 A schematic diagram of a method for predicting RNA sequence modifications in one embodiment;

[0079] Figure 7 A schematic diagram of a positive sample sequence and a negative sample sequence in an application example;

[0080] Figure 8 A structural diagram of a modified prediction model in an application example;

[0081] Fig. 9 This is a schematic diagram of the relationship between the motif position and the corresponding attention weight of each head of the modification prediction model in an application example;

[0082] Fig.10 A schematic diagram of the motif after weight visualization of the modification prediction model in an application example;

[0083] Fig.11 A diagram comparing the prediction effects of the modified prediction model and other models in an application example;

[0084] Fig.12 It is a structural block diagram of a training device for an RNA sequence modification prediction model in one embodiment;

[0085] Fig.13 FIG. 4 is a structural block diagram of an apparatus for predicting RNA sequence modifications in one embodiment. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0087] In one embodiment, Figure 1As shown, a training method for an RNA sequence modification prediction model is provided. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, portable wearable devices, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services, etc.

[0088] At present, there are hundreds of RNA modifications (chemical modifications) identified in cells, but there are not many modifications that are widely present on mRNA. Among them, m6A modification (N6-methyladenylation modification) is a very common type of chemical modification on mRNA, and it has been widely reported that m6A modification is involved in the regulation of the post-transcriptional level of cells and affects gene expression in multiple physiological processes. m6A modification is reversible. It is modified by a "writer" and can be removed by an "eraser". More importantly, m6A modification can be recognized by RNA binding proteins such as YTHDF1 / 2 / 3 and YTHDC1 / 2. m6A recognition proteins can recognize and bind to m6A-modified transcripts and regulate gene expression by regulating multiple processes, such as regulating mRNA stability, mRNA splicing, mRNA structure, translation efficiency, and miRNA biosynthesis. Studies have shown that m6A modification plays an important role in cells facing a variety of physiological or pathological conditions, especially in the occurrence and development of some diseases or cancers.

[0089] Chemical modifications in RNA sequences play a vital role in regulating RNA expression and function. Among them, m6A modification is one of the most abundant modification types in eukaryotic cells. The effects of m6A modification on RNA structure, RNA stability and RNA translation regulate key cellular processes such as disease occurrence and stress response. Therefore, efficient identification of m6A modification sites is one of the urgent needs for studying translation regulation in response to environmental changes.

[0090] Therefore, this embodiment proposes a training method for an RNA sequence modification prediction model, comprising the following steps S101 to S105. Among them:

[0091] Step S101, obtaining modification site sequencing data of RNA sequences.

[0092] Among them, RNA sequences are usually represented in the form of a string, containing four bases A, U, C, and G. The modification site sequencing data of the RNA sequence refers to data containing characteristic information of the modification site, and the characteristic information includes the location of the modification site, the type of modification, the degree of modification, additional information, etc. The modification site usually includes various chemical modifications, such as m6A, m1A, pseudouracil, m5C, etc. In this embodiment, the modification site sequencing data is obtained based on the latest published single nucleotide precision absolute quantitative sequencing data.

[0093] The modification site sequencing data of this embodiment is obtained based on sequencing of at least two types of cell lines.

[0094] Exemplarily, RNA sequences of two different cell lines are obtained respectively, and the modification site sequencing data of the RNA sequences are obtained based on the latest published single nucleotide precision absolute quantitative sequencing data.

[0095] Among them, the above two types of cell lines are 293T cell line and HeLa cell line, each cell line has two sequencing samples, that is, the modification site sequencing data of this embodiment includes the data of the sequencing samples, and these sequencing samples contain single nucleotide precision absolute quantitative information. Based on these sequencing samples, the modification sites in the corresponding cell lines can be identified, and by absolutely quantifying each modification site, the degree of modification at different sites can be understood.

[0096] Step S102, based on the sites with target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths are obtained, and based on the sites without target chemical modification in the modification site sequencing data, a plurality of negative sample sequences with different bases added at the center position are obtained.

[0097] Among them, the target chemical modification refers to the predicted target of the modification prediction model, that is, the type of chemical modification that needs to be determined on the modification site sequencing data. A site with a target chemical modification means that the chemical modification intended to exist in this embodiment exists at the corresponding site on the modification site sequencing data, and a site without a target chemical modification means that the chemical modification intended to exist in this embodiment does not exist at the corresponding site on the modification site sequencing. It can be understood that the site without a target chemical modification simply does not have the chemical modification intended in this embodiment, but may have other types of chemical modifications.

[0098] Among them, the positive sample sequence refers to a sequence composed of sites with target chemical modifications on the RNA sequence. Different lengths mean that the lengths of each positive sample sequence are different, but the lengths of each positive sample within each positive sample sequence are the same. For example, the length of each positive sample in a positive sample sequence of 51nt is 51nt, and the length of each positive sample in a positive sample sequence of 101nt is 101nt. By analogy, several positive sample sequences of different lengths can be constructed according to actual prediction needs.

[0099] Among them, the negative sample sequence refers to a sequence composed of sites or fragments on the RNA sequence that do not have the target chemical modification, so that the positive sample sequence and the negative sample sequence can be ensured to be strictly non-overlapping on the genome. Adding different bases at the center position means restricting the base type at the center position of each negative sample in the negative sample sequence to construct different classification difficulties on the negative sample, thereby obtaining a variety of negative sample sequences.

[0100] Exemplarily, the acquired modification site sequencing data is subjected to detection of target chemical modification, that is, four sequencing samples in the above-mentioned two types of cell lines are detected, and it is determined that the target chemical modification sites exist in the four sequencing samples, which are determined to be positive samples, and the positive samples are length-extended to obtain positive sample sequences of multiple different lengths; then, it is determined that the target chemical modification sites do not exist in the four sequencing samples, which are determined to be negative samples, and different base restrictions are imposed on the center positions of the negative samples to obtain negative sample sequences with multiple different restrictions.

[0101] Step S103, obtaining a multi-level classification database based on the positive sample sequence and the negative sample sequence.

[0102] The multi-level classification database includes multiple sample sequences at different levels, and different levels refer to samples in the sample sequences having different lengths and / or different classification difficulties (ie, different base restrictions).

[0103] Exemplarily, each positive sample sequence and each negative sample sequence are sampled separately to obtain multiple sample sequences including positive samples and negative samples, wherein the length of each positive sample in each sample sequence is the same and the base restriction type of each negative sample is the same, and between different sample sequences, at least the length of the positive samples is different or at least the base restriction type of the negative samples is different, thereby constructing a multi-level classification database.

[0104] Step S104, input each sample sequence in the multi-level classification database into the modification prediction model to be trained, and obtain the prediction result of the modification site of each sample sequence.

[0105] Exemplarily, according to training requirements, each sample sequence in the multi-level classification database is input into the modification prediction model to be trained, and based on the forward transfer of the modification prediction model, the characteristic information of the target chemical modification on the sample sequence is extracted, and the prediction result of the modification site on the corresponding sample sequence is output based on the characteristic information, so as to determine which positions on the sample sequence have the target chemical modification.

[0106] Among them, the modification prediction model realizes the prediction of modification sites based on one or more models including but not limited to convolutional network, BLSTM model, random forest, attention mechanism, and Bert model.

[0107] Step S105 , based on the prediction results and the actual situation of each sample sequence, the model parameters of the modification prediction model to be trained are updated until a trained modification prediction model is obtained.

[0108] The actual situation of each sample sequence refers to the actual position information of the sites with target chemical modification in the positive sample sequence and / or the actual position information of the sites without target chemical modification in the negative sample sequence, which is used to determine the accuracy of the prediction results.

[0109] Exemplarily, based on the prediction results and the actual position information of the sites with target chemical modification in the positive sample sequence, the deviation between the prediction results and the actual position information of the sites with target chemical modification is determined, and the model parameters are updated based on the deviation until the trained modification prediction model is obtained. Since the present embodiment is mainly for the prediction of target chemical modification sites, when updating the model parameters, it is mainly based on the prediction results and the actual position information of the sites with target chemical modification in the positive sample sequence for matching. It is understandable that in some other embodiments, the model parameters can also be updated based on the prediction results and the actual position information of the sites without target chemical modification in the negative sample sequence; or based on the prediction results, the actual position information of the sites with target chemical modification in the positive sample sequence and the actual position information of the sites without target chemical modification in the negative sample sequence for comprehensive matching, the model parameters are updated.

[0110] In the training method of the RNA sequence modification prediction model, the modification site sequencing data of the RNA sequence is obtained; based on the sites with target chemical modification and the sites without target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths and a plurality of negative sample sequences with different bases added at the center are obtained respectively, and a multi-level classification database is obtained based on the positive sample sequences and the negative sample sequences; then, each sample sequence in the multi-level classification database is input into the modification prediction model to be trained respectively to obtain the prediction result of the modification site of each sample sequence; based on the prediction result and the actual situation of each sample sequence, the model parameters of the modification prediction model to be trained are updated until the modified prediction model after training is obtained. A multi-level classification database with different lengths and different classification difficulties can be constructed by simply processing the extracted positive and negative samples, and the data preprocessing efficiency is high; at the same time, the multi-level classification database is used as a standardized database to train the constructed end-to-end modification prediction model, which can improve the generalization ability of the model, and the model is trained by different sample sequences, which is conducive to the horizontal comparison between different model algorithms.

[0111] In one embodiment, in step S102, based on the sites with target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths are obtained, including:

[0112] Extract sites with target chemical modifications from the modification site sequencing data obtained by sequencing at least two types of cell lines respectively; based on the sites with target chemical modifications, perform extensions of different lengths to obtain multiple positive sample sequences.

[0113] Exemplarily, sites with target chemical modifications are extracted from the modification site sequencing data obtained by sequencing at least two types of cell lines respectively, and determined as positive samples; the positive samples are extended to different lengths to obtain positive sample sequences of multiple different lengths.

[0114] In this embodiment, by extracting sites with target chemical modifications as positive samples and extending the positive samples to different lengths, positive sample sequences of various lengths can be obtained to expand the training samples of the model from the length level and improve the diversity of the training samples.

[0115] In one embodiment, the specific method for expanding the positive sample includes:

[0116] A plurality of extension lengths are determined, and the site with the target chemical modification is taken as the central site, and the extensions are respectively extended to the determined extension lengths on both sides of the central site.

[0117] Exemplarily, based on the training requirements, a variety of different extension lengths are determined, and then, with the positive sample as the center site, the extensions are respectively extended to the determined extension lengths on both sides of the center site. For example, based on the training requirements, the extension lengths are determined to be 25nt, 50nt, and 100nt respectively; for a positive sample with an extension length of 25nt, with the positive sample as the center site, the extensions are respectively extended by 25nt on both sides of the positive sample to obtain a positive sample with a length of 51nt (wherein the positive sample itself is 1nt in length, and the lengths of 25nt on both sides are extended to obtain a positive sample with a total length of 51nt), for positive samples with extension lengths of 50nt and 100nt, the aforementioned process can be referred to to obtain positive samples with lengths of 101nt and 201nt respectively; it is understandable that in some other embodiments, the extension length can also be 75nt, 150nt, etc., to obtain positive samples with lengths of 151nt and 301nt, and the specific numerical values ​​are not specifically limited in this embodiment.

[0118] In this embodiment, by extending different extension lengths to both sides of the positive sample as the center, positive samples of various lengths can be obtained, thereby forming a positive sample sequence, and data processing is efficient and simple.

[0119] In one embodiment, in step S102, based on the sites without target chemical modification in the modification site sequencing data, a plurality of negative sample sequences with different bases added at the center position are obtained, including:

[0120] Sites without target chemical modifications are extracted from the modification site sequencing data obtained by sequencing at least two types of cell lines respectively. Based on the sites without target chemical modifications, different bases are added at the center position for restriction to obtain multiple negative sample sequences.

[0121] Exemplarily, sites without target chemical modifications are extracted from the modification site sequencing data obtained by sequencing at least two types of cell lines respectively as negative samples; based on the negative samples, different bases are added to the center position of each negative sample for restriction to obtain multiple negative sample sequences.

[0122] In this embodiment, by extracting sites without target chemical modifications as negative samples and adding different bases to restrict the central positions of the negative samples, a variety of different negative sample sequences can be obtained to expand the training samples of the model from the classification difficulty level and further improve the diversity of training samples.

[0123] In one embodiment, a specific method of adding different bases at the center of a negative sample for restriction includes:

[0124] Determine the similarity between sites without the target chemical modification and sites with the target chemical modification, divide the sites without the target chemical modification into multiple negative sample sets based on the similarity, and determine different base restrictions for the multiple negative sample sets based on the similarity, and add corresponding base restrictions to each negative sample set.

[0125] The similarity between sites without target chemical modification and sites with target chemical modification refers to the similarity of feature information between negative samples and positive samples at the genome level. The negative sample set refers to a set of negative samples whose similarity to the positive samples is within the same threshold range.

[0126] Wherein, base restriction refers to adding different bases to the center position of the negative sample. In this embodiment, base restriction at least includes not adding restriction to the center position of the site without the target chemical modification, restricting the base at the center position of the site without the target chemical modification to adenine, and / or restricting the base at the center position of the site without the target chemical modification to a base arrangement that satisfies DRACH.

[0127] Among them, not adding restrictions to the central position of the site without the target chemical modification means not adding any base to the central position of the negative sample, that is, not performing any restriction processing; limiting the base at the central position of the site without the target chemical modification to adenine (i.e., Adenine, also known as "A") means limiting the nucleotide base pair at the central position of the negative sample to adenine; limiting the base at the central position of the site without the target chemical modification to a base arrangement that satisfies DRACH means limiting the arrangement of the nucleotide base pairs at the central position of the negative sample to DRACH, wherein "D" indicates that the nucleotide base pair at the site is "A", "G" or "U", "R" indicates that the nucleotide base pair at the site is "A" or "G", "A" is adenine "A", "C" indicates that the nucleotide base pair at the site is cytosine "C", and "H" indicates that the nucleotide base pair at the site is "A", "C" or "U".

[0128] Exemplarily, first, determine the feature similarity between the negative samples and each positive sample at the genome level; then, based on the feature similarity, divide the negative samples into multiple negative sample sets according to the size of the feature similarity, and the feature similarity between each negative sample and the positive sample in each negative sample set is equivalent; finally, determine different base restrictions for the multiple negative sample sets based on the feature similarity, and add corresponding base restrictions to each negative sample set.

[0129] Among them, when dividing the negative sample set, the negative sample is compared with the positive sample to determine the feature similarity between the negative sample and the positive sample, and the feature similarity is divided into multiple levels based on its size. In order to facilitate the subsequent sampling of positive and negative samples, the number of levels is the same as the number of positive sample sequences when dividing the levels. For example, taking three levels as an example, the first level of feature similarity is determined to be less than 25% or less than other proportions, the second level of feature similarity is determined to be within the range of 50%±25% or within other proportions, and the third level of feature similarity is determined to be more than 75% or more than other proportions, etc. In this way, according to the feature similarity of positive and negative samples, negative samples with feature similarity of the first level, the second level and the third level are respectively taken as a separate set to form multiple negative sample sets, and base restrictions are added according to different feature similarity levels.

[0130] In specific implementation, for the first level of negative sample set, no restriction is added to the center position of its negative sample, that is, for negative samples with low similarity or completely different from the positive sample, the simplest restriction (that is, zero restriction or no restriction) is made; for the second level of negative sample set, the nucleotide base pair at the center position of the negative sample is restricted to adenine, that is, for negative samples with moderate or slightly similar similarity to the positive sample, a restriction of moderate difficulty is made; and for the third level of negative sample set, the arrangement of the nucleotide base pair at the center position of the negative sample is limited to an arrangement method that satisfies DRACH, that is, for negative samples with high or almost the same similarity to the positive sample, the most difficult or complex restriction is made. In this way, three negative sample sequences with different classification difficulties can be constructed to expand the level of training data. It can be understood that in some other embodiments, when dividing the similarity of negative samples, more levels can be divided more subdivided to define different base restrictions, so as to obtain more levels of sample sequences.

[0131] In this embodiment, by dividing the negative samples into multiple sets based on the similarity with the positive samples and adding different base restrictions to each set, it is helpful to control the variables of the same sample sequence, and then construct negative sample sequences with multiple classification difficulties.

[0132] In one embodiment, Figure 2 As shown, in step S103, obtaining a multi-level classification database based on the positive sample sequence and the negative sample sequence includes the following steps S201 to S202. Among them:

[0133] Step S201 , performing deduplication processing on multiple positive sample sequences and multiple negative sample sequences respectively.

[0134] Among them, deduplication processing refers to deduplication of samples whose similarities meet certain conditions in multiple positive sample sequences and multiple negative sample sequences.

[0135] Exemplarily, a deduplication threshold is set, and for samples in each positive sample sequence and each negative sample sequence, the similarity of each sample with other samples in the sequence is calculated at the genome level, and samples with similarity greater than the deduplication threshold are removed to obtain deduplicated positive sample sequences and negative sample sequences.

[0136] Step S202 , based on the multiple positive sample sequences after deduplication and the multiple negative sample sequences after deduplication, select positive sample sequences and negative sample sequences according to a preset ratio to form a multi-level classification database.

[0137] The multi-level classification database includes multiple sample sequences, and different sample sequences are at least different in length or in bases at the center position.

[0138] Among them, selecting the positive sample sequence and the negative sample sequence according to the preset ratio means selecting a corresponding number of positive samples and negative samples for each positive sample sequence and different negative sample sequences according to the set selection ratio to form a sample sequence.

[0139] Exemplarily, the number of samples in each sample sequence is set, one of the positive sample sequences after deduplication and any one of the negative sample sequences after deduplication are selected, and the samples in the positive sample sequence and the negative sample sequence are cross-selected according to a preset ratio until the set number of samples is selected, and the selected samples are divided into a training set and a test set according to the set ratio to form a sample sequence; then, another negative sample sequence after deduplication is determined, and the positive sample sequence selected in the aforementioned process and the negative sample sequence determined later are subjected to sample selection and set division according to the aforementioned process to form another sample sequence, until the selected positive sample sequence is cross-selected with all negative sample sequences, another positive sample sequence is selected to repeat the aforementioned process until all positive sample sequences are cross-selected with all negative sample sequences, and the obtained multiple sample sequences are integrated to form a multi-level classification database.

[0140] Among them, cross-selecting samples in the positive sample sequence and the negative sample sequence means selecting all positive sample sequences with each negative sample sequence respectively to obtain a sample sequence with positive and negative samples. For example, for a positive sample sequence with a length of 51nt, cross-selecting is performed with a negative sample sequence without adding any restrictions, a negative sample sequence with a nucleotide base pair at the center being adenine, and a negative sample sequence with a nucleotide base pair at the center satisfying the DRACH arrangement, respectively, to form three sample sequences with the same length but different added base restrictions. For another example, for a positive sample sequence with a length of 101nt, cross-selecting is performed with a negative sample sequence without adding any restrictions, a negative sample sequence with a nucleotide base pair at the center being adenine, and a negative sample sequence with a nucleotide base pair at the center satisfying the DRACH arrangement, respectively, to form three sample sequences with the same length but different added base restrictions. By analogy, multiple sample sequences with different lengths and different classification difficulties are formed.

[0141] In specific implementation, public datasets (such as DeepM6ASeq, MASS, etc.) can also be integrated into the multi-level classification database to better expand the sample types of the dataset.

[0142] In this embodiment, deduplication processing is performed on multiple positive sample sequences and multiple negative sample sequences respectively to reduce redundant data and improve analysis efficiency; and based on the multiple positive sample sequences after deduplication and the multiple negative sample sequences after deduplication, positive sample sequences and negative sample sequences are selected according to a preset ratio to form a multi-level classification database. The sample sequence types in the database are diverse and are used for subsequent model training, which is conducive to horizontal evaluation of the model performance.

[0143] In one embodiment, Figure 3 As shown, in step S104, each sample sequence in the multi-level classification database is input into the modification prediction model to be trained, and the prediction result of the modification site of each sample sequence is obtained, which includes the following steps S301 to S305. Among them:

[0144] Step S301: determine any sample sequence in the multi-level classification database as the current sample sequence.

[0145] Exemplarily, a sample sequence is randomly selected from the multi-level classification database, and the sample sequence is determined as the current sample sequence to be used as an input of the modification prediction model for model training.

[0146] Step S302, obtaining an initial modification prediction model.

[0147] Among them, the initial modification prediction model is jointly constructed based on the Bert module and the multi-head attention module. Among them, the initial modification prediction model refers to the initialized modification prediction model, and the model parameters are all untrained initial parameters. The model parameters of the initial modification prediction model refer to the parameters that can be updated and optimized during the model training process to make the prediction results of the initial modification prediction model infinitely close to the true value; the model parameters of this embodiment include but are not limited to the attention layer, attention head, hidden layer dimension, etc. of the Bert module and the multi-head attention module.

[0148] Step S303: Encode the current sample sequence based on the Bert module to obtain an embedded representation of the current sample sequence.

[0149] Among them, the Bert module refers to a module that is built based on the Bert algorithm and can extract contextual information from the input data; the Bert module usually has an embedding layer, an encoding layer, and a Pooler layer connected in sequence. The embedding layer is used to convert the input sample sequence into a token format that the Bert algorithm can understand. During the conversion, a specific vocabulary is required to map each base in the sample sequence to the vocabulary of the Bert module. Since the standard Bert module is designed for natural language processing, the vocabulary of the Bert module can be customized according to the bases in the sample sequence so that the vocabulary of the Bert module can adapt to the sample sequence. The encoding layer is used to convert the sample sequence in token format into a high-dimensional vector representation, that is, embedded information. The Pooler layer is used to obtain the embedded information of each token to obtain the contextual information of the sample sequence, that is, the embedded representation.

[0150] Exemplarily, the current sample sequence is input into the initial modification prediction model, the embedding layer based on the Bert module converts the input current sample sequence into corresponding tokens, and then the encoding layer based on the Bert module converts the tokens into high-dimensional vector representations to obtain the embedding information corresponding to each token. Finally, the Pooler layer based on the Bert module obtains the embedding information of each token to obtain the context information of the RNA sequence, and then forms an embedded representation and inputs it into the multi-head attention module.

[0151] Step S304: extract the attention representation of the embedded representation of the current sample sequence based on the multi-head attention module to obtain the attention representation of the current sample sequence.

[0152] Among them, the multi-head attention module refers to a module that can calculate the attention weight of the input embedding representation based on the multi-head attention mechanism, and after determining the final attention representation, it identifies the sites with target chemical modifications in the sample sequence through subsequent processing. The multi-head attention module has a linear layer, a head matrix and a merging layer connected in sequence. The linear layer is used to perform a linear transformation on the input data (i.e., the embedding representation). The head matrix is ​​used to divide the embedded representation after the linear transformation according to the set number of attention heads and calculate the attention weight of each attention head respectively, and obtain the output of each attention head based on the attention weight. The merging layer is used to connect the output of each attention head end to end to form a new data structure to obtain the attention representation of the sample sequence.

[0153] The attention weight refers to the degree of association or importance between different positions in the embedded representation of the input.

[0154] Among them, attention representation refers to the attention distribution of different subspaces of the segmented embedding representation by running multiple independent attention heads in parallel, thereby obtaining the attention weights and corresponding output vectors.

[0155] Exemplarily, the embedded representation output by the Bert module is input into the multi-head attention module, and the input embedded representation is linearly transformed based on the linear layer of the multi-head attention module. Then, based on the head matrix of the multi-head attention module, the embedded representation after the linear transformation is divided into multiple attention heads according to the set number of attention heads, and the query (Query), key (Key) and value (Value) matrices of each attention head are calculated respectively. The attention weights of each attention head are calculated using the query matrix and the key matrix, and the value matrix is ​​weighted and summed using the attention weights to obtain the output of each attention head; finally, the output of each attention head is connected end to end based on the merging layer of the multi-head attention module to form a new data structure to obtain the attention representation of the current sample sequence.

[0156] Step S305, performing classification prediction based on the attention representation of the current sample sequence to obtain a modification site prediction result of the current sample sequence.

[0157] Exemplarily, the attention representation output by the multi-head attention module is input into the pooling layer for feature extraction to reduce the dimension of the attention representation and ensure that the extracted features contain the important features of the attention representation; the extracted features are then input into the fully connected layer for fusion with the global features (or all features) of the sample sequence and then input into the MLP layer, and the attention representation is classified and predicted based on the MLP layer to obtain the modification site prediction result of the current sample sequence.

[0158] In this embodiment, an end-to-end modification prediction model is constructed by fusing the Bert module and the multi-head attention module, and end-to-end training of the modification prediction model is realized through the constructed multi-level classification database, which can effectively improve the training accuracy and prediction accuracy of the model.

[0159] In one embodiment, Figure 4 As shown, in step S303, encoding the current sample sequence based on the Bert module to obtain the embedded representation of the current sample sequence includes the following steps S401 to S404. Among them:

[0160] Step S401: input the current sample sequence into the Bert module.

[0161] Among them, the Bert module has multiple layers of BERT attention layers, and each BERT attention layer has multiple first attention heads. Each BERT attention layer can process the current input sample sequence in sequence.

[0162] Step S402, vectorize the current sample sequence through the embedding layer of the Bert module to obtain a vector sequence of the current sample sequence.

[0163] Among them, vectorizing the current sample sequence refers to segmenting the input current sample sequence, adding tags, and converting it into digital information that can be recognized by the Bert module.

[0164] Step S403: divide the vector sequence of the current sample sequence into multiple first attention heads through multiple BERT attention layers, determine the first attention weight of each first attention head, and determine the embedding information of the current sample sequence based on each first attention weight.

[0165] Among them, the input of the first BERT attention layer is the vector sequence of the current sample sequence output by the embedding layer, and the input of each subsequent BERT attention layer is the embedding information of the current sample sequence output by the previous BERT attention layer.

[0166] Exemplarily, the vector sequence of the current sample sequence is input into the first BERT attention layer, and based on the number of first attention heads of the first BERT attention layer, the vector sequence is divided into the corresponding number of first attention heads, and the first attention weight of each first attention head is calculated respectively, and then the output of each first attention head is determined respectively based on the first attention weight of each first attention head, and the output of each first attention head is spliced ​​to obtain the embedding information of the current sample sequence on the first BERT attention layer. Then, the embedding information is used as the input of the second BERT attention layer, and the new embedding information is obtained based on the second BERT attention layer and then sequentially input into the third BERT attention layer, until the last BERT attention layer outputs the final embedding information of the current sample sequence.

[0167] Step S404: Process the embedding information of the current sample sequence output by the last BERT attention layer to obtain an embedding representation of the current sample sequence.

[0168] In this embodiment, the current sample sequence is encoded by the Bert module, and the current sample sequence is processed multiple times using the multi-layer BERT attention layer of the Bert module, so that more embedded information in the current sample sequence can be obtained to ensure the accuracy of subsequent prediction results.

[0169] In one embodiment, Figure 5 As shown, after step S104, the training method of the RNA sequence modification prediction model further includes the following steps S501 to S504 to visualize the attention weights to increase the interpretability of the modification prediction model. Among them:

[0170] Step S501, extracting the second attention weight of each second attention head of the multi-head attention module.

[0171] Among them, the head matrix of the multi-head attention module has multiple second attention heads.

[0172] Exemplarily, the embedded representation is input into the head matrix of the linear layer of the multi-head attention module, and based on the number of second attention heads, the embedded representation is divided into a corresponding number of second attention heads, and the second attention weight of each second attention head is calculated respectively.

[0173] Step S502, determine the true positive samples in the modification site prediction results of the current sample sequence, traverse the true positive samples through a preset sliding window, determine the weight sum of the true positive samples covered by each preset sliding window according to the second attention weight, and extract the true positive samples whose weight sum meets the preset conditions to form a motif set.

[0174] Among them, true positive samples refer to samples that are determined to have the target chemical modification in the modification site prediction results and are also samples with the target chemical modification in the corresponding input current sample sequence. The weight sum of true positive samples refers to the sum of the weights obtained by simply adding the second attention weights of all true positive samples within a preset sliding window coverage.

[0175] Exemplarily, a sample determined to have a target chemical modification in the modification site prediction result is determined, and based on the position of the sample, it is determined whether the sample at the corresponding position on the current sample sequence input to the modification prediction model also has the target chemical modification. If so, the sample is determined to be a true positive sample, and this process is repeated to determine all true positive samples in the modification site prediction result; then, the window size and sliding step of the preset sliding window are defined according to the extraction accuracy, and the preset sliding window of the window size and sliding step is used to traverse all true positive samples, and the weight sum of the true positive samples covered by each preset sliding window is calculated, and it is determined whether the weight sum is greater than the preset weight threshold. If so, the true positive sample covered by the sliding window is extracted as a high-weight motif fragment, otherwise, the true positive sample is discarded until all true positive samples are judged, all high-weight motif fragments are extracted, these high-weight motif fragments are sorted according to the weight sum, and the top-ranked high-weight motif fragments are extracted according to the preset number to form a motif set.

[0176] Among them, after sorting the extracted high-weight motif fragments, the top-ranked high-weight motif fragments are extracted to obtain a motif set. In this way, the motif fragments in the motif set are all typical motif fragments, which is conducive to clustering the motif fragments in the subsequent steps, which can improve the accuracy of clustering and the efficiency of clustering.

[0177] Step S503: cluster the motif set to determine key motifs.

[0178] Among them, the key motif refers to the motif fragment in the motif set that can reflect the significant characteristics of the target chemical modification;

[0179] Exemplarily, the motif fragments in the motif set are searched and clustered according to a set number of clusters, and based on the clustering results, the motif fragments with significant characteristics are determined as key motifs.

[0180] In an optional embodiment, clustering the motif set and using the MEME algorithm to determine the key motifs includes:

[0181] The MEME algorithm is used to search for motifs in the motif fragments of the motif set, and the searched motifs are clustered according to the set number of clusters to obtain multiple motif groups; the motifs in each motif group that meet the preset characteristic conditions are determined as key motifs.

[0182] Among them, motif refers to a local conserved region in a gene sequence, or a small sequence pattern shared by a group of gene sequences.

[0183] Exemplarily, the MEME algorithm is used to search for motifs in the motif fragments of the motif set, and the searched motifs are clustered according to the set number of clusters, that is, the motifs are divided into multiple different groups, and the motifs in each group have similar characteristics or functions, and the motifs with significant characteristics in each group of motifs after clustering are determined as key motifs.

[0184] Step S504, matching the key motifs with existing motifs in the public database, extracting successfully matched key motifs and visualizing them.

[0185] Among them, the public database adopts the existing human CIS-BP database, which contains a large number of consistent RNA-binding protein binding motifs, namely, RBP motifs.

[0186] Exemplarily, the existing motifs in the human CIS-BP database are used as reference motifs, the key motifs are compared with the reference motifs, the similarity scores of the key motifs and the reference motifs are calculated respectively, the reference motifs are sorted according to the similarity scores, and the RNA-binding protein binding motifs corresponding to the reference motifs whose similarity scores meet the set conditions are output and visualized.

[0187] Among them, the similarity score is the p-value, which reflects the degree of similarity between the key motif and the reference motif. The smaller the p-value, the higher the similarity, that is, the more similar the key motif is to the reference motif.

[0188] In an optional embodiment, when performing a similarity comparison between a key motif and a reference motif, a TOMTOM algorithm is used for matching, including:

[0189] The TOMTOM algorithm was used to determine the similarity scores between the key motif and each reference motif, and the reference motif corresponding to the minimum similarity score was extracted, and the extracted reference motif was visualized.

[0190] Exemplarily, the TOMTOM algorithm is used to compare the motif with the reference motif in the human CIS-BP database, and the similarity scores (i.e., p-values) between the motif determined by the MEME algorithm and the reference motif are calculated respectively, and the reference motif (i.e., RNA-binding protein binding motif) with the smallest p-value is extracted as the final result for visualization.

[0191] In this embodiment, by directly obtaining the second attention weight determined by the multi-head attention module and clustering the motif fragments, the key motif with significant characteristics can be determined, and the motif is compared with the reference motif for similarity, so that the RNA-binding protein binding motif with significant characteristics can be determined, and then visualized to intuitively evaluate the performance of the model and the interpretability of the intuitive output model.

[0192] In one embodiment, in step S105, determining whether the model training is completed based on a loss function specifically includes:

[0193] Determine the loss function, determine the deviation between the prediction result and the actual situation of each sample sequence based on the loss function, and update the model parameters of the modified prediction model according to the deviation until the loss function converges or reaches the maximum number of iterations, and obtain the trained modified prediction model.

[0194] Exemplarily, the loss function of the modification prediction model is determined, and the loss function is solved according to the prediction results and the actual situation of each sample sequence to obtain the value of the loss function (i.e., the deviation between the prediction results and the actual situation of each sample sequence), and it is judged whether the deviation converges or whether the set maximum number of iterations is reached. If so, the training is terminated to obtain the trained modification prediction model. Otherwise, the model parameters of the modification prediction model are updated and optimized according to the deviation, and a new round of training is performed on the modification prediction model until the deviation converges or the maximum number of iterations is reached to obtain the trained modification prediction model.

[0195] Among them, the loss function may include but is not limited to the logarithmic loss function, mean square error, mean absolute error, etc. When determining the loss function, a loss function can be defined for the Bert module and the multi-head attention module respectively, and then an overall loss function can be defined for the modified prediction model. After solving the loss functions of the Bert module and the multi-head attention module respectively, the loss function of the overall modified prediction model is calculated based on the results of the loss functions of the two to determine the deviation between the prediction results and the actual situation of each sample sequence.

[0196] In this embodiment, by determining the loss functions for the Bert module and the multi-head attention module respectively and then determining an overall loss function based on the modified prediction model, from part to whole, it is beneficial to improve the training efficiency and accuracy of the model.

[0197] In one embodiment, Figure 6 As shown, a method for predicting modification of an RNA sequence is provided. The method of this embodiment can be applied to a terminal or a server, and can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server.

[0198] The modification prediction method of RNA sequence of this embodiment includes the following steps S601 to S605. Among them:

[0199] Step S601, obtaining the RNA sequence to be predicted.

[0200] Step S602: input the RNA sequence to be predicted into the trained modification prediction model.

[0201] Among them, the modification prediction model of this embodiment is trained based on the training method of the RNA sequence modification prediction model of the above embodiment.

[0202] Step S603: Encode the RNA sequence based on the Bert module of the modification prediction model to obtain an embedded representation of the RNA sequence.

[0203] Step S604: The multi-head attention module based on the modification prediction model extracts the attention representation of the embedded representation of the RNA sequence to obtain the attention representation of the RNA sequence.

[0204] Step S605, performing classification prediction based on the attention representation of the RNA sequence to obtain a prediction result of the site of the target chemical modification in the RNA sequence.

[0205] Among them, the specific process of the Bert module in this embodiment encoding the RNA sequence and obtaining the embedded representation of the RNA sequence, the specific process of the multi-head attention module extracting the attention representation of the embedded representation of the RNA sequence to obtain the attention representation of the RNA sequence, and the specific process of classifying and predicting the attention representation to obtain the final target chemical modification site prediction result can be referred to the records in the above embodiments and will not be repeated here.

[0206] In this embodiment, by obtaining an RNA sequence to be predicted, the RNA sequence to be predicted is input into a trained modification prediction model, the RNA sequence is encoded based on the Bert module of the modification prediction model to obtain an embedded representation of the RNA sequence, and the multi-head attention module based on the modification prediction model extracts the attention representation of the RNA sequence from the embedded representation to obtain the attention representation of the RNA sequence, and classification prediction is performed based on the attention representation of the RNA sequence to obtain a prediction result of the site of the target chemical modification in the RNA sequence, which can efficiently and accurately identify the binding sites of a variety of RNA binding proteins that are closely related to the target chemical modification, so that the prediction results can be effectively used to identify cell line-specific binding motifs, thereby improving the identification accuracy.

[0207] In a specific application example, taking the target chemical modification as m6A modification as an example, the single nucleotide precision m6A modification site data quantified by the GLORI (Glyoxaland nitrite-mediated deamination of unmethylated adenosine) technology uses 293T and HeLa cell lines, and both cell lines contain two sequencing samples. The m6A modification site data of RNA sequences refers to N6-methyladenylation modification, including splicing, stability, transport and translation of mRNA. m6A modification is usually achieved by a "methyltransferase complex", including "write" enzymes (such as METTL3, METTL14 and WTAP) and "erase" enzymes (such as FTO and ALKBH5).

[0208] Based on this, in the prediction method of RNA sequence modification model, the construction process of multi-level classification database is as follows:

[0209] First, the sites modified with m6A in all four sequencing samples were extracted as positive samples. At the same time, 25 nt were diffused to both sides from the central site of each positive sample to form a positive sample with a length of 51 nt, 50 nt was diffused to both sides to form a positive sample with a length of 101 nt, and 100 nt was diffused to both sides to form a positive sample with a length of 201 nt, so as to obtain multiple positive sample sequences of different lengths. Then, samples or motif fragments of non-m6A modification sites in the four sequencing samples were extracted as negative samples, and the extracted negative samples strictly did not overlap with the aforementioned positive samples on the genome. Considering the enrichment of m6A modification at the DRACH sequence motif, three different base restrictions were added to the negative samples to form three negative sample sequences with different classification difficulties, namely, the first one was not to add any restrictions to the negative samples, the second one was to restrict the center position of the negative samples to adenine (A), and the third one was to restrict the center position of the negative samples to meet the DRACH (D represents A, G or U, R represents A or G, H represents A, C or U) base arrangement. The genome structures of the positive and negative samples are shown in the figure. Figure 7 Finally, the constructed positive sample sequences and negative sample sequences were deduplicated using CDHIT software with a sequence similarity of 0.9 as the threshold, and finally, a number of sample sequences with 48,000 samples in the training set and 12,000 samples in the test set were obtained according to a one-to-one sampling ratio of positive and negative samples, thereby constructing a multi-level classification database.

[0210] Among them, when constructing a multi-level classification database, this example constructs an m6A classification dataset (i.e., positive sample sequence) containing three length samples from scratch based on the latest single nucleotide precision m6A absolute quantitative sequencing data, and by adding different restrictions to negative samples, it is possible to construct multiple datasets (i.e., negative sample sequences) with different classification difficulties.

[0211] In this example, the Bert module, a deep learning framework based on self-supervised tasks, is combined with a multi-head attention mechanism to build an end-to-end m6A modification prediction algorithm from scratch to form an m6ABert prediction model (i.e., modification prediction model). In specific implementation, when encoding the m6ABert prediction model, the Bert module has 4 BERT attention layers, each of which contains 4 first attention heads and a hidden dimension of 128. When compared with other algorithms, the multi-head attention module uses 8 second attention heads and a hidden dimension of 128. In the subsequent weight visualization part, the maximum pooling is first used to compress the sequence length dimension to 1, and then two fully connected layers are used for classification. The learning rate of the training process is 1e-3, the weight decay is 1e-3, the batch size is 256, the dropout parameter is 0.1, and the model is trained on 4 Nvidia 3090 graphics cards.

[0212] like Figure 8 As shown in the figure, it is a structural block diagram of the m6ABert prediction model. The m6ABert prediction model first uses the Bert module to encode the input sample sequence and obtain the embedded representation of the sample sequence. This process requires training from scratch for each sample sequence in the multi-level classification database. Subsequently, the embedded representation of the sample sequence is transmitted to the multi-head attention module, and the second attention weight is determined by the multi-head attention module. Finally, it passes through the pooling layer and then the fully connected layer for classification, forming an end-to-end m6A prediction process.

[0213] In addition, after the Bert module, the multi-head attention module can also visualize the model weights based on the attention mechanism, and then search for important motif fragments through a sliding window, extract key motifs to form a motif set, and further use the MEME algorithm to cluster key motifs (i.e. key motifs). For the key motifs found, the TOMTOM algorithm is used to search and compare them with existing motifs in a publicly released motif database, and finally the matched motifs are visualized.

[0214] Specifically, when visualizing the second attention weight, the second attention weight is extracted for each second attention head. In this example, the extraction formula of the second attention weight is as follows:

[0215]

[0216] Among them, Attention(Q,K,V) is the calculation process of the attention mechanism, softmax() is the operation of the normalized exponential function, Q, K, and V are the data corresponding to the query, key, and value matrices respectively, and d k is the length of the extracted vector.

[0217] For the extracted second attention weight, a sliding window of 18 nt in length was used to traverse all true positive samples, and the high-weight motif fragments of each sample were extracted with a threshold value greater than 2, and then the top 200 motif fragments were extracted by sorting. The MEME algorithm was used to search for motifs and cluster the 200 typical motif fragments extracted. When clustering, the number of motif clusters was set to 3-5. For significant motifs, the TOMTOM algorithm was used to compare with existing motifs in the human CIS-BP database, and the RNA-binding protein binding motif with the smallest p-value was further extracted for visualization.

[0218] Please refer to Fig. 9 , Fig. 9 The figure shows the relationship between the position of m6A modification on the sample sequence of the m6A modification site prediction data obtained based on the m6ABert prediction model and the attention weight at that position. The larger the value of the attention weight corresponding to the position with m6A modification, the more accurate the m6ABert prediction model of this embodiment can be. Fig.10 , Fig.10 The most common motifs in m6A modification are shown, from which it can be seen that the m6ABert prediction model of this example successfully identified the key motifs and visualized them, achieving intuitive evaluation of model performance and interpretability of the output model.

[0219] In an application example, in order to further illustrate that the method of this embodiment has a more accurate prediction effect compared with other models (including UniRNA, MASS, One-hot, DeepM6ASeq-RNN, DeepM6ASeq-CNN), the prediction effect of the m6ABert prediction model is compared with the prediction effect of the aforementioned other models based on the constructed multi-level classification database. At the same time, in order to ensure the fairness of the comparison, all models are retrained on each sample sequence respectively. In addition, One-hot encoding access to a simple convolutional neural network is added as a benchmark for horizontal comparison. The prediction effect is as follows Fig.11As shown in the figure, on three data sets with different classification difficulties, the m6ABert prediction model achieved better prediction accuracy, and as the classification difficulty increased, the advantage became more obvious. On the two data sets with no negative sample restrictions and A site restrictions, even a simple One-hot encoding plus classifier can achieve a high accuracy rate. This shows that the difference between the m6A modification site sequence and the random sequence is relatively easy to distinguish.

[0220] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0221] Based on the same inventive concept, the embodiment of the present application also provides a training device for an RNA sequence modification prediction model for implementing the training method of the RNA sequence modification prediction model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the training device for one or more RNA sequence modification prediction models provided below can be referred to the limitations of the training method for the RNA sequence modification prediction model above, and will not be repeated here.

[0222] In one embodiment, Fig.12 As shown, a training device for an RNA sequence modification prediction model is provided, comprising: a data acquisition module 1201, a sample extraction module 1202, a database construction module 1203, a site prediction module 1204 and a model update module 1205, wherein:

[0223] The data acquisition module 1204 is used to obtain modification site sequencing data of the RNA sequence; the modification site sequencing data is obtained based on sequencing of at least two types of cell lines.

[0224] The sample extraction module 1202 is used to obtain multiple positive sample sequences of different lengths based on the sites with target chemical modification in the modification site sequencing data, and to obtain multiple negative sample sequences with different bases added at the center position based on the sites without target chemical modification in the modification site sequencing data; the positive sample sequences and the negative sample sequences do not overlap in the genomic position.

[0225] The database construction module 1203 is used to obtain a multi-level classification database based on the positive sample sequence and the negative sample sequence.

[0226] The site prediction module 1204 is used to input each sample sequence in the multi-level classification database into the modification prediction model to be trained to obtain the prediction result of the modification site of each sample sequence.

[0227] The model updating module 1205 is used to update the model parameters of the modification prediction model to be trained based on the prediction results and the actual situation of each sample sequence until a trained modification prediction model is obtained.

[0228] In an optional embodiment, the training device for the above RNA sequence modification prediction model further includes:

[0229] The weight extraction module is used to extract the second attention weight of each second attention head of the multi-head attention module.

[0230] The motif extraction module is used to determine the true positive samples in the modification site prediction results of the current sample sequence, traverse the true positive samples through a preset sliding window, determine the weight and weight of the true positive samples covered by each preset sliding window according to the second attention weight, and extract the true positive samples whose weight and weight meet the preset conditions to form a motif set.

[0231] The clustering module is used to cluster the motif set and determine the key motifs.

[0232] The matching module is used to match key motifs with existing motifs in public databases, extract key motifs that have been successfully matched, and visualize them.

[0233] Based on the same inventive concept, the embodiment of the present application also provides a modification prediction device for RNA sequences for implementing the modification prediction method of RNA sequences involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiment of the modification prediction device for one or more RNA sequences provided below can be referred to the limitations of the modification prediction method for RNA sequences above, and will not be repeated here.

[0234] In one embodiment, Fig.13 As shown, a modification prediction device for RNA sequence is provided, comprising: a sequence acquisition module 1301, a sequence input module 1302, an embedding representation extraction module 1303, an attention representation extraction module 1304 and a classification prediction module 1305, wherein:

[0235] The sequence acquisition module 1301 is used to acquire the RNA sequence to be predicted.

[0236] The sequence input module 1302 is used to input the RNA sequence to be predicted into the trained modification prediction model.

[0237] The embedded representation extraction module 1303 is used to encode the RNA sequence based on the Bert module of the modification prediction model to obtain the embedded representation of the RNA sequence.

[0238] The attention representation extraction module 1304 is used to extract the attention representation of the embedded representation of the RNA sequence based on the multi-head attention module of the modified prediction model to obtain the attention representation of the RNA sequence.

[0239] The classification prediction module 1305 is used to perform classification prediction based on the attention representation of the RNA sequence to obtain the modification site prediction result of the RNA sequence.

[0240] Each module in the training device of the RNA sequence modification prediction model and the RNA sequence modification prediction device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0241] In an exemplary embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented by WIFI, a mobile cellular network, a near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a training method for a RNA sequence modification prediction model and / or a modification prediction method for an RNA sequence are implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0242] It is understandable that in some other embodiments, the computer device may also be a server, including a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Wherein, the processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for RNA sequence and its modification site sequencing data, positive sample sequence, negative sample sequence, model parameters and attention weights and other data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a training method for an RNA sequence modification prediction model and / or a modification prediction method for an RNA sequence are implemented.

[0243] Those skilled in the art will appreciate that the structure of the computer device described above is merely a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than described, or combine certain components, or have a different arrangement of components.

[0244] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the training method of the RNA sequence modification prediction model and / or the RNA sequence modification prediction method of the above-mentioned embodiment is implemented.

[0245] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the training method of the RNA sequence modification prediction model and / or the RNA sequence modification prediction method of the above-mentioned embodiment are implemented.

[0246] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the RNA sequence modification prediction model training method and / or RNA sequence modification prediction method of the above-mentioned embodiment.

[0247] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0248] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0249] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0250] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for training an RNA sequence modification prediction model, characterized in that: The method comprises: Obtaining modification site sequencing data of RNA sequences; wherein the modification site sequencing data is obtained based on sequencing of at least two types of cell lines respectively; Based on the sites with target chemical modification in the modification site sequencing data, a plurality of positive sample sequences with different lengths are obtained, and based on the sites without the target chemical modification in the modification site sequencing data, a plurality of negative sample sequences with different bases added at the center position are obtained; the positive sample sequences and the negative sample sequences do not overlap in genomic position; Obtain a multi-level classification database based on the positive sample sequence and the negative sample sequence; Inputting each sample sequence in the multi-level classification database into the modification prediction model to be trained to obtain the prediction result of the modification site of each sample sequence; Based on the prediction results and the actual situation of each sample sequence, the model parameters of the modification prediction model to be trained are updated until a trained modification prediction model is obtained.

2. The method according to claim 1, characterized in that The method of obtaining a plurality of positive sample sequences of different lengths based on the sites with target chemical modification in the modification site sequencing data includes: Extracting sites with target chemical modifications from the modification site sequencing data obtained by sequencing the at least two types of cell lines respectively; Based on the sites with target chemical modifications, extensions of different lengths are performed to obtain a plurality of positive sample sequences; The extension of different lengths based on the site with target chemical modification includes: A plurality of extension lengths are determined, and the site with the target chemical modification is taken as the central site, and the extensions are respectively extended to the determined extension lengths on both sides of the central site.

3. The method according to claim 1, characterized in that The method of obtaining a plurality of negative sample sequences with different bases added at the center position based on the sites without the target chemical modification in the modification site sequencing data includes: Extracting sites without target chemical modification from the modification site sequencing data obtained by sequencing the at least two types of cell lines respectively, and adding different bases at the center position for restriction based on the sites without target chemical modification to obtain a plurality of negative sample sequences; The method of adding different bases at the central position for restriction based on the site without target chemical modification includes: determining a similarity between the site without the target chemical modification and the site with the target chemical modification; Based on the similarity, the site without the target chemical modification is divided into a plurality of negative sample sets, and different base restrictions are respectively determined for the plurality of negative sample sets based on the similarity; the base restrictions at least include not adding restrictions to the central position of the site without the target chemical modification, restricting the base at the central position of the site without the target chemical modification to adenine, and / or restricting the base at the central position of the site without the target chemical modification to a base arrangement that satisfies DRACH; Add corresponding base restrictions to each negative sample set respectively.

4. The method according to any one of claims 1 to 3, characterized in that: The step of inputting each sample sequence in the multi-level classification database into the modification prediction model to be trained to obtain the prediction result of the modification site of each sample sequence comprises: Determine any sample sequence in the multi-level classification database as a current sample sequence; Obtaining an initial modification prediction model; the initial modification prediction model is constructed based on a Bert module and a multi-head attention module; Encoding the current sample sequence based on the Bert module to obtain an embedded representation of the current sample sequence; Extracting attention representation from the embedded representation of the current sample sequence based on the multi-head attention module to obtain the attention representation of the current sample sequence; Classification prediction is performed based on the attention representation of the current sample sequence to obtain a modification site prediction result of the current sample sequence.

5. The method according to claim 4, characterized in that The encoding of the current sample sequence based on the Bert module to obtain an embedded representation of the current sample sequence includes: Input the current sample sequence into the Bert module; the Bert module has multiple BERT attention layers, and each BERT attention layer has multiple first attention heads; Vectorize the current sample sequence through the embedding layer of the Bert module to obtain a vector sequence of the current sample sequence; The vector sequence of the current sample sequence is divided into multiple first attention heads through the multi-layer BERT attention layer and the first attention weight of each first attention head is determined, and the embedding information of the current sample sequence is determined based on each first attention weight; the input of the first layer of the BERT attention layer is the vector sequence of the current sample sequence output by the embedding layer, and the input of each subsequent layer of the BERT attention layer is the embedding information of the current sample sequence output by the previous layer of the BERT attention layer; The embedding information of the current sample sequence output by the last BERT attention layer is processed to obtain the embedding representation of the current sample sequence.

6. The method according to claim 4, characterized in that After performing classification prediction based on the attention representation of the current sample sequence to obtain the modification site prediction result of the current sample sequence, the method further includes: Extracting the second attention weight of each second attention head of the multi-head attention module; Determine true positive samples in the modification site prediction result of the current sample sequence, traverse the true positive samples through a preset sliding window, determine the weight sum of the true positive samples covered by each preset sliding window according to the second attention weight, and extract the true positive samples whose weight sum meets the preset condition to form a motif set; Clustering the motif set to determine key motifs; The key motifs are matched with existing motifs in public databases, and the successfully matched key motifs are extracted and visualized.

7. The method according to claim 6, characterized in that The step of clustering the motif set to determine key motifs comprises: The MEME algorithm is used to search for motifs in the motif fragments of the motif set, and the searched motifs are clustered according to a set number of clusters to obtain multiple motif groups; The motifs in each motif group that meet the preset characteristic conditions are identified as key motifs.

8. The method according to claim 6, characterized in that The step of matching the key motif with existing motifs in a public database, extracting the successfully matched key motifs and visualizing them comprises: The TOMTOM algorithm is used to determine the similarity scores between the key motif and each reference motif, extract the reference motif corresponding to the minimum similarity score, and visualize the extracted reference motif.

9. The method according to claim 1, characterized in that: The updating of the model parameters of the modification prediction model to be trained based on the prediction result and the actual situation of each sample sequence until a trained modification prediction model is obtained comprises: Based on the prediction results and the actual position information of the sites with the target chemical modification in the positive sample sequence and the actual position information of the sites without the target chemical modification in the negative sample sequence, the model parameters of the modification prediction model to be trained are updated until a trained modification prediction model is obtained.

10. A method for predicting modification of RNA sequences, characterized in that: The method comprises: Obtaining the RNA sequence to be predicted; Inputting the RNA sequence to be predicted into a trained modification prediction model; the modification prediction model is trained based on the method according to any one of claims 1 to 9; Encoding the RNA sequence based on the Bert module of the modification prediction model to obtain an embedded representation of the RNA sequence; Extracting attention representation from the embedded representation of the RNA sequence using a multi-head attention module based on the modification prediction model to obtain an attention representation of the RNA sequence; Classification prediction is performed based on the attention representation of the RNA sequence to obtain a prediction result of the site of the target chemical modification in the RNA sequence.

Images