A Multi-Type RNA Modification Prediction Method Based on Vector Quantization Variational Autoencoder

Abstract

This invention discloses a method for predicting multiple RNA modifications based on a vector quantization variational autoencoder, belonging to the field of bioinformatics. The method includes: acquiring and preprocessing the RNA sequence to be tested; obtaining a latent feature representation using convolutional coding based on the preprocessed result; determining the embedding vector with the smallest Euclidean distance to the latent feature representation from the trained discrete codebook, thereby obtaining a discrete vector quantization vector; obtaining a reconstructed RNA sequence using deconvolutional decoding based on the discrete vector quantization vector; and extracting feature vectors from the discrete vector quantization vector using a Transformer coding operation incorporating a multi-head attention mechanism, thereby obtaining the probabilities of different RNA modification types and determining the RNA modification type. This significantly improves the accuracy and generalization of prediction and enhances the biological interpretability of the results.

Description

Technical Field

[0001] This invention relates to the field of bioinformatics, and in particular to a method for predicting multiple types of RNA modifications based on a vector quantization variational autoencoder. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Posttranscriptional RNA modifications play a crucial role in regulating RNA structure and function. To date, over 170 different RNA chemical modifications have been identified, widely distributed across messenger RNA (mRNA), transfer RNA (tRNA), ribosomal RNA (rRNA), and various non-coding RNAs. Among these, the most extensively studied are 2'-O-methyladenosine (Am), 2'-O-methylcytidine (Cm), 2'-O-methylguanosine (Gm), 2'-O-methyluridine (Um), N1-methyladenosine (m1A), 5-methylcytidine (m5C), 5-methyluridine (m5U), N6-methyladenosine (m6A), N6,2'-O-dimethyladenosine (m6Am), and pseudouridine (Ψ). They can regulate gene expression, affect RNA stability, and influence translation efficiency and accuracy. Dysregulation of modification pathways has been shown to be associated with various human cancers, making them highly promising therapeutic targets. Furthermore, RNA modification participates in various biological processes in immune cells, influencing the development and progression of immune-related diseases by regulating immune responses. Therefore, accurately identifying RNA modification sites is crucial for elucidating their functions in gene regulation, cellular homeostasis, and disease development.

[0004] Currently, various experimental techniques have been developed for detecting RNA modification sites. Among them, liquid chromatography-mass spectrometry (LC-MS) can accurately quantify the overall modification level by enzymatically digesting RNA into individual nucleosides. Meanwhile, the development of high-throughput sequencing technology has also driven the development of methods such as MeRIP-seq, CLIP-seq, and nanopore sequencing, enabling the mapping of specific modification sites at single-nucleoside resolution. MeRIP-seq and CLIP-seq are two commonly used techniques for studying RNA modification and protein interactions. MeRIP-seq enriches m6A-modified RNA fragments with m6A-specific antibodies, followed by high-throughput sequencing for localization and quantification. CLIP-seq uses ultraviolet cross-linking to bind proteins to DNA / RNA, followed by enrichment of the RNA fragment corresponding to the target protein with specific antibodies, and then sequencing analysis. Despite their significant value, these methods still suffer from time-consuming, labor-intensive, and costly processes. Furthermore, these methods face significant challenges in terms of technological integration and scalability, limiting their comprehensive application across the transcriptome.

[0005] In recent years, machine learning and deep learning have become powerful tools for predicting RNA modification sites, with widely used algorithms including support vector machines, random forests, convolutional neural networks, recurrent neural networks, and Transformers. However, most existing models are limited to a single modification type, have limited generalization ability, lack interpretability, and struggle to reveal the gene regulatory mechanisms of modifications. Traditional feature representations lack clear biological meaning, limiting the capture of key regulatory patterns. Furthermore, there is a lack of publicly available integrated tools that can provide both accurate predictions and biological explanations, hindering the systematic comparison and functional exploration of regulatory patterns among different modification types. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a multi-type RNA modification prediction method based on vector quantization variational autoencoders, which significantly improves the accuracy and generalization of predictions and enhances the biological interpretability of the results. This solves the problems of limited feature representation capabilities, limitation to a single modification type, limited generalization capabilities, and poor model interpretability in existing technologies.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a method for predicting multiple types of RNA modifications based on a vector quantization variational autoencoder, comprising:

[0009] Obtain the RNA sequence to be tested and perform preprocessing;

[0010] Based on the obtained preprocessing results, convolutional coding is used to obtain the latent representation of the features;

[0011] For the latent feature representation, in the discrete codebook after training, the embedding vector with the smallest Euclidean distance to the latent feature representation is determined, thereby obtaining the discrete vector quantization vector;

[0012] Based on the discrete vector quantization vector, a deconvolution decoding operation is performed to obtain the reconstructed RNA sequence, thereby optimizing the encoding and decoding operation by minimizing the reconstruction loss;

[0013] For discrete vector quantization vectors, a Transformer encoding operation with multi-head attention mechanism is used to extract feature vectors, thereby obtaining the probabilities of different RNA modification types and determining the RNA modification type.

[0014] As an alternative implementation, preprocessing includes: taking the RNA sequence to be tested, extending symmetrically upstream and downstream around the modification site, and extracting base sequence fragments of different window lengths; dividing the base sequence fragments into continuous substrings of fixed length using a sliding window method, and performing One-Hot encoding on the obtained k-mer index sequence.

[0015] As an alternative implementation, the convolutional encoding operation is performed using three layers of one-dimensional convolution; the deconvolutional decoding operation is performed using three layers of deconvolution.

[0016] As an alternative implementation method, the training process of the discrete codebook includes:

[0017] Codebook loss : ;

[0018] Promised loss : ;

[0019] in, , This refers to the size of the codebook; It stops the gradient operation; It is the weight of the promised loss; and These are the k-th and j-th codebook vectors, respectively; The embedded representation generated for the encoder; argmin represents the function that takes the minimum value.

[0020] As an alternative implementation, the Transformer encoding operation includes: using 6 encoding layers, with each encoding layer including 8 attention heads for feature vector extraction.

[0021] As an alternative implementation, the probabilistic prediction process includes: the extracted feature vector is used as the input of a fully connected network, which gradually reduces the dimension of the feature vector through multiple fully connected layers, Leaky ReLU activation function and batch normalization operation, and finally outputs a two-dimensional vector, which is then mapped to the classification result through a Softmax layer.

[0022] Secondly, the present invention provides a multi-type RNA modification prediction system based on a vector quantization variational autoencoder, comprising:

[0023] The preprocessing module is configured to acquire and preprocess the RNA sequence to be tested.

[0024] The encoding module is configured to use convolutional encoding operations to obtain the latent representation of the features based on the obtained preprocessing results;

[0025] The vector quantization module is configured to determine the embedding vector with the smallest Euclidean distance to the feature latent representation in the trained discrete codebook, thereby obtaining the discrete vector quantization vector.

[0026] The decoding module is configured to quantize the vector from the discrete vector, perform a deconvolution decoding operation to obtain the reconstructed RNA sequence, and optimize the encoding and decoding operation by minimizing the reconstruction loss.

[0027] The attention processing module is configured to quantize discrete vectors and extract feature vectors using Transformer encoding operations that include a multi-head attention mechanism, thereby obtaining the probabilities of different RNA modification types and determining the RNA modification type.

[0028] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0029] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0030] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] This invention proposes a multi-type RNA modification prediction method based on vector quantization variational autoencoder. By introducing a vector quantization module into the VQ-RNA prediction framework, a discrete latent representation space with clear biological meaning is constructed, which significantly improves the unified prediction performance for ten RNA modifications while enhancing model interpretability. This model can not only accurately identify modification sites, but also reveal the sequence specificity and conserved patterns of RNA modifications through ISM analysis, attention visualization, and feature spectrum construction. At the same time, the model has demonstrated excellent accuracy, robustness and generalization ability on multiple independent datasets, solving the problems of limited feature representation ability, limitation to a single modification type, limited generalization ability and poor model interpretability in existing technologies.

[0033] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0035] Figure 1 The process for constructing a dataset of 10 RNA modification types provided in Embodiment 1 of the present invention;

[0036] Figure 2 This is the VQ-RNA model framework provided in Embodiment 1 of the present invention;

[0037] Figure 3 This is a schematic diagram of the data preprocessing module provided in Embodiment 1 of the present invention;

[0038] Figure 4 This is a schematic diagram of the CNN encoding module provided in Embodiment 1 of the present invention;

[0039] Figure 5 This is a schematic diagram of the CNN decoding module provided in Embodiment 1 of the present invention;

[0040] Figure 6 A schematic diagram of the vector quantization module provided in Embodiment 1 of the present invention;

[0041] Figure 7 This is a schematic diagram of the Transformer feature alignment module provided in Embodiment 1 of the present invention;

[0042] Figure 8A schematic diagram of the classification module provided in Embodiment 1 of the present invention;

[0043] Figure 9 This is a schematic diagram of the interpretability analysis module provided in Embodiment 1 of the present invention;

[0044] Figure 10 Box plot comparison of the prediction task performance of various models on the existing dataset;

[0045] Figure 11 A comparison chart of ACC metrics for various models on 10 RNA modification datasets for different prediction tasks;

[0046] Figure 12 A comparison chart of AUC metrics for various models on 10 RNA modification datasets for different prediction tasks;

[0047] Figure 13 A comparison chart of MCC metrics for various models predicting tasks on 10 RNA modification datasets;

[0048] Figure 14 Visualization of uniformity analysis under Am, Cm, and Gm modification types;

[0049] Figure 15 A bar chart comparing the performance of each component in the ablation model;

[0050] Figure 16 A bar chart comparing the performance of each loss function in the ablation loss function;

[0051] Figure 17 A visualization of the UMAP features of each component of the ablation model;

[0052] Figure 18 A visualization of the UMAP features of each loss term in the ablation loss function;

[0053] Figure 19 Box plot comparison of performance for different input sequence lengths;

[0054] Figure 20 Box plot comparison of performance for different codebook sizes;

[0055] Figure 21 A visualization of the comprehensive attention matrix for different sequence lengths;

[0056] Figure 22 Visualization of the comprehensive attention matrix for different codebook sizes;

[0057] Figure 23 A visualization of sequence motifs identified in the VQ-RNA model based on ISM experiments;

[0058] Figure 24 Predict importance score plots for each point in the VQ-RNA model;

[0059] Figure 25 The process of constructing RNA modification feature maps and visualization of specific feature maps of 10 RNA modifications are presented. Detailed Implementation

[0060] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0061] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0062] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0063] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0064] Example 1

[0065] This embodiment provides a method for predicting multiple types of RNA modifications based on a vector quantization variational autoencoder, which specifically includes the following steps:

[0066] Obtain the RNA sequence to be tested and perform preprocessing;

[0067] Based on the obtained preprocessing results, convolutional coding is used to obtain the latent representation of the features;

[0068] For the latent feature representation, in the discrete codebook after training, the embedding vector with the smallest Euclidean distance to the latent feature representation is determined, thereby obtaining the discrete vector quantization vector;

[0069] Based on the discrete vector quantization vector, a deconvolution decoding operation is used to obtain the reconstructed RNA sequence;

[0070] For the reconstructed RNA sequence, a Transformer encoding operation with a multi-head attention mechanism is used to extract feature vectors, thereby obtaining the probability of different RNA modification types and determining the RNA modification type.

[0071] like Figure 1 The diagram illustrates the complete data processing workflow for the prediction of 10 RNA modification sites in this embodiment. First, raw data is collected and integrated; epitranscriptome data from public databases (GEO, RMBase, RADAR), experimental technology data (Nm-seq, miCLIP, RBS-Saq), and the MultiRM benchmark dataset are used; among them, the MultiRM benchmark dataset generated 20 epitranscriptome datasets using 15 different genome resolution techniques.

[0072] Then, data filtering and quality control; further processing of the collected raw data to ensure the quality of the input data, including data cleaning, quality assessment and standardization.

[0073] Thus, positive samples with known modification sites were obtained, including 10 RNA modification types: Am, Cm, Gm, Um, m 1 A、m 5 C, m 5 U、m 6 A、m 6 Am and Ψ. Negative samples are randomly selected from unmodified bases in the same transcript containing positive sites, with the number of positive and negative samples being as close as possible to each other.

[0074] Subsequently, the dataset was divided; the dataset for each RNA modification type was divided into training, testing, and validation sets in an 8:1:1 ratio.

[0075] Next, sequence preprocessing and feature engineering were performed. A k-mer-based encoding method was used to extract eight inputs for each RNA modification sequence. Specifically, fragments with window sizes of 51, 201, 301, 401, 501, 701, 901, and 1001 were centered on the modification site and preprocessed with k-mer encoding and One-Hot encoding to generate the model inputs.

[0076] In this embodiment, after obtaining RNA sequence data, fragments of different lengths are extracted centered on the modification site and k-mer encoding preprocessing is performed. The preprocessed sequence is then input into the trained VQ-RNA prediction model to obtain the modification probability of the RNA sequence in 10 RNA modification types.

[0077] like Figure 2As shown, the VQ-RNA prediction model includes seven core components: a data preprocessing module, a CNN encoder, a Vector Quantization (VQ) module, a CNN decoder, a Transformer feature alignment module, a classifier, and an interpretability analysis module.

[0078] The following is a detailed analysis of each module.

[0079] like Figure 3 As shown, the data preprocessing module is used for dataset partitioning, RNA sequence truncation, k-mer encoding, and 0-1 encoding.

[0080] To obtain high-quality, interpretable, and directly usable input for model training, this embodiment systematically cleans, encodes, and partitions raw epitranscriptome data collected from public databases, experimental technical data, and the MultiRM benchmark dataset. The final result is a dataset covering ten common RNA modifications (Am, Cm, Gm, Um, m...). 1 A、m 5 C, m 5 U、m 6 A、m 6 The standard inputs are Am and Ψ. The entire process includes four sequentially executed sub-steps: dataset partitioning, sequence truncation, k-mer encoding, and One-Hot encoding. This ensures that both local and global information are preserved and meets the discrete embedding requirements of the subsequent VQ-RNA model.

[0081] Specifically:

[0082] (1) Data set division: All samples of each RNA modification type were randomly divided into training set, validation set and test set without replacement in a ratio of 8:1:1 to ensure that positive and negative sites within the same transcript do not appear in different sets.

[0083] (2) RNA sequence extraction: Using experimentally validated modification sites as the center, symmetrical extensions were made upstream and downstream to extract fragments of eight different window lengths (51, 201, 301, 401, 501, 701, 901, and 1001 nt). This multi-scale strategy can simultaneously capture short sequence context (such as motif) and long-range structural information, providing the model with a rich spatial receptive field.

[0084] (3) k-mer encoding: The original base sequence obtained above is divided into continuous substrings (k-mer) of length k (k=1 in this embodiment) by sliding window method. Each k-mer is regarded as a discrete symbol, thereby mapping the variable-length sequence to a fixed-length integer index sequence, which reduces the dimensionality and preserves the local combination features.

[0085] (4) One-Hot Encoding: Perform One-Hot encoding on the obtained k-mer index sequence to generate a sparse high-dimensional vector (dimension 4). k This step allows the model to directly accept numerical inputs while maintaining the mutual exclusivity and interpretability of base combinations in the discrete space; it also provides a feature distribution that is easy to cluster for subsequent vector quantization codebooks.

[0086] like Figure 4 As shown, the CNN encoding module is used to extract local features of the input sequence and generate continuous latent representations to map the input sequence into continuous latent representations.

[0087] The CNN encoding module consists of three layers of one-dimensional convolutions (CNN1, CNN2, and CNN3). The given input sequence is a matrix. ,in For sequence length, For encoding dimensions. The... The encoding operation of a one-dimensional convolution layer is formalized as follows:

[0088] (1);

[0089] in, Here, ReLU is the activation function, and Conv1D represents a one-dimensional convolution operation. and The first Kernel weights and biases for one-dimensional convolutions; For the first The output of a one-dimensional convolution layer; For the first The output of a one-dimensional convolution layer.

[0090] After three convolutional layers, a continuous latent representation is obtained for subsequent vector quantization modules. :

[0091] (2).

[0092] in, This is the output of the third layer of one-dimensional convolution; The original input sequence; The length of the sequence after convolution; This represents the number of output channels.

[0093] like Figure 5 As shown, the CNN decoding module is used to quantize the discrete vectors based on the quantized discrete vectors, and then deconvolve the low-dimensional discrete features back to the original data space to obtain the reconstructed RNA sequence.

[0094] The CNN decoding module is structurally symmetrical to the CNN encoding module and uses three layers of deconvolution (DeCNN1, DeCNN2, DeCNN3) for sequence reconstruction.

[0095] Given a quantized discrete vector quantization vector The decoding operation of the j-th deconvolutional layer is as follows:

[0096] (3);

[0097] in, This represents the deconvolution operation; For the first The output of the deconvolution layer; For the first The output of the deconvolution layer; and For respectively the first Convolution kernel weights and biases of deconvolution layers.

[0098] The final output reconstructed RNA sequence for:

[0099] (4);

[0100] in, This is the output of the third deconvolutional layer.

[0101] Finally, minimize the reconstruction loss. The goal is to minimize the error between the reconstructed RNA sequence and the original RNA sequence, thereby optimizing the parameters of the CNN decoding and encoding modules and ensuring consistency and biological relevance between the potential embedding and the input RNA sequence.

[0102] Reconstruction loss uses the original value With reconstruction value Calculation of mean squared error (MSE loss):

[0103] (5);

[0104] Where N is the total number of elements in the sequence, i.e., the sequence length; and i is the characteristic value corresponding to the RNA base at the i-th position.

[0105] like Figure 6As shown, the vector quantization module is the core module of the VQ-RNA prediction model. It contains a discrete embedding space—the discrete codebook, which is a learnable tensor. Through the mapping of the discrete codebook, the vector quantization module quantizes the continuous feature latent representation into discrete vector quantization vectors, thereby constructing a modification-specific discrete latent space with rich information. This achieves the transformation from continuous features to discrete embeddings, enhancing the interpretability and discriminability of the features.

[0106] Specifically:

[0107] The encoder generates a latent representation of features. Then, first, based on the minimum Euclidean distance, find the corresponding value in the discrete codebook. Index of the closest embedding vector And represent the index as ,therefore It is a discrete index value.

[0108] Next, according to Calculate the posterior distribution This distribution is a 0-1 vector;

[0109] (6);

[0110] in, Let be the j-th embedding vector in the discrete codebook.

[0111] It should be noted that the posterior distribution It is a discrete, non-parametric one-hot distribution, indicating which codebook vector the input sample x is mapped to in the discrete latent space. The purpose of this is to explain the discretization selection mechanism in the encoding stage.

[0112] Ultimately, the decoder with As input, Corresponding to the selected embedding vector in the discrete codebook :

[0113] (7).

[0114] The final potential embedding of RNA modifications consists of embedding vectors in a discrete codebook. RNA modifications of the same type are mapped to the same vector in the discrete codebook, while RNA modifications of different types are distributed into different embedding vectors.

[0115] Next, the code itself is lost. Training is performed on a discrete codebook, allowing the vectors in the discrete codebook to... The tendency towards encoder-generated embeddings ;

[0116] (8);

[0117] in, , This refers to the size of the codebook; It stops the gradient operation, in the code-based loss. The use of the encoder prevented Gradient backpropagation only optimizes the code vector. However, the encoder will not change.

[0118] Finally, construct the commitment loss. This can be viewed as the opposite of codebook loss, allowing the encoder to generate embeddings... Embedding vectors in the codebook This prevents excessive fluctuations in the embedding values ​​output by the encoder;

[0119] (9);

[0120] in, This is the weight of the commitment loss, defaulting to i0.25. In the commitment loss, The use of embedding vectors in discrete codebooks prevents Gradient backpropagation, optimizing only the encoder. The parameters are updated without updating the discrete codebook.

[0121] It is worth noting that because the argmin operation is not differentiable, gradient propagation between the decoder and encoder is interrupted. Therefore, a detach operation is needed to directly copy the gradients at the decoder input to the encoder output to achieve efficient gradient propagation.

[0122] like Figure 7 As shown, the Transformer feature alignment module is used to capture long-range dependencies in sequences and perform global context modeling on discrete embeddings. The Transformer feature alignment module consists of 6 Transformer layers, each of which includes a multi-head self-attention mechanism and a feedforward network, and is connected to a residual connection and layer normalization. The multi-head attention mechanism executes 8 independent attention heads in parallel, which can distribute the focus of attention, better balance the semantic differences between features, effectively capture global information, and improve the global consistency and robustness of feature representation.

[0123] The core calculations of the attention mechanism are as follows:

[0124] (10);

[0125] in, These represent the Query, Key, and Value obtained from the input mapping, respectively. The dimension of the Key.

[0126] It should be noted that the discrete vector quantization vector obtained after vector quantization is used to obtain the reconstructed RNA sequence through deconvolution decoding, thereby optimizing the model performance through reconstruction loss; and the discrete vector quantization vector is also input into the Transformer feature alignment module, and then the modified prediction result is obtained through the classification module.

[0127] like Figure 8 As shown, the classification module is used to discriminate latent features and output the probabilities of multiple types of RNA modifications. The classification module extracts feature vectors from the Transformer feature alignment module as input to the fully connected network. The fully connected network gradually reduces the dimensionality of the feature vectors through multiple fully connected layers, the Leaky ReLU activation function, and batch normalization operations, finally outputting a two-dimensional vector, which is then mapped to the classification result through a Softmax layer.

[0128] (11);

[0129] in, is the unnormalized prediction value output by the classification network for the i-th class, i.e., the output result of the fully connected layer for the i-th class; n is the total number of classes in the classification task; This is the unnormalized predicted value corresponding to the j-th category, used for the normalization calculation of the Softmax function.

[0130] like Figure 9 As shown, to enhance the biological interpretability of the model, this embodiment includes an interpretability analysis module in addition to the prediction results. This module provides homogeneity analysis, UMAP clustering analysis (UMAP, Uniform Manifold Approximation and Projection, is a technique for data dimensionality reduction and visualization), self-attention mechanism analysis, motif identification based on virtual saturation mutations (ISM), and TF-IDF-based feature mapping analysis to reveal specific patterns and underlying biological mechanisms of RNA modifications.

[0131] The following is an introduction.

[0132] Motif identification based on virtual saturation mutation.

[0133] ISM is a method that simulates mutations to assess the impact of each position in a sequence on model predictions. In this process, for each input sequence... This generates all sequences with an edit distance of 1, i.e., each base in the sequence is mutated one by one. For each generated replacement sequence... Use the model to make predictions and obtain the replaced predicted values. :

[0134] (12);

[0135] in, Indices representing bases (A, C, G, U). Indicates the site where the replacement occurred. Indicates replacement [ The bases in ] For other bases, It is the prediction function of the model.

[0136] Next, ISM calculates the difference in model predictions before and after the mutation, based on the original prediction values. and the replaced predicted value Calculate the attribution score for each replacement. :

[0137] (13).

[0138] To eliminate systematic bias, the attribution score for each locus is normalized by subtracting the mean of each locus, resulting in a normalized attribution score:

[0139] (14);

[0140] in, This indicates the number of base types in a sequence, in an RNA sequence. =4, thus obtaining an importance matrix.

[0141] Feature map construction.

[0142] The feature spectrum is generated by performing a term frequency-inverse document frequency (TF–IDF) transformation on the discrete latent embeddings of RNA modifications. A codebook feature is treated as a "word," an RNA modification type as a "document," and all RNA modification types as a "corpus," thus constructing the feature spectrum matrix. ,in The number of RNA modification types, This represents the number of codebook features. During TF-IDF conversion, the TF of a feature is first calculated, representing the frequency of a feature within a specific RNA modification type. For the ... The first of the three RNA modification types Codebook characteristics:

[0143] (15);

[0144] in, Indicates the first The total frequency of all features in each RNA modification type is used to avoid bias caused by uneven sample sizes of modification types.

[0145] Next, the IDF (Indexed Function Value) is calculated to measure the rarity of a feature across the entire dataset. A higher IDF value for a rare feature indicates that it is more important in distinguishing RNA modification types.

[0146] (16);

[0147] in, Indicates the inclusion of features The number of RNA modification types.

[0148] Finally, multiplying TF and IDF yields the characteristic spectrum matrix. elements in :

[0149] (17).

[0150] Characteristic spectral matrix Each row corresponds to a type of RNA modification, and each column corresponds to a codebook feature; matrix elements Quantitatively, this reflects the importance of a feature within a specific RNA modification type. To make matrix T more biologically interpretable, its columns are rearranged: first, Z-score normalization is performed based on the highest specificity score for each feature. The corresponding modification type is used to group features belonging to the same modification type together, and then the features within each modification type are sorted from high to low according to their specificity scores.

[0151] Dataset construction and data preprocessing.

[0152] The data used in this embodiment was obtained from public databases such as GEO, RMBase, and RADAR, and supplemented with data from technical reports such as Nm-seq, miCLIP, and RBS-Seq. This data underwent further filtering to ensure the quality of the input data, ultimately resulting in a dataset covering 10 RNA modification types (Am, Cm, Gm, Um, m...). 1 A、m 5 C, m 5 U、m 6 A、m 6 Am、Ψ).

[0153] It is worth noting that the original base for each RNA modification is as follows: (Am, m) 1 A、m 6 A、m 6 Am): A; (Cm, m) 5 C): C; Gm: G; (Um, m) 5 U, Ψ): U. Negative sites are randomly selected from unmodified bases in the same transcript containing positive sites. In this embodiment, the dataset for each RNA modification type is divided into training, test, and validation sets in an 8:1:1 ratio. Model parameters are optimized using the validation set, and the final prediction performance and interpretability analysis report is obtained on the test set.

[0154] To preprocess the RNA sequence data, this embodiment employs a k-mer-based encoding method. To investigate the impact of local and global information on RNA modification, eight inputs were extracted for each RNA modification sequence. Fragments with window sizes of 51, 201, 301, 401, 501, 701, 901, and 1001 pixels were extracted centered on the modification site to provide input for subsequent model training.

[0155] For model evaluation, this embodiment selects five widely used binary classification metrics for performance assessment, including accuracy (ACC), sensitivity (SE), specificity (SP), Matthews correlation coefficient (MCC), and area under the ROC curve (AUC). The formulas for these metrics are as follows:

[0156] (18);

[0157] (19);

[0158] (20);

[0159] (twenty one);

[0160] (twenty two);

[0161] in, , , and These represent true positives, true negatives, false positives, and false negatives, respectively. MCC ranges from -1 to 1, where 1 represents a perfect prediction, 0 represents no different from random guessing, and -1 represents a prediction completely opposite to the true label. AUC refers to the area under the ROC curve, typically between 0.5 and 1.0. The closer the AUC is to 1, the stronger the model's ability to distinguish between positive and negative samples.

[0162] Performance analysis of VQ-RNA input sequence length and codebook size.

[0163] To achieve state-of-the-art performance in predicting multiple RNA modification sites, this embodiment optimized the input sequence length and codebook size. The experimental results in this embodiment are derived from five replicate experiments with different random seeds. With other variables controlled, RNA sequences of 51bp, 201bp, 301bp, 401bp, 501bp, 701bp, 901bp, and 1001bp were used as inputs, and the model's performance was evaluated considering multiple evaluation metrics. It was found that the 501bp sequence input achieved the best ACC (mean ACC: 0.8605), MCC (mean MCC: 0.7227), and Se (mean Se: 0.8420) among the nine modifications, and the best AUC (mean AUC: 0.9172) among the six modifications. Next, the codebook size was adjusted accordingly. For 32, 128, 512, 1024, and 2048, it was found that... When the value is 512, the model's ACC and AUC are both at their highest. For detailed results, please refer to [link / reference needed]. Figure 19 and Figure 20 .

[0164] To further explore the mechanisms of these changes, this embodiment performs a visual analysis of the integrated attention matrix under different sequence lengths and codebook sizes. Figure 21 and Figure 22 This demonstrates the comprehensive attention matrix (integrating the attention distribution across all layers and heads) for different sequence lengths and codebook sizes. Figure 21 and Figure 22 The horizontal and vertical axes represent positions within the sequence. It can be observed that, regardless of sequence length and codebook size, the model consistently focuses its attention on the central region of the sequence, particularly RNA modification sites and their adjacent regions. This consistent attention pattern indicates that information from the central part of the sequence is crucial for predicting RNA modifications.

[0165] A comparison of the predictive performance of VQ-RNA with existing models.

[0166] To comprehensively evaluate the performance of the VQ-RNA model, this embodiment also conducted a comparative analysis with several mainstream classifiers that perform well in RNA modification site prediction tasks. For baseline models, including CNN (Convolutional Neural Network) predictors, GRU (Gated Recurrent Unit) predictors, LSTM (Long Short-Term Memory) predictors, Transformer predictors, and Moss-m7g (based on motif design, using deep learning methods to predict RNA sites, mainly used for RNA N7-methylguanine site prediction), this embodiment trained and evaluated them at a sequence length of 501 bp.

[0167] according to Figure 10 It is evident that the VQ-RNA model proposed in this embodiment has the highest average ACC, Se, Sp, MCC, and AUC, representing increases of 2.11%, 2.98%, 1.24%, 4.06%, and 1.31%, respectively, compared to the second-best Moss-m7G. Figures 11-13 As shown, the model achieved the best ACC, MCC, and AUC among the eight modification types, indicating that the model performs well on both positive and negative samples, demonstrating good robustness and generalization ability.

[0168] VQ-RNA uniformity analysis.

[0169] like Figure 14 As shown, this embodiment analyzes the uniformity of samples in the embedding space. The horizontal axis in the figure represents the first dimension of the normalized embedding vector, and the vertical axis represents the second dimension. t-distributed random neighborhood embedding (t-SNE) is used to map the high-dimensional embedding vector onto a unit hypersphere, thus visualizing the clustering structure. Furthermore, the Davies-Bouldin index (DBI) is calculated to quantify the model's ability to distinguish RNA modification states. The results show that the vector-quantized latent embeddings learned by VQ-RNA achieve excellent separation of modified and unmodified RNA data in the two-dimensional t-SNE space. Taking Am, Cm, and Gm modification types as examples, their modified and unmodified samples form clearly separated clusters in the embedding space constructed by VQ-RNA. In contrast, other methods produce clusters with more overlap and poorer separation. Overall, the vector-quantized embeddings learned by VQ-RNA provide an effective representation of RNA modification, contributing to accurate prediction and interpretation.

[0170] VQ-RNA ablation analysis.

[0171] refer to Figure 15 and Figure 17 In this embodiment, a series of ablation experiments were conducted to analyze the contributions of each component and related loss terms. Among them, Figure 17In the graph, the x-axis represents the first low-dimensional embedding dimension obtained after dimensionality reduction, and the y-axis represents the second low-dimensional embedding dimension obtained after dimensionality reduction. When the VQ module is removed (without VQ), the accuracy drops from 86.05% to 84.33%, while the classification boundaries become blurred and the sample distributions show significant overlap, highlighting the module's role in generating discriminative discrete embeddings. Removing the Transformer module (without transformer) further reduces the accuracy to 81.06%, with even more chaotic classification boundaries, demonstrating the module's importance in capturing long-range dependencies in RNA sequences. Furthermore, replacing the three-layer CNN encoder with a single-layer or two-layer CNN reduces the accuracy to 78.05% and 81.67%, respectively, validating that deeper CNN structures can better capture complex hierarchical features.

[0172] refer to Figure 16 and Figure 18 This embodiment evaluates the impact of different loss terms. Figure 18 In the diagram, the horizontal axis represents the first low-dimensional embedding dimension obtained after dimensionality reduction, and the vertical axis represents the second low-dimensional embedding dimension obtained after dimensionality reduction. Removing the reconstruction loss slightly reduces the accuracy to 84.31%, while removing the VQ loss causes a significant drop in accuracy to 57.60%, highlighting its crucial role in optimizing the latent space. When both types of losses are removed simultaneously, the accuracy further decreases to 57.56%. UMAP visualization results further validate this conclusion: removing the reconstruction loss has a relatively small impact on the feature distribution, while removing the VQ loss leads to disordered feature distribution and blurred class boundaries. This emphasizes the central role of the VQ loss in ensuring a well-structured latent space.

[0173] Overall, only the complete VQ-RNA model can produce clear and consistent class separation, effectively distinguishing modified and unmodified RNA sequences. Removing any component leads to increased sample overlap and misclassification, indicating that the synergy between modules and loss terms is key to achieving optimal performance. Compared to direct sequence encoding, discrete latent embeddings significantly improve classification performance, demonstrating their ability to capture richer features. VQ-RNA is not simply a combination of CNN and Transformer; rather, by introducing VQ modules and multiple loss terms, it effectively fuses local and global features, constructing a more expressive and structured latent representation space.

[0174] Interpretability analysis based on In Silico Saturation Mutagenesis.

[0175] This embodiment uses the ISM method to systematically analyze the importance matrix of the VQ-RNA model across 10 different types of RNA modification data. For this purpose, a 501-base RNA sequence was selected and divided into five regions: 0-100 bp, 100-200 bp, 200-300 bp, 300-400 bp, and 400-500 bp. The center sites of each region were 50, 150, 250, 350, and 450, respectively, with site 250 being the center of the entire sequence and corresponding to the RNA modification site. ISM analysis was performed within these regions, and an importance matrix was returned to quantitatively represent the contribution of each position to the model's predictions.

[0176] Figure 24 The importance scores of 10 RNA modifications at different positions, obtained based on ISM analysis, are presented. The x-axis represents sequence position, and the y-axis represents importance score. It can be seen that motifs are relatively evenly distributed on both sides of the sequence, while significantly higher at the central site and its vicinity. This indicates that the central modification site and its vicinity have higher importance in the model prediction. Furthermore, a MotifLogo diagram was plotted based on the importance matrix to visualize the potential motifs identified by VQ-RNA. Figure 22 The results show the motif visualization centered on the modification site. By observation, it can be found that the central site of the sequence (index = 250) is mostly accurately identified as its original base, and the central region is highly significant. These results further support the key role of the central modification site and its neighboring region in RNA modification recognition.

[0177] like Figure 23As shown, ISM analysis revealed several significant motif patterns. The x-axis represents sequence location, and the y-axis represents importance score. First, some of the most common motif boxes were identified. For example, the "GC box" frequently appeared near the center of modifications such as Cm, Gm, m1A, m5C, m6A, and Ψ; the "CAAT box" was significantly enriched in Gm and Um modifications; and the "CAAU box" was prominent in m6A and Ψ modifications. Furthermore, different RNA modifications exhibited specific motif patterns in their centers and surrounding regions. The motif “GA” frequently appears in Am modifications; enriched repeat patterns such as “CCC,” “GGG,” and “UUU” appear in Cm modifications; “AG” and “AU” frequently appear in Gm modifications; “AGAUCU,” U-rich, and A-rich sequences are significantly enriched in Um modifications; “GC” and “CCCCC” frequently appear in m1A modifications; “GCCC” is prominent in m5C modifications; “AU,” C-rich, and A-rich patterns frequently appear in m5U modifications; “AC” is significantly enriched in m6A modifications; “UC” is common in m6Am modifications; and “UUUG” and “AUUU” frequently appear in Ψ modifications. The frequent occurrence of these motifs suggests that they may play a crucial role in the recognition of specific RNA modifications, revealing the specificity and conservation patterns of RNA modifications and providing clues for further research into the regulatory mechanisms of RNA modifications.

[0178] Specific analysis of RNA modification feature maps.

[0179] VQ-VAE generates discrete latent embeddings based on a shared codebook, and uses TF-IDF transformation to intuitively and quantitatively measure the specificity between codebook features and RNA modification types. The generated feature maps validate the VQ-RNA model's ability to specifically capture RNA modification types, and its complementarity with traditional motif analysis results.

[0180] like Figure 25As shown, a significant concentration of features corresponding to each RNA modification type can be observed, indicating that feature profiling can effectively capture modification-type-specific signals. For example, in certain specific modification types, the specificity scores of some features are significantly higher than those of other modification types, revealing the potential importance of these features in distinguishing RNA modification types. Furthermore, even for RNA modification types with limited data (e.g., Gm), a set of features with significant specificity scores can still be identified. This demonstrates that feature profiling also exhibits good robustness in small sample modification types. Compared to traditional motif analysis, which relies on explicit sequence features, feature profiling utilizes latent embeddings to capture rare specific information in a high-dimensional feature space. Moreover, feature profiling emphasizes the quantitative relationship between features and modification types, providing a more insightful tool for studying the regulatory mechanisms of RNA modifications.

[0181] Example 2

[0182] This embodiment provides a multi-type RNA modification prediction system based on a vector quantization variational autoencoder, including:

[0183] The preprocessing module is configured to acquire and preprocess the RNA sequence to be tested.

[0184] The encoding module is configured to use convolutional encoding operations to obtain the latent representation of the features based on the obtained preprocessing results;

[0185] The vector quantization module is configured to determine the embedding vector with the smallest Euclidean distance to the feature latent representation in the trained discrete codebook, thereby obtaining the discrete vector quantization vector.

[0186] The decoding module is configured to quantize the vector from the discrete vector, perform a deconvolution decoding operation to obtain the reconstructed RNA sequence, and optimize the encoding and decoding operation by minimizing the reconstruction loss.

[0187] The attention processing module is configured to quantize discrete vectors and extract feature vectors using Transformer encoding operations that include a multi-head attention mechanism, thereby obtaining the probabilities of different RNA modification types and determining the RNA modification type.

[0188] In further embodiments, the following is also provided:

[0189] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0190] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0191] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0192] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0193] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0194] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0195] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0196] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for multi-type RNA modification prediction based on vector quantization variational autoencoder, characterized in that, The method comprises the following steps: obtaining a to-be-tested RNA sequence and performing preprocessing; obtaining a feature latent representation by using a convolutional encoding operation according to the obtained preprocessing result; determining an embedding vector with the minimum Euclidean distance from the feature latent representation in a trained discrete codebook, to obtain a discrete vector quantization vector; obtaining a reconstructed RNA sequence by using an inverse convolutional decoding operation according to the discrete vector quantization vector, to optimize the encoding and decoding operations by minimizing a reconstruction loss; extracting a feature vector by using a Transformer encoding operation comprising a multi-head attention mechanism, to obtain probabilities of different RNA modification types, and thus determine the RNA modification type.

2. The method of claim 1, wherein the method is based on a vector quantization variational autoencoder. The preprocessing comprises: extending the to-be-tested RNA sequence symmetrically upwards and downwards from a modification site, and respectively cutting base sequence fragments of different window lengths; and cutting the base sequence fragments into continuous substrings of a fixed length in a sliding window manner, and performing One-Hot encoding on the obtained k-mer index sequence.

3. The method of claim 1, wherein the method is based on a vector quantization variational autoencoder. The convolutional encoding operation is performed by using three one-dimensional convolutional layers; and the inverse convolutional decoding operation is performed by using three inverse convolutional layers.

4. The method of claim 1, wherein the method is based on a vector quantization variational autoencoder. The training process of the discrete codebook comprises: codebook loss : ; Commitment loss : ; where, , is the codebook size; is the stop gradient operation; is the weight of the commitment loss; and are the kth and jth codebook vectors, respectively; is the embedding representation generated by the encoder; argmin denotes a function that takes the minimum value.

5. The method of claim 1, wherein the method is based on a vector quantization variational autoencoder. The Transformer encoding operation comprises: using 6 encoding layers, and each encoding layer comprises 8 attention heads to extract the feature vector.

6. The method of claim 1, wherein the method is based on a vector quantization variational autoencoder. The probability prediction process comprises: taking the extracted feature vector as the input of a fully connected network, and gradually reducing the dimension of the feature vector through multiple fully connected layers, Leaky ReLU activation functions and batch normalization operations in the fully connected network, and finally outputting a two-dimensional vector, and mapping the two-dimensional vector to a classification result through a Softmax layer.

7. A multi-type RNA modification prediction system based on vector quantization variational autoencoder, characterized in that, The method comprises the following steps: a preprocessing module configured to obtain a to-be-tested RNA sequence and perform preprocessing; an encoding module configured to obtain a feature latent representation by using a convolutional encoding operation according to the obtained preprocessing result; a vector quantization module configured to determine an embedding vector with the minimum Euclidean distance from the feature latent representation in a trained discrete codebook, to obtain a discrete vector quantization vector; a decoding module configured to obtain a reconstructed RNA sequence by using an inverse convolutional decoding operation according to the discrete vector quantization vector, to optimize the encoding and decoding operations by minimizing a reconstruction loss; an attention processing module configured to extract a feature vector by using a Transformer encoding operation comprising a multi-head attention mechanism, to obtain probabilities of different RNA modification types, and thus determine the RNA modification type.

8. An electronic device, comprising: A computer program product comprising a memory and a processor, and computer instructions stored on the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method of any one of claims 1-6 is implemented.

9. A computer-readable storage medium, characterized in that, A computer program product for storing computer instructions, wherein when the computer instructions are executed by a processor, the method of any one of claims 1-6 is implemented.

10. A computer program product, characterised in that, A computer program product for storing computer instructions, wherein when the computer instructions are executed by a processor, the method of any one of claims 1-6 is implemented.

Images