An RNA base pairing prediction method fusing multi-scale spatial features and long-range dependencies
By fusing multi-scale spatial features and long-range dependencies using the MS-Conv-Transformer model, the accuracy and stability issues in RNA base pairing prediction are resolved, achieving high-precision and stable base pairing prediction, which is suitable for mRNA vaccine design and disease molecular biomarker discovery.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FIRST AFFILIATED HOSPITAL OF LIAONING UNIV OF TRADITIONAL CHINESE MEDICINE
- Filing Date
- 2026-05-12
- Publication Date
- 2026-07-31
AI Technical Summary
Existing RNA base pairing prediction techniques suffer from limited prediction accuracy, insufficient long sequence processing capabilities, incomplete feature extraction, and poor training stability. In particular, they fail to effectively integrate multi-scale spatial features and long-range dependencies.
The MS-Conv-Transformer model is adopted, combined with optimization techniques such as batch normalization and sequence masking. Global long-range dependency features are extracted through a multi-head self-attention mechanism, and local features are extracted through a one-dimensional convolutional neural network, outputting a base pairing probability vector.
It significantly improves the accuracy and stability of RNA base pairing prediction, enhances the ability to predict local structures, reduces feature engineering costs, and provides quantitative output to meet the needs of practical applications.
Smart Images
Figure CN122493944A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of bioinformatics and deep learning technology, specifically to a method for predicting RNA base pairings that integrates multi-scale spatial features and long-range dependence. Background Technology
[0002] RNA, as a highly flexible linear biological macromolecule, needs to fold to form specific secondary structures such as stem-loop to achieve its biological functions. Base pairing probability is a core indicator for assessing RNA stability and function, providing key data support for mRNA vaccine design, disease molecular biomarker discovery, and RNA-targeted drug development.
[0003] Currently, RNA base pairing probability calculation techniques are mainly divided into thermodynamic model-guided methods (such as ViennaRNA), allocation function methods (such as the McCaskill algorithm), and machine learning methods that have emerged in recent years (CNN, LSTM, Transformer, etc.). However, existing technologies have significant limitations: 1. Limited prediction accuracy due to reliance on thermodynamic models: Traditional methods assume that RNA stability is determined solely by free energy, ignoring the influence of sequence context and complex structural features, and thus failing to capture the true sequence-structure relationship; 2. Insufficient long sequence processing capability: Classical RNN / LSTM models suffer from gradient vanishing / exploding problems when processing long RNA sequences, making it difficult to extract long-distance dependence features of base pairing; 3. Incomplete feature extraction: A single deep learning model cannot take into account both the global correlation (Transformer advantage) and local correlation (CNN advantage) of RNA base pairing, while the real RNA structure is affected by both global constraints and local environment; 4. Poor training stability: RNA data itself has a large variance, and traditional layer standardization techniques are prone to data oscillation and outlier interference during the training process, affecting model convergence and prediction consistency.
[0004] Currently, there are only prediction tools on the market based on a single thermodynamic model or a single deep learning model. There are no patents related to the MS-Conv-Transformer model that integrates multi-scale spatial features and long-range dependencies. There is a significant market gap for high-precision and high-stability RNA base pairing prediction technology. Summary of the Invention
[0005] To address the problems existing in the prior art, this invention provides an RNA base pairing prediction method that integrates multi-scale spatial features and long-range dependencies. This invention is based on an MS-Conv-Transformer-based RNA base pairing probability prediction method. By fusing the global long-range dependency capture capability of the Transformer with the local feature extraction capability of a one-dimensional convolutional neural network, and combining optimization techniques such as batch normalization and sequence masking, it achieves high-precision prediction of RNA base pairing probabilities.
[0006] To achieve the above-mentioned objectives, the present invention provides the following technical solutions.
[0007] This invention discloses a method for predicting RNA base pairings that integrates multi-scale spatial features and long-range dependence, characterized by comprising the following steps: S1: Obtain the primary RNA sequence and secondary structure symbol sequence, and perform word embedding encoding and sine-cosine position encoding on the primary sequence and secondary structure symbol sequence respectively to generate a primary structure encoding matrix and a secondary structure encoding matrix; S2: Construct the MS-Conv-Transformer model, which adopts an encoder-decoder architecture. The encoder takes the primary structure encoding matrix as input, extracts global long-range dependency features through a multi-head self-attention mechanism, and passes them to the decoder. The decoder takes the secondary structure encoding matrix as input, ensures prediction causality through a sequence masking mechanism, and fuses the global features output by the encoder. The decoder output is then concatenated with a one-dimensional convolutional neural network, which extracts local interaction patterns by sliding multiple convolutional kernels along the sequence dimension, and outputs a base pairing probability vector of the same length as the sequence after passing through a fully connected layer and a Softmax layer. In the encoder and decoder, batch normalization is used instead of layer normalization for normalization in each sub-module, and the training loss function is a joint loss function of root mean square error and mean absolute error.
[0008] Furthermore, the encoder is composed of six stacked encoding sub-modules, each encoding sub-module containing a multi-head self-attention layer and a feedforward neural network layer, and adding residual connections and batch normalization; the decoder is composed of six stacked decoding sub-modules, each decoding sub-module containing a masked multi-head self-attention layer, a multi-head attention layer and a feedforward neural network layer, and adding residual connections and batch normalization.
[0009] Furthermore, in step S1, the word embedding encoding has a dimension of 64. The word embedding encoding of the primary sequence and the word embedding encoding of the secondary structure symbol sequence are performed independently to generate word embedding matrices with dimensions [1, L, 64], where L is the sequence length. Then, the word embedding matrix and the position encoding matrix are added together to obtain the final numerical encoding output.
[0010] Furthermore, the sequence masking mechanism is as follows: a mask matrix with negative infinity upper triangular elements is generated and added to the multi-head self-attention layer of the decoder in an additive masking manner, so that the attention at the current position depends only on the information of the previous position.
[0011] Furthermore, the hyperparameters during training were set as follows: batch size 16, word embedding dimension 64, number of encoder layers 6, number of decoder layers 6, and the Adam optimizer was used, combined with dynamic learning rate decay and early stopping mechanism.
[0012] Furthermore, the primary RNA sequence obtained in step S1 is an A / U / C / G base sequence, and the RNA secondary structure symbol sequence is a dotted bracket notation symbol sequence, where “.” represents an unpaired free base, and “(” and “)” represent the positions where base pairs are formed.
[0013] Furthermore, when the secondary structure of the RNA sequence to be predicted is unknown, the following steps are used for prediction: a secondary structure prediction tool is used to make a preliminary prediction of the secondary structure of the sequence to be predicted, generating an estimated secondary structure sequence in bracket notation, and the primary sequence and the estimated secondary structure sequence are encoded in step S1 and then input into the trained MS-Conv-Transformer model to output a base pairing probability vector.
[0014] The present invention also discloses an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, characterized in that the processor executes the program to implement the steps of any of the above-described methods for predicting RNA base pairings that integrate multi-scale spatial features and long-range dependence.
[0015] The present invention also discloses a computer-readable storage medium storing a computer program thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the RNA base pairing prediction method that integrates multi-scale spatial features and long-range dependence as described above.
[0016] The present invention also discloses the application of the method, electronic device, or computer-readable storage medium according to any one of the above-described methods in the evaluation of mRNA vaccine sequence stability, dynamic analysis of disease-related RNA structures, or design of RNA-targeted drugs.
[0017] Compared with the prior art, the beneficial effects of the present invention are as follows.
[0018] (I) Significantly Improved Prediction Accuracy: This invention was compared with seven models, including TextCNN, RNN, LSTM, Bi-LSTM, Transformer, and BPfold, on the same test set. Experimental results show that this invention achieves the best performance in both the root mean square error (RMSE) and mean absolute error (MAE), the two core evaluation metrics. Specifically, compared with the currently representative BPfold model, this invention reduces RMSE by 16% and MAE by 20%, demonstrating that this invention achieves a significant improvement in the accuracy of RNA base pairing probability prediction.
[0019] (II) Enhanced Local Structure Prediction Ability: In the single RNA sequence prediction alignment line graph, the prediction curve of this invention (green line) almost perfectly matches the actual base pairing probability curve, especially in the peak region representing the stem region, where the predicted value is highly consistent with the actual value. While BPfold (red line) can roughly track the overall trend, it exhibits obvious spikes and under-prediction in the peak region. This experimental result proves that the architecture design of this invention, which uses an encoder-decoder followed by a cascaded one-dimensional convolutional neural network, enables the model to achieve sharper local edge detection capabilities than the BPfold embedded convolutional structure when processing local "hairpin structures," realizing deep complementarity between global dependencies and local features.
[0020] (III) Optimization of Training Stability and Convergence Speed: A comparison of the loss convergence curves shows that BPfold (dashed line) exhibits significant oscillations and slow convergence in the early stages of training, easily getting trapped in local optima. In contrast, the present invention (solid line), by using batch normalization instead of layer normalization, results in a rapidly decreasing loss curve with a steeper slope, significantly reduced oscillation amplitude, and eventual stabilization at a lower error floor. These experimental results demonstrate that the batch normalization improvement of this invention, addressing the physical characteristics of RNA sequence data—large variance and susceptibility to data oscillations during training—effectively smooths out data variance fluctuations, leading to the unexpected technical effect of "easier training and faster convergence."
[0021] (iv) Eliminating dependence on physical energy repositories and reducing feature engineering costs: This invention adopts a purely data-driven input representation method, using only the primary RNA sequence (A / U / C / G) and secondary structure symbols (dotted bracket notation). Through learnable word embedding layers and sinusoidal positional encoding, it transforms these into numerical matrices, implicitly learning the mapping relationship between sequences and structures entirely from the data. Compared to methods such as BPfold, which require pre-constructing a base pair Motif energy repository and using BRIQ force fields to calculate thermodynamic energies, this invention significantly reduces feature engineering costs, eliminates dependence on the completeness of physical energy repositories, and enables the model to have stronger generalization ability and deployment convenience across different types of RNA sequences.
[0022] (V) Quantitative Output Meets Practical Application Needs: This invention defines the prediction task as a regression problem, directly outputting the pairing probability value of each base, forming a probability vector of the same length as the sequence. Compared to the existing technology that uses a classification task to output a discrete contact matrix, the quantitative output of this invention can be directly used in uncertainty quantification scenarios without post-processing transformation, such as mRNA vaccine sequence stability assessment and dynamic analysis of disease-related RNA structures, providing more accurate quantitative basis for downstream applications.
[0023] (vi) Systematic hyperparameter optimization ensures the reproducibility of model performance: This invention determines the optimal hyperparameter combination through grid search, including a batch size of 16, a word embedding dimension of 64, and a encoder-decoder layer of 6, balancing training efficiency, feature representation ability, and generalization ability. Combined with the Adam optimizer, dynamic learning rate decay, and early stopping mechanism, the reproducibility and transferability of model performance are guaranteed, providing an efficient, stable, and quantifiable engineering solution for RNA base pairing probability prediction. Attached Figure Description
[0024] Figure 1 This is an overall flowchart of the method of the present invention.
[0025] Figure 2 This is a diagram of the MS-Conv-Transformer model architecture of the present invention.
[0026] Figure 3 This is a flowchart of the multi-head self-attention mechanism calculation of the present invention.
[0027] Figure 4 This is a comparison chart of layer standardization and batch standardization in this invention.
[0028] Figure 5 This is a flowchart of the sequence masking process of the present invention.
[0029] Figure 6 This is a diagram showing the base pairing relationships of the primary RNA structure of this invention.
[0030] Figure 7 This is a diagram showing the base pairing relationships of the RNA secondary structure in this invention.
[0031] Figure 8 This diagram illustrates the impact of the number of codec layers on RMSE and MAE performance in this invention.
[0032] Figure 9 This is a performance comparison chart of the present invention and different models on the test set.
[0033] Figure 10 This is a line graph showing the alignment between the prediction of this invention and the single RNA sequence predicted by BPfold.
[0034] Figure 11 This is a comparison of the convergence curves of the present invention and the BPfold loss curve. Detailed Implementation
[0035] The present invention will be further described in detail below with reference to specific embodiments. However, this should not be construed as limiting the scope of the above-described subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.
[0036] Unless otherwise specified, all reagents and materials used in this invention are commercially available.
[0037] Example 1: Method for predicting the probability of RNA base pairing.
[0038] 1. Experimental dataset and settings.
[0039] 1.1 Data sources and composition.
[0040] This invention is based on a dataset of 5,000 publicly available RNA sequences, sourced from the RNA STRAND v3.0 and Rfam 14.9 databases. The dataset comprises 40% tRNA sequences, 30% rRNA sequences, 20% mRNA sequences, and 10% lncRNA sequences. The sequence lengths range from 50 to 500 nucleotides, with an average length of 180 nucleotides.
[0041] 1.2 Data cleaning rules.
[0042] The data cleaning rules are as follows: (1) Remove sequences with a length less than 50 nt or greater than 500 nt; (2) Remove sequences containing non-standard bases (non-A / U / C / G); (3) Eliminate sequences with incomplete secondary structure annotations or mismatched parentheses; (4) Use the RNAplfold module of ViennaRNA 2.5.1 software package to calculate the pairing probability (window size 150, maximum pairing distance 120) and generate the "true pairing probability" label.
[0043] 1.3 Dataset composition and partitioning.
[0044] The dataset contains four labels: "ID, RNA sequence, RNA structure, and true pairing probability". RNA sequence: composed of four bases: A, U, C, and G, following the AU pairing, GC pairing, and GU wobble pairing rules; RNA secondary structure: The dot-bracket notation is used, where "." represents an unpaired free base and "()" represents the position where a base pair will form. True pairing probability: The pairing probability value for each base position calculated using ViennaRNA's RNAplfold module.
[0045] The dataset was divided into a training set (4000 records) and a validation set (1000 records) in an 8:2 ratio. Batch training was iterated for 100 rounds. The test set used independent RNA sequencing samples to ensure the objectivity and reliability of the experimental results.
[0046] 2. Model building and training (training phase).
[0047] 2.1 Data preprocessing.
[0048] Data cleaning was performed on the training and validation sets to remove incomplete and erroneous sequences; the primary RNA structure (A / U / C / G) and secondary structure (dotted bracket notation) were converted into numerical codes to complete data standardization.
[0049] 2.2 Sequence encoding.
[0050] Step S1 completes word embedding encoding (64 dimensions) and sine-cosine positional encoding to generate the encoding matrix for the model input. Specifically, the symbol sequence of dimension [1, L] (L is the sequence length) is converted into a word embedding matrix of dimension [1, L, 64]. The primary structure sequence (A / U / C / G) and the secondary structure sequence (dotted bracket notation) are independently word embedded and encoded. Then, the positional encoding matrix is superimposed to form the final numerical input representation.
[0051] 2.3 Model Construction.
[0052] Follow step S2 to build the MS-Conv-Transformer model. The specific configuration is as follows: Encoder: 6 layers, each layer contains a multi-head self-attention layer and a feedforward neural network layer; Decoder: 6 layers, each layer contains a masked multi-head self-attention layer, a multi-head attention layer, and a feedforward neural network layer; Batch size: 16; Optimization techniques: Incorporate residual join, batch normalization, and sequence masking mechanisms.
[0053] 2.4 Model training.
[0054] Set the training parameters in step S3: Loss function: The root mean square error (RMSE) and mean absolute error (MAE) are used as the joint loss function; Optimizer: Adam optimizer is used; Training strategy: Combine dynamic learning rate decay with early stopping mechanism to train until the model converges.
[0055] 2.5 Model Validation and Testing.
[0056] Optimize hyperparameters on the validation set and validate model performance on the independent test set.
[0057] 3. RNA base pairing probability prediction (prediction phase).
[0058] 3.1 Predicted Input Description.
[0059] In the actual prediction stage, this invention provides two prediction modes: Mode 1: Prediction Assisted by Known Secondary Structure (High-Precision Mode) When the secondary structure information of the RNA sequence to be predicted has been obtained through experimental means (such as SHAPE-MaP, DMS-seq, etc.), the primary structure and secondary structure can be input at the same time, and the model will output high-precision base pairing probability prediction results. Mode 2: Prediction based solely on primary structure (general mode) When the secondary structure of the RNA sequence to be predicted is unknown (the most common situation in practical applications), this invention adopts the following strategy: Initial estimation of secondary structure: First, use a fast secondary structure prediction tool (such as RNAfold, LinearFold, etc.) to make an initial prediction of the secondary structure of the sequence to be predicted, and generate a secondary structure estimate in bracket notation; Pairing probability prediction: Input the primary structure sequence and the estimated secondary structure simultaneously into the trained MS-Conv-Transformer model; Output: The model calculates the pairing probability vector for each base position through forward propagation.
[0060] 3.2 Prediction process.
[0061] The primary structure sequence of RNA to be predicted is encoded in step S1 and then input into the trained MS-Conv-Transformer model. The model calculates the pairing probability vector of each base position through forward propagation. The prediction results can be directly applied to scenarios such as mRNA vaccine sequence optimization, disease diagnosis, and drug target design.
[0062] 4. Detailed explanation of the model architecture.
[0063] 4.1 Step S1: Encoding of RNA primary and secondary structures.
[0064] This step forms the foundation of the model, transforming the discrete sequence / structural symbols of RNA into a numerical representation suitable for deep learning. It includes three sub-steps: Data Acquisition and Preprocessing: RNA data was acquired from public databases / experimental sequencing results, including primary structure sequences composed of A / U / C / G, and secondary structures represented by dotted bracket notation. The basic linear arrangement of bases and potential pairing positions of the primary RNA structure are as follows: Figure 6 As shown, typical patterns of RNA secondary structure, such as stems and loops formed by base pairing, are as follows: Figure 7 As shown, the dotted bracket notation can intuitively correspond to the secondary structure features of this class; Word embedding encoding: Through a trainable embedding layer, a sequence of symbols of dimension [1, L] is converted into a word embedding matrix of dimension [1, L, 64] (L is the sequence length, 64 is the word embedding dimension), mapping discrete base / structure symbols into continuous vectors, capturing deep semantic and functional associations; Positional encoding: A sine-cosine function is used for positional encoding to add positional information to the word embedding matrix, ensuring that the model can distinguish the same symbols in different positions. The positional encoding matrix is added to the word embedding matrix as the final encoding output.
[0065] 4.2 Step S2: MS-Conv-Transformer model construction.
[0066] The core architecture of the model integrates the Transformer encoder-decoder architecture with one-dimensional convolutional layers, simultaneously extracting global long-range dependencies and local structural features. The overall architecture design and the integration relationships of each module are as follows: Figure 2 As shown, the entire structure is divided into three parts: encoder, decoder, and one-dimensional convolutional and output layers. The encoder consists of six stacked encoder submodules. Each submodule contains a multi-head self-attention layer and a feedforward neural network layer, with residual connections and batch normalization added. Taking the primary structure encoding matrix as input, it captures the global long-range dependencies of the RNA sequence through a multi-head self-attention mechanism. The interaction between query, key, and value vectors and the generation of attention weights in the multi-head self-attention mechanism are as follows: Figure 3 As shown, the global features are finally extracted and passed to the decoder; The decoder consists of six stacked decoder submodules. Each submodule contains a masked multi-head self-attention layer, a multi-head attention layer, and a feedforward neural network layer. It takes a two-level structure encoding matrix as input and generates a mask matrix with an upper triangular radius of negative infinity through a sequence masking mechanism. The specific processing flow and information masking principle of the sequence masking are as follows... Figure 5 As shown, this effectively prevents the decoder from obtaining future information in advance, ensuring the causality of the prediction; at the same time, it integrates the global features output by the encoder to achieve deep integration of primary / secondary structural information. One-dimensional convolution and output: The decoder output matrix is fed into a one-dimensional convolutional layer. This layer has 128 convolutional kernels, each with a size of 3, meaning each kernel slides along the sequence dimension, covering the features of 3 adjacent bases at a time. Each kernel slides along the sequence dimension, extracting local interaction patterns between adjacent bases. Subsequently, after ReLU activation and max pooling to compress the features, a fully connected layer and a softmax layer output a base pairing probability vector of the same length as the sequence.
[0067] In all sub-modules of the encoder and decoder, this invention innovatively introduces batch normalization to replace the traditional layer normalization technique. The performance and stability comparison between layer normalization and batch normalization in RNA data training is as follows: Figure 4 As shown, batch standardization is better suited to the large variance characteristics of RNA data and can effectively solve the training oscillation problem.
[0068] 4.3 Step S3: Model training and optimization.
[0069] Employing a systematic training strategy to improve model prediction performance and training stability, the core points are: Joint loss function: The root mean square error (RMSE) and the mean absolute error (MAE) are used as the joint loss function to balance the penalty for large errors and the control of the overall error distribution. Batch normalization optimization: Batch normalization is introduced in all sub-modules of the encoder / decoder to replace traditional layer normalization, which solves the problem of large variance in RNA data, avoids training oscillations, and improves convergence speed and prediction consistency. Hyperparameter optimization: The optimal combination of hyperparameters was determined by grid search—batch size 16, word embedding dimension 64, and encoder-decoder layers 6, which balances training efficiency, feature representation ability and generalization ability. Overfitting prevention strategies: Use the Adam optimizer to update parameters; introduce dynamic learning rate decay and early stopping mechanisms.
[0070] 4.4 Step S4: Prediction of RNA base pairing probability.
[0071] The primary / secondary structure sequence of RNA to be predicted is encoded in step S1 and then input into the trained MS-Conv-Transformer model. The model calculates the pairing probability vector of each base position through forward propagation. The prediction results can be directly applied to scenarios such as mRNA vaccine sequence optimization, disease diagnosis, and drug target design.
[0072] Example 2: Experimental Verification.
[0073] 1. Validation of training stability.
[0074] Batch normalization reduces the fluctuation of loss values during training by more than 60%, improves model convergence speed by 30%, and reduces the error difference between the test set and the training set to within 5%. A comparison of training stability between batch normalization and layer normalization is provided. Figure 4 As shown.
[0075] 2. Validation of hyperparameter validity.
[0076] The impact of the number of encoder / decoder layers on model performance is as follows: Figure 8 As shown, the 6-layer encoder-decoder structure achieves the best RMSE and MAE metrics. Too few layers can easily lead to underfitting, while too many layers can easily introduce overfitting and increase the difficulty of training.
[0077] 3. Comparison of prediction accuracy.
[0078] On the same test set, this invention was compared fairly with six other models: TextCNN, RNN, LSTM, Bi-LSTM, Transformer, and BPfold. All models were run in a unified experimental environment: hardware consisted of an Intel Xeon Gold6248R × 24-core CPU, an NVIDIA Tesla V100 32GB × 2 GPU, and 256 GB of memory; software consisted of Ubuntu 20.04 LTS, Python 3.8.10, PyTorch 1.10.0 + CUDA 11.3; the dataset consisted of 5,000 RNA sequences (40% tRNA, 30% rRNA, 20% mRNA, and 10% lncRNA, 50–500 nt in length) from the RNA STRAND v3.0 and Rfam 14.9 databases, divided in an 8:2 ratio into a training set of 4,000 sequences and a validation set of 1,000 sequences, with the test set consisting of 500 independent RNA sequencing samples. The random seed was fixed at 42 to ensure reproducibility.
[0079] All six comparative models strictly followed the standard configuration of the original open-source code in the paper for reproduction, without any hyperparameter tuning for this RNA dataset, to ensure the objectivity and reproducibility of the comparative experiments: TextCNN used the default configuration of the original paper (convolutional kernel [3,4,5]×100, Dropout=0.5); RNN / LSTM / Bi-LSTM used the official PyTorch implementation (128-dimensional hidden layers, 2 LSTM layers, Dropout=0.3); Transformer used the standard configuration of the original paper (6 layers each for the encoder and decoder, 8 heads, d_model=512); BPfold used its open-source code with default parameters (beam size=5, partition function threshold=0.01). All models used the exact same dataset partitioning, evaluation metrics (RMSE / MAE), and training epochs (100 epochs) as this invention.
[0080] Experimental results are as follows Figure 9 As shown, this invention achieves optimal performance in both RMSE and MAE, the two core evaluation indicators. Specifically, the RMSE of this invention is 0.82, and the BPfold is 0.98, a relative reduction of 16.33%; the MAE of this invention is 0.63, and the BPfold is 0.79, a relative reduction of 20.25%.
[0081] 4. Comparison of prediction performance of single RNA sequences.
[0082] Single RNA sequence prediction alignment line graph as shown Figure 10 As shown, the black line represents the actual fluctuations in RNA base pairing probabilities (peaks represent the stem region, and troughs represent the free loop region). While BPfold (red line) can roughly track the trend, it exhibits significant spikes and under-prediction in the peak region; whereas the prediction curve of this invention (green line) almost perfectly matches the actual peaks and troughs. This demonstrates that the addition of a one-dimensional convolutional layer enables the model to achieve sharper local edge detection capabilities than the BPfold embedded convolutional structure when handling local "hairpin structures."
[0083] 5. Comparison of convergence speeds.
[0084] Loss convergence curve as shown Figure 11 As shown, the dashed line represents BPfold, and the solid line represents the present invention. BPfold exhibits significant oscillations and slow convergence in the early stages of training, easily getting trapped in local optima. In contrast, the present invention, by using batch normalization instead of layer normalization, results in a loss curve that drops rapidly with a steeper slope, significantly reducing oscillations and eventually stabilizing at a lower error floor. This demonstrates that batch normalization effectively mitigates the problem of drastic variance fluctuations in RNA data, leading to the technical effect of "easier training and faster convergence."
[0085] In summary, this invention constructs an MS-Conv-Transformer multi-scale fusion model, replacing the traditional physical energy pool dependency with a purely data-driven input representation. It introduces a multi-head self-attention mechanism in the encoder-decoder architecture to capture global long-range dependencies and enhances local structural feature extraction capabilities through a cascaded one-dimensional convolutional neural network. Simultaneously, addressing the large variance of RNA data, batch normalization replaces layer normalization to improve training stability and convergence speed. By combining the root mean square error and mean absolute error as a joint loss function, it directly outputs a base pairing probability vector of the same length as the sequence. This invention achieves deep complementarity between global dependencies and local features, significantly improving prediction accuracy while enhancing model training stability and generalization ability. It provides efficient, stable, and quantifiable technical support for mRNA vaccine sequence optimization, disease molecular biomarker discovery, and RNA-targeted drug development.
[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the patent scope of the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for predicting RNA base pairings that integrates multi-scale spatial features and long-range dependence, characterized in that, Includes the following steps: S1: Obtain the primary RNA sequence and secondary structure symbol sequence, and perform word embedding encoding and sine-cosine position encoding on the primary sequence and secondary structure symbol sequence respectively to generate a primary structure encoding matrix and a secondary structure encoding matrix; S2: Construct the MS-Conv-Transformer model, which adopts an encoder-decoder architecture. The encoder takes the primary structure encoding matrix as input, extracts global long-range dependency features through a multi-head self-attention mechanism, and passes them to the decoder. The decoder takes the secondary structure encoding matrix as input, ensures the causality of the prediction through a sequence masking mechanism, and fuses the global features output by the encoder. The decoder output is concatenated with a one-dimensional convolutional neural network, which extracts local interaction patterns by sliding multiple convolutional kernels along the sequence dimension. The output is a base pairing probability vector of the same length as the sequence after passing through a fully connected layer and a Softmax layer. In the encoder and decoder, batch normalization is used instead of layer normalization for normalization in each sub-module, and the training loss function is a joint loss function of root mean square error and mean absolute error.
2. The method according to claim 1, characterized in that, The encoder consists of six stacked encoding sub-modules. Each encoding sub-module contains a multi-head self-attention layer and a feedforward neural network layer, and residual connections and batch normalization are added. The decoder consists of six stacked decoding sub-modules. Each decoding sub-module contains a masked multi-head self-attention layer, a multi-head attention layer, and a feedforward neural network layer, and residual connections and batch normalization are added.
3. The method according to claim 1, characterized in that, In step S1, the word embedding encoding has a dimension of 64. The word embedding encoding of the primary sequence and the word embedding encoding of the secondary structure symbol sequence are performed independently, generating word embedding matrices of dimension [1, L, 64], where L is the sequence length. Then, the word embedding matrix is added to the position encoding matrix as the final numerical encoding output.
4. The method according to claim 1, characterized in that, The sequence masking mechanism is as follows: a mask matrix with negative infinity upper triangular elements is generated and added to the multi-head self-attention layer of the decoder in an additive masking manner, so that the attention at the current position depends only on the information of the previous position.
5. The method according to claim 1, characterized in that, The hyperparameters during training were set as follows: batch size 16, word embedding dimension 64, number of encoder layers 6, number of decoder layers 6, and the Adam optimizer was used, combined with dynamic learning rate decay and early stopping mechanism.
6. The method according to claim 1, characterized in that, The primary RNA sequence obtained in step S1 is an A / U / C / G base sequence, and the RNA secondary structure symbol sequence is a dotted bracket notation ...
7. The method according to claim 1, characterized in that, When the secondary structure of the RNA sequence to be predicted is unknown, the following steps are used for prediction: a secondary structure prediction tool is used to make a preliminary prediction of the secondary structure of the sequence to be predicted, generating an estimated secondary structure sequence in bracket notation. The primary sequence and the estimated secondary structure sequence are encoded in step S1 and then input into the trained MS-Conv-Transformer model, which outputs a base pairing probability vector.
8. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the RNA base pairing prediction method that integrates multi-scale spatial features and long-range dependence as described in any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the RNA base pairing prediction method that integrates multi-scale spatial features and long-range dependence as described in any one of claims 1 to 7.
10. The application of the method according to any one of claims 1 to 7, the electronic device according to claim 8, or the computer-readable storage medium according to claim 9 in the evaluation of mRNA vaccine sequence stability, dynamic analysis of disease-related RNA structure, or design of RNA-targeted drugs.