A method and system for RNA secondary structure prediction and folding path deduction
By combining deep learning pre-trained models and reinforcement learning agents, and utilizing local structure modules and composite reward functions, the problems of computational complexity and generalization ability in RNA secondary structure prediction and folding path deduction are solved, achieving efficient and accurate RNA folding path deduction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING NEOCURNA BIOTECHNOLOGY CORP
- Filing Date
- 2026-02-12
- Publication Date
- 2026-06-05
Smart Images
Figure CN122157792A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of biological sequence data processing technology, specifically relating to a method and system for predicting RNA secondary structure and deducing folding paths. Background Technology
[0002] RNA secondary structure is the core foundation for RNA molecules to perform their biological functions, and its folding path directly reflects the dynamic process of RNA forming a functional spatial conformation from a linear sequence. The accurate analysis of both is of key significance for non-coding RNA function research, RNA-targeted drug design, and other fields.
[0003] Current research uses graph neural networks for RNA structure modeling, optimizes based on free energy, and simulates using molecular dynamics. However, this approach suffers from high computational complexity, low efficiency, insufficient generalization ability for non-homologous unknown RNA sequences, and an inability to accurately capture conformational dynamics during the folding process, making it difficult to reproduce RNA folding pathways. Summary of the Invention
[0004] The purpose of this invention is to provide a method and system for predicting RNA secondary structure and deducing folding pathways, which can improve computational efficiency and generalization ability, and reproduce RNA folding pathways.
[0005] The first aspect of this invention discloses a method for predicting RNA secondary structure and deducing folding pathways, comprising:
[0006] The RNA sequence to be predicted is input into a deep learning pre-trained model for feature extraction and pattern recognition to obtain several local structural modules composed of at least 3 nucleotide pairs;
[0007] All the aforementioned local structural modules are used as the action space of the reinforcement learning agent. The reinforcement learning agent performs the screening and pairing actions of local structural modules, gradually outputs the prediction results of RNA secondary structure, and records the local structural modules selected at each step and the pairing order to obtain the folding path. The reinforcement learning agent adopts a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability, and adaptively adjusts the weights of the composite reward function according to the folding stage and the current state.
[0008] In some implementations, a learnable dynamic weight adjuster is used to adjust the weights of the composite reward function according to the state code. The dynamic weight adjuster is a lightweight neural network module. The state code includes: folding progress index, structural complexity index, energy status index, and topology health index.
[0009] In some implementations, the composite reward function includes: a pairing quantity reward, a structural logic reward, and a local free energy reward; the pairing quantity reward is obtained based on the number of newly formed pairs; the structural logic reward is obtained based on whether the new stack conflicts with the existing structure, whether the new stack is well nested in the existing structure, and whether the new stack is spatially continuous with the previous action; the local free energy reward is obtained based on the free energy of the local region affected by the new stack.
[0010] In some implementations, obtaining the local free energy reward includes:
[0011] Obtain the local region affected by the new stack and obtain the sequence fragment of the local region;
[0012] The local free energy of the sequence fragment was calculated using a fast energy model;
[0013] Calculate the stability of the new stack;
[0014] The local free energy reward is obtained based on the stability and the local free energy.
[0015] In some implementations, after obtaining the pairing quantity reward, when the length of the new stack is greater than a preset length, the pairing quantity reward is increased, and / or, during the training phase, the composite reward function further includes a supervised reward, which is obtained based on the matching degree between the new stack and the target structure.
[0016] In some implementations, determining whether the new stack conflicts with the existing structure includes:
[0017] If any base in the new stack pairs with another position, a conflict is determined.
[0018] If the new stack forms a pseudo-junction topology with the existing structure, a conflict is determined.
[0019] The nesting score is used to determine whether a new stack is well nested within an existing structure. The steps for calculating the nesting score include:
[0020] When a new stack is adjacent to an existing stack and expands the existing stack, the nesting score is set to the first score.
[0021] When a new stack is wrapped by an existing structure, the nesting score is set as the second score based on the number of existing pairs that are wrapped by it;
[0022] When a new stack conflicts with an existing structure or is poorly nested, the nesting score is set to the third score.
[0023] When the new stack is a completely isolated stack, the nesting score is set to the fourth score.
[0024] In some implementations, after calculating the reward value based on the composite reward function, the method further includes:
[0025] The number of consecutive steps in which the single-step reward value of the reinforcement learning agent is lower than a preset reward threshold is counted in real time. When the number of consecutive steps exceeds a first preset value, the reward value is increased according to a preset coefficient.
[0026] The total number of steps taken by the reinforcement learning agent to complete the folding of RNA secondary structure is counted in real time. When the total number of steps exceeds a second preset value, the reward value is reduced based on the difference between the total number of steps and the second preset value.
[0027] The second aspect of this invention discloses an RNA secondary structure prediction and folding path deduction system, comprising:
[0028] A reinforcement learning agent is used to perform the screening and pairing of local structural modules in an action space, progressively outputting the prediction results of RNA secondary structure, and recording the selected local structural modules and pairing order at each step to obtain the folding path. The reinforcement learning agent adopts a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability. The local structural modules are obtained by feature extraction and pattern recognition of the RNA sequence to be predicted by a deep learning pre-trained model. The local structural modules contain at least 3 nucleotide pairs.
[0029] A learnable dynamic weight adjuster is used to adjust the weights of the composite reward function according to the state encoding. The dynamic weight adjuster is a lightweight neural network module. The state encoding includes: folding progress index, structural complexity index, energy status index, and topology health index.
[0030] A third aspect of the present invention discloses an electronic device, including a memory storing executable program code and a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the RNA secondary structure prediction and folding path deduction method disclosed in the first aspect.
[0031] The fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute the RNA secondary structure prediction and folding path deduction method disclosed in the first aspect.
[0032] The beneficial effects of this invention are that by combining a deep learning pre-trained model and a reinforcement learning agent, the action space of the reinforcement learning agent becomes the local structural modules output by the deep learning pre-trained model. It performs the screening and pairing actions of these local structural modules, rather than selecting single base pairs or pairing single nucleotides. This makes the decision-making process highly consistent with biophysical laws and improves efficiency. By employing a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability, and dynamically adjusting the weights of each item, the reinforcement learning agent is guided to explore conformations and paths that conform to biophysical laws, accurately capturing the dynamic temporal characteristics of RNA folding, reproducing RNA folding paths, and improving generalization ability. Attached Figure Description
[0033] The accompanying drawings illustrate specific examples of the technical solutions described in this invention and, together with the detailed embodiments, form part of the specification, serving to explain the technical solutions, principles, and effects of this invention.
[0034] Unless otherwise specified or defined, the same reference numerals in different figures represent the same or similar technical features, and different reference numerals may be used to represent the same or similar technical features.
[0035] Figure 1 This is a flowchart of an embodiment of the RNA secondary structure prediction and folding path deduction method according to an embodiment of the present invention;
[0036] Figure 2 This is a flowchart illustrating the process of obtaining a local free energy reward according to an embodiment of the present invention;
[0037] Figure 3 This is a schematic diagram of the RNA secondary structure prediction and folding path deduction system according to an embodiment of the present invention;
[0038] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0039] Unless otherwise specified or defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. When combined with the technical solutions of the invention in a real-world scenario, all technical and scientific terms used herein may also have meanings corresponding to the purpose of achieving the technical solutions of the invention. The terms "first," "second," etc., used herein are merely for distinguishing names and do not represent a specific number or order. The term "and / or," as used herein, includes any and all combinations of one or more of the associated listed items.
[0040] It should be noted that when a component is considered "fixed" to another component, it can be directly fixed to the other component or there can be an intervening component; when a component is considered "connected" to another component, it can be directly connected to the other component or there can be an intervening component; when a component is considered "mounted" on another component, it can be directly mounted on the other component or there can be an intervening component; when a component is considered "placed" on another component, it can be directly placed on the other component or there can be an intervening component.
[0041] Unless otherwise specified or defined, the terms "described" or "the" as used herein refer to the technical features or technical content mentioned or described prior to the relevant section, which may be the same as or similar to the technical features or technical content mentioned herein. Furthermore, the terms "comprising" and "having," and any variations thereof, as used herein, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.
[0042] To facilitate understanding of the present invention, specific embodiments of the present invention will be described in more detail below with reference to the accompanying drawings.
[0043] RNA folding is a dynamic, multi-step physical process, but current experimental techniques can only capture a "snapshot" of its final static state. This limitation of "knowing the end but not the path" severely restricts drug design and functional prediction targeting the dynamic intermediate state of RNA.
[0044] Based on this, this invention utilizes reinforcement learning and deep learning pre-trained models to fill the cognitive gaps in dynamic processes, enabling the reverse deduction and simulation of complete folding paths from limited static structural data. Specifically, it creatively shapes reinforcement learning as the "digital deduction engine" for this process, and internalizes the structural knowledge of deep learning pre-trained models as "physical intuition." That is, it allows deep learning pre-trained models to replace / assist researchers' empirical physical judgments, providing a priori structural cognition that conforms to biophysical laws for RNA structure calculations, eliminating the need to deduce RNA structural laws from scratch through complex free energy calculations. Furthermore, it designs a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability, combined with a dynamic weight adjustment mechanism, guiding the agent to explore various possible paths from linear sequences to functional structures under the premise of conforming to biophysical laws. This inventive concept not only transcends simple structural prediction but also aims to reveal the dynamic trajectory and key intermediate states of folding, thereby providing computational assistance and theoretical guidance for discovering novel drug binding targets and designing functional RNA molecules with specific folding dynamics. Moreover, this invention abandons the limitations of existing technologies based on single-base pairing actions and innovatively proposes a new paradigm with local structural modules as the basic decision-making units, which not only improves computational efficiency but also makes the decision-making process highly consistent with biophysical laws.
[0045] like Figure 1 As shown, the embodiments of the present invention specifically include the following steps:
[0046] Step S100: Input the RNA sequence to be predicted into the deep learning pre-trained model for feature extraction and pattern recognition to obtain several local structural modules composed of at least 3 nucleotide pairs;
[0047] Specifically, local structural modules refer to local structural units in RNA that have stable biophysical properties, such as stem regions, hairpin loops, and internal loops. For example, positions 1-10 are predicted to form a stable hairpin loop, and positions 15-25 form a stem region.
[0048] The deep learning pre-trained model is trained on massive amounts of RNA sequence-structure data and biophysical rules (such as base pairing principles, steric hindrance of RNA strands, and thermodynamic stability laws). It autonomously learns and extracts the underlying rules governing RNA structure formation, such as: which sequence features are more likely to form pseudoknots, folding preferences of RNAs of different lengths, and implicit rules of base pairing. After training, the RNA sequence to be predicted is input into the deep learning pre-trained model, which is trained on a large amount of data with secondary structure annotations and is only applicable after accuracy verification. Based on the input sequence fragment (fragment length N), the model outputs an N*N probability matrix of pairing probabilities within the fragment. After sorting and filtering, it extracts pairing groups that may form local structures, outputting several local structural modules composed of at least three nucleotide pairings.
[0049] Among them, the deep learning pre-trained model can also adopt the general pre-trained model in the field of RNA data processing, such as MXFold, RNA-FM, ErnieRNA, etc.
[0050] By first extracting local structural modules using a deep learning pre-trained model, meaningless structural combinations and computational paths can be directly eliminated through prior structural judgments, eliminating the need to traverse all possibilities from scratch and solving the problems of high computational cost and low efficiency in traditional methods. Moreover, the deep learning pre-trained model internalizes the underlying patterns extracted from massive amounts of data, rather than features specific to homologous sequences. It can make judgments that conform to physical laws for non-homologous and unknown RNA sequences, solving the problem of poor generalization ability in traditional methods.
[0051] Step S200: Treat all local structural modules as the action space of the reinforcement learning agent. The reinforcement learning agent performs the screening and pairing actions of local structural modules, gradually outputs the prediction results of RNA secondary structure, and records the local structural modules selected and the pairing order at each step to obtain the folding path. The reinforcement learning agent adopts a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability, and adaptively adjusts the weights of each item of the composite reward function according to the folding stage and the current state.
[0052] RNA secondary structure prediction is a combination decision of base pairing. Reinforcement learning is usually responsible for selecting the base sites to be paired and judging the pairing type. Through multiple rounds of iterative training, reinforcement learning can quickly screen out the optimal base pairing combination that is thermodynamically stable and conforms to biophysical laws. This avoids the full traversal of traditional free energy algorithms and solves the problems of overfitting and large prediction deviations for complex structures in deep learning models. The final output secondary structure has both thermodynamic rationality and biological authenticity.
[0053] Folding path deduction reconstructs the temporal sequence of conformational transitions in RNA from a linear sequence to a functional secondary structure. Reinforcement learning's step-by-step dynamic decision-making is perfectly suited to this need. Based on the RNA conformational state of the previous step and physiological constraints, reinforcement learning determines the folding action at each step, such as the start of base pairing, stem-loop formation, or slight conformational adjustments. It deduces feasible, low-energy-barrier, and temporally reasonable folding paths, not only outputting complete intermediate conformational sequences but also identifying key nodes in the folding process. This solves the problems of low sampling efficiency and difficulty in capturing rare conformational transitions in traditional molecular dynamics simulations.
[0054] The reinforcement learning agent of this invention differs from conventional reinforcement learning, abandoning the limitations of existing technologies based on single-base pairing actions and innovatively proposing a novel paradigm with local structural modules as the basic decision-making units. All local structural modules are treated as the action space of the reinforcement learning agent, which performs the screening and pairing actions of these modules, rather than selecting individual base pairs or pairing individual nucleotides. Since the physical essence of RNA folding begins with local nucleation and gradually forms stable secondary structural units, this invention directly simulates this physical process through the manipulation of local structural modules, making the decision-making process highly consistent with biophysical laws. Furthermore, traditional methods require evaluating all possible base pairings (O(N²) complexity), while this invention, through prior screening using a pre-trained model, compresses the search space to O(N×K), where K is the number of local structural modules. This compression not only improves efficiency but, more importantly, eliminates a large number of thermodynamically infeasible pairing combinations.
[0055] Specifically, this embodiment uses a reinforcement learning agent as an intelligent decision-making optimization layer for RNA secondary structure prediction and folding path deduction. The biophysical laws of RNA structure internalized by the deep learning pre-trained model are used as the initial decision priors for the reinforcement learning agent. A composite reward function is adopted, which includes rewarding the formation of structural modules, penalizing topological conflicts, and evaluating local stability. Through the agent-environment-reward closed-loop decision-making framework, the base pairing decision of RNA secondary structure and the conformational change timing decision of folding path are transformed into a stepwise optimization intelligent decision-making process. This guides the reinforcement learning agent to explore only conformations and paths that conform to biophysical laws, which reduces the computational complexity of traditional free energy optimization and accurately captures the dynamic temporal characteristics of RNA folding. This achieves more accurate secondary structure prediction results and physiological realism in folding path deduction, while also improving the generalization and adaptation ability to non-homologous, complex long RNA sequences.
[0056] In other words, this embodiment uses the local structural modules identified by the deep learning pre-trained model as the basic operation units, instead of the traditional single base operation units. In each step of the reinforcement learning decision, the local structural modules composed of 5-10 nucleotides are used as the basic units for action execution, rather than single base pairs.
[0057] To address the core challenges of convergence difficulties and susceptibility to local optima in RNA folding problems, this invention innovatively employs a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability, assessing decision quality from different physical levels. Traditional methods typically use a single free energy minimization or structural similarity as the reward function. Free energy itself cannot reconcile dynamics and synergy. Minimum free energy models pursue the globally stable state, but biological functions often depend on specific folding paths, metastable states, or dynamic characteristics. Minimum free energy models mostly fail to characterize these dynamic features crucial to function. Simply pursuing a precise match between the sequence folding result and the target structure, this "one-size-fits-all" reward approach, cannot design optimal sequences that balance stability, foldability, and functionality.
[0058] Specifically, the composite reward function in this embodiment includes: a pairing quantity reward R1, a structural logic reward R2, and a local free energy reward R3. The pairing quantity reward R1 is obtained based on the number of newly formed pairs, used to encourage rapid structure formation; the structural logic reward R2 is obtained based on whether the new stack conflicts with existing structures, whether the new stack is well nested within existing structures, and whether the new stack is spatially continuous with the previous action, used to ensure topological rationality; the local free energy reward R3 is obtained based on the free energy of the local region affected by the new stack, used to provide guidance on thermodynamic stability. During the training phase, the composite reward function also includes a supervised reward R4, which is obtained based on the matching degree between the new stack and the target structure, used to accelerate training using known data. The target structure is a known, correct, standard structure used as a training benchmark.
[0059] It is important to emphasize that this invention innovatively proposes a local free energy assessment rather than a global free energy calculation through a local free energy reward R3. In the early stages of folding, global free energy calculation is neither reliable nor meaningful. The local free energy assessment of this invention provides timely and accurate physical feedback for each step of the decision-making process, avoiding the blind spots inherent in traditional methods during the initial exploration phase.
[0060] Specifically, such as Figure 2 As shown, the steps to obtain the local free energy bonus R3 include:
[0061] Step S210: Obtain the local region affected by the new stack and obtain the sequence fragment of the local region;
[0062] In this context, the new stack refers to the local structural modules selected by the reinforcement learning agent in the current decision-making step and to be added to the RNA structure, such as a stem region continuous base pair unit. Based on the base range covered by the new stack on the RNA sequence, and a certain extended region around this range (usually 3-5 bases on each side), a local region is obtained. The base strings of this local region are extracted from the RNA sequence to obtain sequence fragments. For example, if the new stack directly covers base positions 5, 6, 7, 18, 19, and 20; and the local regions affected by the new stack are positions 2-10 and 15-24; the corresponding sequence fragments are the base strings extracted from positions 2-10 and 15-24 of the RNA sequence.
[0063] Step S220: Calculate the local free energy of the sequence fragment using a fast energy model;
[0064] Based on sequence fragments of local regions, a fast energy model (such as the local folding of Vienna RNA) is used to calculate the free energy of that region, obtaining the local free energy. It should be noted that this local free energy only evaluates the local region and does not consider the globally folded portion. Among them, ViennaRNA is a fast energy assessment method for RNA secondary structure based on thermodynamic parameterization and dynamic programming. Its core is a dual framework of minimum free energy (MFE) and partition function. With simplified ring-stack energy rules and efficient DP algorithm, it can achieve RNA structure energy calculation and folding prediction in the range of seconds to minutes while ensuring biologically meaningful accuracy. It is a classic toolchain in RNA bioinformatics.
[0065] Step S230: Calculate the stability of the new stack;
[0066] Stability is scored within a range of [0,1]. First, the proportion of base pairs conforming to the Watson-Crick (AU / UA, GC / CG) or Wobble (GU / UG) specifications in the stack is calculated and weighted at 0.4 for stability. Next, the proportion of CG bases in the stack region is calculated, and the stability score is increased by 0.3 / 0.2 / 0.1 based on the proportion range (0.6-0.8 is optimal, 0.5-0.9 is next best, and others are worse). Then, a length bonus is awarded based on the stack length: ≥4 stack lengths are increased by 0.2, ≥3 stack lengths by 0.1, and ≥2 stack lengths by 0.05. Finally, additional points are awarded for the first pairing type in the stack: 0.15 for GC / CG, 0.1 for AU / UA, and 0.05 for GU / UG. Ultimately, stability is limited to the [0,1] range; higher stability indicates stronger thermodynamic stability of the stack module.
[0067] Step S240: Obtain a local free energy bonus based on stability and local free energy.
[0068] Specifically, the local free energy reward R3 = -local_free_energy * 0.01 + stack_stability * 0.05. Here, local_free_energy is the local free energy, and stack_stability is the stability.
[0069] In this embodiment, the process of obtaining the pairing quantity reward R1 is as follows: The number of pairs in the new stack, base_pair_count, is counted, and the pairing quantity reward R1 = base_pair_count * 0.1. To encourage stable core formation, after obtaining the pairing quantity reward, the reward is further increased when the length of the new stack is greater than a preset length. For example, if the length of the new stack is greater than or equal to 3, the pairing quantity reward R1 is increased by 0.2.
[0070] The process of obtaining the above-mentioned structural logic reward R2 in this embodiment is as follows: First, it is determined whether the new stack conflicts with the existing structure. When a conflict occurs, the structural logic reward R2 = R2 - 1.5. The conflict determination method is as follows: if any base in the new stack pairs with other positions (not within the stack), a conflict is determined; if the new stack forms a pseudo-knot topology with the existing structure, a conflict is determined. Then, the structural logic reward R2 is changed accordingly based on whether the new stack is well nested in the existing structure: when a good nesting is formed, which contributes to the global structural stability, the structural logic reward R2 = R2 + 0.5; when an isolated or abrupt stack is formed, which may be detrimental to subsequent folding, the structural logic reward R2 = R2 - 0.2. Specifically, the nesting score determines whether a new stack is well nested within an existing structure. The steps for calculating the nesting score include: when the new stack is adjacent to and extends an existing stack, a high score is awarded, with a nesting score of 1.0; when the new stack is wrapped by an existing structure, a second nesting score is awarded based on the number of existing pairs it is wrapped by (e.g., 0.9 for multiple pairs and 0.7 for one pair); when the new stack conflicts with an existing structure (existing intersections or overlaps) or is poorly nested, a third nesting score of 0.1 is awarded; and when the new stack is completely isolated, a fourth nesting score of 0.5 is awarded. A nesting score greater than 0.7 indicates good nesting; a nesting score less than 0.3 indicates isolated or abrupt stacking. Finally, a continuity test is performed to check whether the new stack is spatially continuous with the previous action: the spatial distance between the new stack and the stack corresponding to the previous action of the reinforcement learning agent is calculated. If the spatial distance is ≤4 (indicating that the two are spatially close and can form continuous folds), the reward value R2 is increased by 0.3 to encourage the agent to select spatially adjacent stack modules and promote the formation of continuous and stable folded structures of RNA.
[0071] As described above, in this embodiment, the reward R1 for the number of pairs is supplemented with an additional reward for long stacks to prevent the agent from favoring scattered short pairs. The structural logic reward R2 is refined into three sub-items: conflict detection, nesting evaluation, and continuity reward. This constrains the structural logic from three dimensions: legality, rationality, and continuity, preventing illegal structures such as intersections / overlaps and guiding the agent to choose natural folding patterns that conform to nesting and continuous growth. The local free energy reward R3 is limited to local free energy, focusing on the energy state of local regions of the stack and flanks, rather than global energy. This captures local stability features that are easily overlooked by global energy, making the reward more consistent with the folding mechanism of RNA local nucleation and gradual expansion. Therefore, the overall reward signal is more consistent with the biophysical laws of RNA folding, while solving the problems of sparse rewards and ambiguous signals in reinforcement learning training. Ultimately, this achieves the goals of more accurate predicted structures, more stable model training, and folding paths that better match real biological processes.
[0072] The process of obtaining the supervised reward R4 in this embodiment is as follows: calculate the matching degree between the new stack and the target structure, and the supervised reward R4 = matching degree * 0.3. An additional reward of 0.2 can be given for cases where the target stack is a perfect match. Specifically, the matching degree between the new stack and the known target RNA secondary structures in the training data is obtained by quantifying the scores of pairing overlap, positional consistency, and topological matching degree, and weighting and accumulating these scores from 0 to 1.
[0073] To address the weighting of each term in the composite reward function, this invention innovatively introduces a learnable dynamic weight adjuster. This mechanism transforms the reward function from a static linear combination into an intelligent evaluation system capable of adaptively adjusting based on the folding stage and the current state context. Specifically, it can increase the weights of structural logic and local energy in the early stages of folding to ensure nucleation stability; balance the number of pairs and energy optimization in the middle stages of folding to accelerate structural extension; and reduce intermediate reward weights as the target approaches, focusing on global optimality.
[0074] Specifically, the dynamic weight adjuster in this embodiment is a lightweight neural network module that adjusts the weights of the composite reward function based on state codes formed by indicators such as folding progress, structural complexity, energy status, and topological health. The state code is a four-dimensional state code vector S=[s1,s2,s3,s4], where folding progress s1 = number of paired bases / total potential bases; and structural complexity s2 represents the number and complexity of loops, bulges, and multi-branch loops in the currently formed structure. ,in, This is the set of rings in the current structure (including loops, bulges, and multi-branch rings). For ring type weights, The cyclic complexity is represented by the energy state index s3, which indicates the trend of local free energy reward (R3) over the most recent steps. ,in, The most recent decision step, For the local free energy reward at step t, The mean value represents the trend of energy change. The topological health index s4 is the score of the most recent structural logical reward and punishment (R2), such as whether conflicts occur frequently. The dynamic weight adjuster can also include sequence context. Through an embedding layer, the features of the current sequence segment to be folded (such as GC content and conservatism) are also taken into consideration. For example, the GC content and conservatism score of the current sequence segment to be folded are converted into a 16-dimensional feature vector E through an embedding layer. Then, the state encoding vector S is concatenated with the embedding vector E and then input into the neural network module. The specific structure of the lightweight neural network module is not limited. This embodiment includes two fully connected layers: the first layer has 32 neurons and the activation function is ReLU; the second layer has the same number of neurons as the number of weight terms in the composite reward function (R1 / R2 / R3 / R4, a total of 4 terms, so the number of neurons is 4), and the output layer activation function is Softmax to ensure that the sum of the output weights is 1. The dynamic weight adjuster receives input features and outputs a weight vector W=[w1,w2,w3,w4] for each reward item. The final output of the composite reward function is R=w1×R1+w2×R2+w3×R3+w4×R4. During the inference phase, w4=0.
[0075] During training, the dynamic weight adjuster and the reinforcement learning master policy network (PPO algorithm) are trained together. The parameters of the dynamic weight adjuster and the master policy network are updated synchronously through gradient backpropagation of the PPO algorithm. The training batch size is 64 and the learning rate is 1e-4.
[0076] The trained dynamic weight regulator automatically increases the weights of R2 (structural logic) and R3 (local free energy) in the early stages. This is because random exploration is frequent at this point, leading to a high risk of topological conflicts, necessitating a priority on ensuring the physical rationality and stability of nucleation. In the mid-stage, when the structural framework is basically reasonable, the weight of R1 (pairing quantity / long stacking reward) is automatically increased to encourage rapid structural extension and consolidation, while maintaining attention to R3 to avoid forming high-energy unstable regions. In the late stage, when the structure is nearing completion, the weight of R1 is reduced (as pairing has become saturated), while the weights of R3 (local free energy) and R4 (supervised reward) are significantly increased for fine-tuning and global optimization to find the conformation with the lowest energy or closest to the target. Furthermore, if the reinforcement learning agent gets stuck in a local optimum (e.g., R3 shows no improvement for several consecutive steps), the dynamic weight regulator can dynamically fine-tune the weight allocation to break the deadlock and encourage further exploration.
[0077] This embodiment, after calculating the reward value according to the composite reward function, further includes: real-time counting of the number of consecutive steps in which the single-step reward value of the reinforcement learning agent is lower than a preset reward threshold; when the number of consecutive steps exceeds a first preset value, increasing the reward value by a preset coefficient, for example, when the number of consecutive steps exceeds 5, the reward value = reward value * 1.1; real-time counting of the total number of steps in which the reinforcement learning agent completes RNA secondary structure folding; when the total number of steps exceeds a second preset value, decreasing the reward value according to the difference between the total number of steps and the second preset value, for example, when the total number of steps exceeds 100, the reward value = reward value - 0.01 * (total number of steps - 100).
[0078] Experiments have shown that this embodiment demonstrates significant performance improvements and efficiency advantages in RNA secondary structure prediction and folding path deduction tasks, specifically in the following aspects:
[0079] Significantly improved prediction accuracy: On the Eterna 100 v1 benchmark set, this embodiment achieved an average accuracy of 67% in inference prediction of 100 RNA sequences, which is 12 percentage points higher than the current mainstream reinforcement learning method DRAG (55%), demonstrating stronger structural modeling and generalization capabilities.
[0080] Faster convergence: This embodiment integrates a pre-trained structural prediction model as prior guidance, significantly reducing the blind spots in the exploration phase of reinforcement learning. Experiments show that, to achieve the same prediction accuracy, this embodiment requires only about 75% of the training steps of DRAG, and the convergence process is more stable.
[0081] Training and inference efficiency optimization: By expanding the action space from single-base pairing to the selection of local structural modules, this embodiment can consider multiple pairing combinations simultaneously at each decision, thereby reducing ineffective exploration and shortening inference time. In actual testing, the time required to complete the folding path deduction of a medium-length RNA sequence was reduced by an average of 30%.
[0082] Enhanced path interpretability and intermediate state identification capability: This embodiment can not only output the final secondary structure, but also provide reasonable folding paths and key intermediate states, which helps to reveal the dynamic mechanism of RNA folding and provides a theoretical basis for drug target design based on folding dynamics.
[0083] In summary, this embodiment combines a deep learning pre-trained model with a reinforcement learning agent, making the action space of the reinforcement learning agent the local structural modules output by the deep learning pre-trained model. The agent performs the selection and pairing of these local structural modules, rather than selecting individual base pairs or pairing individual nucleotides. This ensures the decision-making process is highly consistent with biophysical laws and improves efficiency. By employing a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability, and dynamically adjusting the weights of each component, the reinforcement learning agent is guided to explore conformations and paths that conform to biophysical laws. This accurately captures the dynamic temporal characteristics of RNA folding, reproduces RNA folding paths, and improves generalization ability.
[0084] like Figure 3 As shown, based on the above-mentioned RNA secondary structure prediction and folding path deduction method, this embodiment of the invention discloses an RNA secondary structure prediction and folding path deduction system, comprising:
[0085] A reinforcement learning agent 610 is used to perform the screening and pairing of local structural modules in an action space, step by step outputting the prediction results of RNA secondary structure, and recording the selected local structural modules and pairing order at each step to obtain the folding path. The reinforcement learning agent adopts a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability. The local structural modules are obtained by feature extraction and pattern recognition of the RNA sequence to be predicted by a deep learning pre-trained model. The local structural modules contain at least 3 nucleotide pairs.
[0086] A learnable dynamic weight adjuster 620 is used to adjust the weights of the composite reward function according to the state encoding. The dynamic weight adjuster is a lightweight neural network module. The state encoding includes: folding progress index, structural complexity index, energy status index, and topology health index.
[0087] like Figure 4 As shown, an embodiment of the present invention discloses an electronic device, including a memory 401 storing executable program code and a processor 402 coupled to the memory 401;
[0088] The processor 402 calls the executable program code stored in the memory 401 to execute the RNA secondary structure prediction and folding path deduction methods described in the above embodiments.
[0089] This invention also discloses a computer-readable storage medium storing a computer program that causes a computer to execute the RNA secondary structure prediction and folding path deduction methods described in the above embodiments.
[0090] The purpose of the above embodiments is to reproduce and derive the technical solution of the present invention by way of example, and to fully describe the technical solution, purpose and effect of the present invention. The purpose is to enable the public to have a more thorough and comprehensive understanding of the disclosure of the present invention, and not to limit the scope of protection of the present invention.
[0091] The above embodiments are not an exhaustive list based on the present invention, and there may be many other embodiments not listed. Any substitutions and improvements made without departing from the concept of the present invention are within the protection scope of the present invention.
Claims
1. A method for predicting RNA secondary structure and deducing folding pathways, characterized in that, include: The RNA sequence to be predicted is input into a deep learning pre-trained model for feature extraction and pattern recognition to obtain several local structural modules composed of at least 3 nucleotide pairs; All the aforementioned local structural modules are used as the action space of the reinforcement learning agent. The reinforcement learning agent performs the screening and pairing actions of local structural modules, gradually outputs the prediction results of RNA secondary structure, and records the local structural modules selected at each step and the pairing order to obtain the folding path. The reinforcement learning agent adopts a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability, and adaptively adjusts the weights of the composite reward function according to the folding stage and the current state.
2. The method for predicting RNA secondary structure and deducing folding pathways as described in claim 1, characterized in that, A learnable dynamic weight adjuster is used to adjust the weights of the composite reward function according to the state code. The dynamic weight adjuster is a lightweight neural network module. The state code includes: folding progress index, structural complexity index, energy status index and topology health index.
3. The method for predicting RNA secondary structure and deducing folding pathways as described in claim 1, characterized in that, The composite reward function includes: pairing quantity reward, structural logic reward, and local free energy reward; the pairing quantity reward is obtained based on the number of newly formed pairs; the structural logic reward is obtained based on whether the new stack conflicts with the existing structure, whether the new stack is well nested in the existing structure, and whether the new stack is spatially continuous with the previous action; the local free energy reward is obtained based on the free energy of the local area affected by the new stack.
4. The method for predicting RNA secondary structure and deducing folding pathways as described in claim 3, characterized in that, Obtaining the local free energy reward includes: Obtain the local region affected by the new stack and obtain the sequence fragment of the local region; The local free energy of the sequence fragment was calculated using a fast energy model; Calculate the stability of the new stack; The local free energy reward is obtained based on the stability and the local free energy.
5. The method for predicting RNA secondary structure and deducing folding pathways as described in claim 3, characterized in that, After obtaining the pairing quantity reward, when the length of the new stack is greater than the preset length, the pairing quantity reward is increased, and / or, during the training phase, the composite reward function also includes a supervised reward, which is obtained based on the matching degree between the new stack and the target structure.
6. The method for predicting RNA secondary structure and deducing folding pathways as described in claim 3, characterized in that, Determining whether the new stack conflicts with the existing structure includes: If any base in the new stack pairs with another position, a conflict is determined. If the new stack forms a pseudo-junction topology with the existing structure, a conflict is determined. The nesting score is used to determine whether a new stack is well nested within an existing structure. The steps for calculating the nesting score include: When a new stack is adjacent to an existing stack and expands the existing stack, the nesting score is set to the first score. When a new stack is wrapped by an existing structure, the nesting score is set as the second score based on the number of existing pairs that are wrapped by it; When a new stack conflicts with an existing structure or is poorly nested, the nesting score is set to the third score. When the new stack is a completely isolated stack, the nesting score is set to the fourth score.
7. The method for predicting RNA secondary structure and deducing folding pathways as described in claim 1, characterized in that, After calculating the reward value based on the composite reward function, the following is also included: The number of consecutive steps in which the single-step reward value of the reinforcement learning agent is lower than a preset reward threshold is counted in real time. When the number of consecutive steps exceeds a first preset value, the reward value is increased according to a preset coefficient. The total number of steps taken by the reinforcement learning agent to complete the folding of RNA secondary structure is counted in real time. When the total number of steps exceeds a second preset value, the reward value is reduced based on the difference between the total number of steps and the second preset value.
8. A system for predicting RNA secondary structure and deducing folding pathways, characterized in that, include: A reinforcement learning agent is used to perform the screening and pairing of local structural modules in an action space, progressively outputting the prediction results of RNA secondary structure, and recording the selected local structural modules and pairing order at each step to obtain the folding path. The reinforcement learning agent adopts a composite reward function that includes rewarding structural module formation, penalizing topological conflicts, and evaluating local stability. The local structural modules are obtained by feature extraction and pattern recognition of the RNA sequence to be predicted by a deep learning pre-trained model. The local structural modules contain at least 3 nucleotide pairs. A learnable dynamic weight adjuster is used to adjust the weights of the composite reward function according to the state encoding. The dynamic weight adjuster is a lightweight neural network module. The state encoding includes: folding progress index, structural complexity index, energy status index, and topology health index.
9. An electronic device, characterized in that, It includes a memory storing executable program code and a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the RNA secondary structure prediction and folding path deduction method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program causes a computer to execute the RNA secondary structure prediction and folding path deduction method according to any one of claims 1-7.