A method for nucleic acid sequence generation based on reinforcement learning and context-free grammar

Abstract

The present invention discloses a method for generating nucleic acid sequences based on reinforcement learning and context-free grammar. The structure of the nucleic acid sequence is described by a context-free grammar tree. An intelligent agent uses a neural network based on a self-attention mechanism to encode and process the grammar tree of the nucleic acid sequence, selects non-terminal symbols and production rules, and transmits them to the environment as actions. The environment updates the grammar tree of the nucleic acid sequence based on the actions of the intelligent agent and generates a new environment. A reward function evaluates the actions of the intelligent agent based on the current environmental information and returns a reward value to guide the next action of the intelligent agent. The above operation is repeated until the environment determines that the current nucleic acid sequence has been generated. This method can solve the problem of random libraries containing a large number of useless sequences in SELEX screening experiments.

Description

Technical Field

[0001] The present invention relates to the technical field of nucleic acid sequence generation, and in particular to a nucleic acid sequence generation method based on reinforcement learning and context-free grammar. Background Art

[0002] The basic idea behind using a formal grammar to describe a formal language is to generate a set of strings by continuously applying some production rules starting from a specific initial symbol. In computer science, a formal grammar G = (V, Σ, P, S) is called a context-free grammar if its production rules all take the form A->α, where A∈V and α∈(V∪Σ)*. Context-free grammars are named "context-free" because the character A can always be freely replaced by the string α, regardless of the context in which the character A appears. If a formal language is generated by a context-free grammar, then it is said to be context-free.

[0003] A context-free grammar G is a quadruple (V, Σ, S, R), where V is a finite set of "non-terminal" symbols or variables. They represent different types of phrases or clauses in a sentence, and in this method, they represent the higher-level structure of nucleic acids. Σ is a finite set of "terminals" that have no intersection with V. They constitute the actual content of the sentence and in this method represent different bases (A, T, C, G, U). S is the start variable, which must be an element of V and is used to represent the entire sentence, and in this method, it represents the entire nucleic acid sequence. R is the rule or production of the grammar, which represents the relationship from non-terminal symbols to non-terminal symbols or terminals, and in this method, it represents the relationship between the higher-level structure of nucleic acids and base sequences.

[0004] Context-free grammars are important because they are expressive enough to represent the syntax of most programming languages. Therefore, it is feasible to describe nucleic acid sequences using context-free grammars. Generally, context-free grammars are used to parse sentences from the bottom up, forming a grammar tree for the sentence. However, in this method, context-free grammars are used to generate nucleic acid sequences from the top down. The top-down construction of the grammar tree is the process of sequence generation, and the addition of specific non-terminal symbols in the grammar tree adds specific high-level structure to the nucleic acid sequence.

[0005] Reinforcement learning (RL) is a field in machine learning that emphasizes how to act based on the environment to maximize expected benefits. It explores how an intelligent agent can maximize its rewards in a complex and uncertain environment. By perceiving the state of the environment and its responses to actions, it guides the agent to take better actions, thereby maximizing its benefits. Reinforcement learning is the third fundamental machine learning method, alongside supervised and unsupervised learning. Unlike supervised learning, reinforcement learning does not require labeled input-output pairs or precise corrections for suboptimal solutions. Its focus is on finding a balance between exploration (of unknown areas) and exploitation (of existing knowledge).

[0006] The self-attention mechanism is a specialized attention mechanism that bypasses the recurrent neural networks used in the attention mechanism. Instead, it aims to make the neural network notice the correlations between different parts of the input sequence. In the self-attention mechanism, each input vector is correlated with all other input vectors to produce the output vector corresponding to that input vector.

[0007] The concept of aptamer was first proposed in 1990 and is defined as a synthetic single-stranded nucleic acid short chain. As a ligand, it can bind to a variety of targets, including metal ions, compounds, proteins, cells and whole microorganisms. Compared with traditional antibodies, aptamers have unique advantages, such as long shelf life, small batch-to-batch differences, low or no immunogenicity, and the ability to flexibly combine chemical modifications to enhance stability and target affinity. Therefore, in the past few decades, aptamers have quickly found a way into various applications such as therapy, drug delivery, diagnosis, functional genomics and biosensing.

[0008] Aptamers are usually developed in vitro, and the development process is called systematic evolution of ligands by exponential enrichment (SELEX). SELEX is an iterative process, and each round of iteration usually includes the steps of library design, target preparation, counter-screening, positive screening, separation of bound / unbound sequences, reverse transcription (for RNA aptamers), PCR amplification, single-stranded DNA separation (for DNA aptamers) and transcription (for RNA aptamers). The specific approach can be simplified as follows: the target is added to a library containing at least 10^12 unique sequences. After a period of time, the sequences that are not bound to the target are washed away, and the bound sequences are retained and amplified by PCR to form an enrichment pool. The target is added to the enrichment pool again, and the above process is repeated many times. Finally, the sequence with the most counts after multiple rounds of evolution is selected as the candidate aptamer, which will be further validated for affinity after chemical synthesis.

[0009] Nucleic acid secondary structure (NAS) refers to base pair interactions within a single nucleic acid polymer or between two nucleic acid polymers. Double-stranded nucleic acids primarily exist as fully base-paired double helices, while single-stranded nucleic acids can form complex base-pairing interactions due to the increased hydrogen bonding capacity derived from the additional hydroxyl groups in the ribose sugar. Single-stranded nucleic acid secondary structures include stem-loops, pseudoknots, and G-quadruplexes. Nucleic acid sequences with a certain secondary structure tend to form more complex structures in solution and are therefore more likely to have higher affinity.

[0010] Currently, SELEX screening experiments use random sequences as the initial library for screening in the hope of obtaining aptamer sequences with high affinity. Random sequences refer to sequences where the entire nucleic acid sequence contains random bases. Therefore, the random initial library contains a large number of useless sequences. Summary of the Invention

[0011] To overcome at least one of the above-mentioned drawbacks of the prior art, the present invention provides a method for generating nucleic acid sequences based on reinforcement learning and context-free grammar, which can solve the problem of a large number of useless sequences in a random initial library in a SELEX screening experiment.

[0012] The technical solution adopted by the present invention to solve the problem is:

[0013] A method for generating nucleic acid sequences based on reinforcement learning and context-free grammar, comprising the following steps:

[0014] S1. Provide the required nucleic acid secondary structure and base number constraints, and the initial context-free grammar tree to form the initial environment;

[0015] S2, transmit the environment to the agent as the content that the agent can observe;

[0016] S3, the agent uses a neural network to encode and process the observed environment;

[0017] S4. The agent selects a non-terminal symbol from the list of optional non-terminal symbols in the context-free grammar tree of the current environment according to the encoding and processing in step S3, and then selects a production rule corresponding to the non-terminal symbol. The non-terminal symbol and the corresponding production rule are the agent's action;

[0018] S5, transmit the action of the agent in step S4 to the environment;

[0019] S6, the environment updates the grammar tree of the nucleic acid sequence according to the actions of the agent and generates a new environment;

[0020] S7, the reward function evaluates the agent's actions based on the current environment information and obtains a reward value, which is used to guide the agent's update using the reinforcement learning algorithm;

[0021] S8, repeating steps S2 to S7 until the environment determines that the current nucleic acid sequence has been generated;

[0022] S9. After the model process ends, the generated nucleic acid sequence and its corresponding grammar tree are output.

[0023] By adopting this scheme, the method provides a novel approach for generating initial libraries for SELEX experiments. Random libraries contain a vast number of nucleic acid sequences, most of which are useless. However, the sequences in the libraries generated using this method are all sequences with specific secondary structures or properties. This reduces noise compared to random libraries, reduces the number of nucleic acid sequences that need to be screened, and reduces the difficulty of SELEX experiments, thereby improving their success rate and efficiency.

[0024] Furthermore, the non-terminal symbol set, the terminal symbol set and the production rules of the non-terminal symbols of the context-free syntax tree are defined for use in generating subsequent sequences.

[0025] By adopting the above scheme, the non-terminal symbol set and terminal symbol set of the context-free syntax tree and the production rules of the non-terminal symbols, once determined, can be used continuously without subsequent modification. By defining the generation strategy of nucleic acid sequences through production rules, this method introduces a more standardized, controllable and flexible design mechanism in nucleic acid sequence generation. This scheme makes the generation process more regular and predictable. Users can achieve fine-grained control over the generation process by customizing production rules, thereby improving the diversity and adaptability of the generated nucleic acid sequences. At the same time, by directly connecting structure and sequence, non-terminal symbols can represent structure and directly participate in the generation of nucleic acid sequences, making the generated sequences more in line with the requirements of nucleic acid secondary structure. This design scheme improves the flexibility and customizability of the generation process, making the method better adapted to different experimental needs and research purposes.

[0026] Furthermore, before applying this method to generate nucleic acid sequences, the agent's neural network needs to be trained. Training involves using a reinforcement learning algorithm and deep learning techniques to update the parameters of the agent's neural network based on the value of the reward function calculated in step S7, thereby obtaining a better agent. When applying this method to generate nucleic acid sequences, there is no need to use the value of the reward function to guide the agent's updates.

[0027] Furthermore, the encoding and processing described in step S3 specifically includes the following steps:

[0028] S31, passing the current context-free grammar tree and the constraint information on the nucleic acid secondary structure and base ratio to the neural network as input;

[0029] S32. Convert the context-free grammar tree from a tree structure to a sequence structure, so as to meet the input form requirements of the neural network;

[0030] S33. Use a neural network based on the self-attention mechanism to encode the grammar tree of the input sequence. The self-attention mechanism allows the network to focus on information at different positions in the input to better capture long-range dependencies in the sequence.

[0031] S34. After processing, the neural network generates an encoding representation of each terminal and non-terminal symbol in the sequence;

[0032] S35. The output encoding representation is used as the output of the intelligent agent for subsequent decision-making steps.

[0033] By adopting this approach, the self-attention mechanism allows the neural network to pay more attention to other non-terminal symbols related to the current non-terminal when generating the encoded representation, while also helping to better capture the grammatical and structural information in nucleic acid sequence generation. The encoded representation contains key feature information from the input, which helps the agent select appropriate non-terminal symbols to update the grammar tree in subsequent decisions to achieve nucleic acid sequence generation. Furthermore, through the processing of the self-attention mechanism, the neural network can more comprehensively understand the context of the current non-terminal list, thereby more accurately guiding the selection of subsequent actions. Furthermore, the multi-head attention mechanism can also be used to improve the model's expressive power.

[0034] Furthermore, the optional non-terminal symbol list in step S4 is a list of all non-terminal symbols that are leaf nodes in the context-free grammar tree. These non-terminal symbols need to grow new terminals or non-terminal symbol elements on the grammar tree according to production rules, thereby completing the generation of the nucleic acid sequence.

[0035] Furthermore, the updating of the grammar tree of the nucleic acid sequence in step S6 specifically includes the following steps:

[0036] S61. Find the node corresponding to the non-terminal symbol in the grammar tree according to the selected non-terminal symbol and production rule;

[0037] S62: Based on the production, take the node found in step S61 as the parent node and add the terminal or non-terminal elements generated by the production to the child nodes of the parent node to update the grammar tree;

[0038] S63. Add the new non-terminal symbol generated in step S62 to the non-terminal symbol list of the grammar tree.

[0039] By adopting the above scheme, a method for generating nucleic acid sequences using a context-free grammar tree was designed. During the grammar tree update process, adding specific types of non-terminal symbols can add specific secondary structures or bases to the nucleic acid sequence. This method of generating nucleic acid sequences allows for the convenient addition of constraints. By using reinforcement learning techniques to generate nucleic acid sequences using a context-free grammar tree, nucleic acid libraries with specified properties can be rapidly designed.

[0040] In summary, the method for generating nucleic acid sequences based on reinforcement learning and context-free grammar provided by the present invention has the following technical effects:

[0041] This method provides a novel approach for generating initial libraries for SELEX experiments. Random libraries contain a vast number of nucleic acid sequences, most of which are useless. This method generates libraries containing sequences with specific secondary structures or properties, reducing noise compared to random libraries and the number of nucleic acid sequences that need to be screened. This reduces the difficulty of SELEX experiments, thereby improving their success rate and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a reference diagram of the entire process of generating nucleic acid sequences according to the present invention;

[0043] Figure 2 Schematic diagram of the intelligent agent workflow of the present invention;

[0044] Figure 3 This is a schematic diagram of the working process of step S2 of the present invention;

[0045] Figure 4 This is a schematic diagram of the steps S3-S4 of the present invention;

[0046] Figure 5 This is a working diagram of steps S5-S7 of the present invention. DETAILED DESCRIPTION

[0047] For better understanding and implementation, the technical solutions in the embodiments of the present invention will be clearly and completely described and discussed below in conjunction with the drawings of the present invention. Obviously, what is described here is only a part of the examples of the present invention, not all the examples. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0048] To facilitate understanding of the embodiments of the present invention, the following will be further explained with reference to specific embodiments as examples in conjunction with the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0050] The present invention discloses a method for generating nucleic acid sequences based on reinforcement learning and context-free grammar, comprising the following steps:

[0051] S1. Provide the required nucleic acid secondary structure and base number constraints, and the initial context-free grammar tree to form the initial environment;

[0052] S2, transmit the environment to the agent as the content that the agent can observe;

[0053] S3, the agent uses a neural network to encode and process the observed environment;

[0054] S4. The agent selects a non-terminal symbol from the list of optional non-terminal symbols in the context-free grammar tree of the current environment according to the encoding and processing in step S3, and then selects a production rule corresponding to the non-terminal symbol. The non-terminal symbol and the corresponding production rule are the agent's action;

[0055] S5, transmit the action of the agent in step S4 to the environment;

[0056] S6, the environment updates the grammar tree of the nucleic acid sequence according to the actions of the agent and generates a new environment;

[0057] S7, the reward function evaluates the agent's actions based on the current environment information and obtains a reward value, which is used to guide the agent's update using the reinforcement learning algorithm;

[0058] S8, repeating steps S2 to S7 until the environment determines that the current nucleic acid sequence has been generated;

[0059] S9. After the model process ends, the generated nucleic acid sequence and its corresponding grammar tree are output.

[0060] This method provides a novel approach for generating initial libraries for SELEX experiments. Random libraries contain a vast number of nucleic acid sequences, most of which are useless. This method generates libraries containing sequences with specific secondary structures or properties, reducing noise compared to random libraries and the number of nucleic acid sequences that need to be screened. This reduces the difficulty of SELEX experiments, thereby improving their success rate and efficiency.

[0061] For ease of understanding, the following will be explained with one of the examples. For details, refer to Figure 1 As shown, a total of 10 production formulas are used. It should be noted that in order to distinguish non-terminal symbols from nodes expanded after using the production formula, Figure 1-5 The non-terminal symbols are outlined in a thick black border. For example, Figure 1 The s in the a, Figure 1 The item and s in b; Figure 1-5 The nodes with dark background are expanded after adopting the generative formula, for example, Figure 1 The item and s in b, Figure 1 The a, s, and u in the d.

[0062] Among the 10 productions, production No. 1 is: s->item s;

[0063] Production No. 2 is: s->item;

[0064] Production No. 3 is: item->n;

[0065] Production No. 4 is: item->pai r;

[0066] Production No. 5 is: n->A;

[0067] Production No. 6 is: n->U;

[0068] Production No. 7 is: n->C;

[0069] Production No. 8 is: n->G;

[0070] Production No. 9 is: pai r->A s U;

[0071] Production No. 10 is: pai r->U s A

[0072] Figure 1 Aj is the whole process of generating nucleic acid sequences using the above method, specifically:

[0073] exist Figure 1 The a corresponds to: the initial non-terminal symbol s;

[0074] exist Figure 1 The corresponding b is: select non-terminal symbol s and production No. 1, and expand the grammar tree;

[0075] exist Figure 1 The corresponding c is: select the non-terminal symbol item, and the production rule No. 4, and expand the grammar tree;

[0076] exist Figure 1 The corresponding d is: select the non-terminal symbol pai r, and the production rule No. 9, and expand the grammar tree;

[0077] exist Figure 1 The corresponding e is: using non-terminal symbol s and production No. 2 to expand the grammar tree;

[0078] exist Figure 1 The corresponding f is: using the non-terminal symbol item and production No. 3 to expand the grammar tree;

[0079] exist Figure 1 The corresponding g is: using non-terminal symbol n and production rule 8 to expand the grammar tree;

[0080] exist Figure 1 The corresponding h is: using non-terminal symbol s and production No. 2 to expand the grammar tree;

[0081] exist Figure 1 The i corresponds to: using the non-terminal symbol item and production No. 3 to expand the grammar tree;

[0082] exist Figure 1 The corresponding j is: using non-terminal symbol n and production rule 8 to expand the grammar tree;

[0083] exist Figure 1 When there are no candidate nonterminal symbols for j, the grammar tree expansion ends. Based on the terminal symbols of the leaf nodes, the final sequence is generated as AGUG. It should be noted that the above expansion process, nonterminal symbols, terminals, and productions are simplified for ease of understanding. In actual applications, they can be set according to specific needs.

[0084] Regarding the workflow of the agent described in step S2, refer to Figure 2 As shown in the figure, after receiving the content of the environment (the grammar tree of the nucleic acid sequence and the constraints on the nucleic acid properties), the agent uses the feature network to encode and process this content to obtain features. These features are then fed into the non-terminal selection network, which selects a candidate non-terminal and outputs it. The production selection network then selects a production from the available options based on the features and the selected non-terminal and outputs it. The selected non-terminal and production together constitute the agent's action. Furthermore, the features are fed into the value network to obtain an estimated reward value, which is used to train the agent's model.

[0085] Regarding the above S2 step, please refer to Figure 3 As shown, the context-free grammar tree of the environment contains two non-terminal symbols item and s generated by the non-terminal symbol s according to the production rule s->item s.

[0086] Regarding the above steps S3 and S4, please refer to Figure 4As shown in the figure, after encoding and processing the environment, the agent selects the non-terminal symbol item from the optional non-terminal symbols item and s, and then selects item->pai r from the optional two productions item->n and item->pai r. The selected non-terminal symbol item and the corresponding production item->pai r are the actions generated by the agent.

[0087] Regarding the above steps S5, S6 and S7, please refer to Figure 5 As shown in the figure, based on the agent's actions (non-terminal symbol item and production item->pai r), the grammar tree is updated, expanding the pai r node from the item node in the original grammar tree. At this point, the grammar tree's list of optional non-terminal symbols is pa ir and s. The reward function evaluates the agent's actions based on environmental information and provides a reward value to guide the agent's updates.

[0088] The non-terminal symbol set, terminal symbol set and production rules of the context-free syntax tree are defined for the generation of subsequent sequences.

[0089] Once the non-terminal symbol set and terminal symbol set of the context-free syntax tree and the production rules for non-terminal symbols are determined, they can be used continuously without subsequent modification. By defining the generation strategy of nucleic acid sequences through production rules, this method introduces a more standardized, controllable and flexible design mechanism in nucleic acid sequence generation. This solution makes the generation process more regular and predictable. Users can achieve fine-grained control over the generation process by customizing production rules, thereby improving the diversity and adaptability of the generated nucleic acid sequences. At the same time, by directly connecting structure and sequence, non-terminal symbols can represent structure and directly participate in the generation of nucleic acid sequences, making the generated sequences more in line with the requirements of nucleic acid secondary structure. This design scheme improves the flexibility and customizability of the generation process, making the method better adapted to different experimental needs and research purposes.

[0090] Before applying this method to generate nucleic acid sequences, the agent's neural network must be trained. Training involves using a reinforcement learning algorithm and deep learning techniques to update the parameters of the agent's neural network based on the value of the reward function calculated in step S7, thereby improving the agent. When applying this method to generate nucleic acid sequences, the reward function value does not need to be used to guide the agent's updates.

[0091] Specifically, the optional non-terminal symbol list described in step S4 is a list of all non-terminal symbols that are leaf nodes in the context-free grammar tree. These non-terminal symbols need to grow new terminals or non-terminal symbol elements on the grammar tree according to production rules to complete the generation of the nucleic acid sequence.

[0092] Specifically, the encoding and processing described in step S3 specifically includes the following steps:

[0093] S31, passing the current context-free grammar tree and the constraint information on the nucleic acid secondary structure and base ratio to the neural network as input;

[0094] S32. Convert the context-free grammar tree from a tree structure to a sequence structure, so as to meet the input form requirements of the neural network;

[0095] S33. Use a neural network based on the self-attention mechanism to encode the grammar tree of the input sequence. The self-attention mechanism allows the network to focus on information at different positions in the input to better capture long-range dependencies in the sequence.

[0096] S34. After processing, the neural network generates an encoding representation of each terminal and non-terminal symbol in the sequence;

[0097] S35. The output encoding representation is used as the output of the intelligent agent for subsequent decision-making steps.

[0098] The self-attention mechanism allows the neural network to pay more attention to other non-terminal symbols related to the current non-terminal when generating the encoded representation, while also helping to better capture the grammatical and structural information in nucleic acid sequence generation. The encoded representation contains key feature information of the input, which helps the agent select appropriate non-terminal symbols to update the grammar tree in subsequent decisions to achieve nucleic acid sequence generation. Furthermore, through the processing of the self-attention mechanism, the neural network can more comprehensively understand the context of the current non-terminal list, thereby more accurately guiding the selection of subsequent actions. Furthermore, the multi-head attention mechanism can also be used to improve the model's expressive power.

[0099] Furthermore, the optional non-terminal symbol list in step S4 is a list of all non-terminal symbols that are leaf nodes in the context-free grammar tree. These non-terminal symbols need to grow new terminals or non-terminal symbol elements on the grammar tree according to production rules, thereby completing the generation of the nucleic acid sequence.

[0100] Specifically, the updating of the grammar tree of the nucleic acid sequence in step S6 specifically includes the following steps:

[0101] S61. Find the node corresponding to the non-terminal symbol in the grammar tree according to the selected non-terminal symbol and production rule;

[0102] S62: Based on the production, take the node found in step S61 as the parent node and add the terminal or non-terminal elements generated by the production to the child nodes of the parent node to update the grammar tree;

[0103] S63. Add the new non-terminal symbol generated in step S62 to the non-terminal symbol list of the grammar tree.

[0104] This method uses a context-free grammar tree to generate nucleic acid sequences. By adding specific types of non-terminal symbols during the tree update process, specific secondary structures or bases can be added to the nucleic acid sequence. This method allows for the convenient addition of constraints to nucleic acid sequences. Using reinforcement learning techniques, nucleic acid sequences generated from a context-free grammar tree can be rapidly designed to design nucleic acid libraries with specified properties.

[0105] In summary, the method for generating nucleic acid sequences based on reinforcement learning and context-free grammar provided by the present invention has the following technical effects:

[0106] This method provides a novel approach for generating initial libraries for SELEX experiments. Random libraries contain a vast number of nucleic acid sequences, most of which are useless. This method generates libraries containing sequences with specific secondary structures or properties, reducing noise compared to random libraries and the number of nucleic acid sequences that need to be screened. This reduces the difficulty of SELEX experiments, thereby improving their success rate and efficiency.

[0107] Within the framework of reinforcement learning, after training, the model can complete the following tasks: given a constraint combination (that is, constraints on the secondary structure and base distribution of nucleic acids), the algorithm can randomly generate nucleic acid sequences that meet the constraints, and can provide statistical information of the sequence (number of bases, number of pairs, etc.), and can also visually represent the structure of the sequence.

[0108] Example of generating a nucleic acid sequence:

[0109] UUGGGAGGGGUGGGUUUUAUGGGAAUAUUAUUAUAUGCCAU

[0110] Examples of secondary structure of sequences:

[0111] ..[||*|||*|||**||].(..(...........))

[0112] In the examples of secondary structure, "." represents a base that does not form a secondary structure; "()" represents a base that forms a pair, such as the pairing of base A and base U, and the pairing of base C and base G; "...(....)...." represents a base sequence that forms a stem-loop, in which "()" represents a base that forms a pair; "[||*|||*|||**||]" represents a base sequence that forms a G-quartet (a secondary structure of nucleic acid), in which "[" and "]" represent the beginning and end of the G-quartet, which are base G, "|" also represents base G of nucleic acid, and "*" represents any type of base in the G-quartet.

[0113] Example of secondary structure and base information of a sequence:

[0114] 2 paired structures, 1 G-quartet structure, 9 adenines, 16 uracils, 2 cytosines, and 13 guanines.

[0115] The technical means disclosed in the solutions of the present invention are not limited to those disclosed in the above-mentioned embodiments, but also include technical solutions composed of any combination of the above-mentioned technical features. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also considered to be within the scope of protection of the present invention.

[0116] The technical means disclosed in the solutions of the present invention are not limited to those disclosed in the above-mentioned embodiments, but also include technical solutions composed of any combination of the above-mentioned technical features. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for generating nucleic acid sequences based on reinforcement learning and context-free grammar, characterized in that: The following steps are involved: S1. Provide the required nucleic acid secondary structure and base number constraints, and the initial context-free grammar tree to form the initial environment; S2, transmit the environment to the agent as the content that the agent can observe; S3, the agent uses a neural network to encode and process the observed environment; S4. The agent selects a non-terminal symbol from the list of optional non-terminal symbols in the context-free grammar tree of the current environment according to the encoding and processing in step S3, and then selects a production rule corresponding to the non-terminal symbol. The non-terminal symbol and the corresponding production rule are the agent's action; S5, transmit the action of the agent in step S4 to the environment; S6, the environment updates the grammar tree of the nucleic acid sequence according to the actions of the agent and generates a new environment; S7, the reward function evaluates the agent's actions based on the current environment information and obtains a reward value, which is used to guide the agent's update using the reinforcement learning algorithm; S8, repeating steps S2 to S7 until the environment determines that the current nucleic acid sequence has been generated; S9. After the model process ends, the generated nucleic acid sequence and its corresponding grammar tree are output.

2. A method for generating nucleic acid sequences based on reinforcement learning and context-free grammar according to claim 1, characterized in that: The non-terminal symbol set, terminal symbol set and production rules of the context-free grammar tree are defined for the generation of subsequent sequences.

3. The method for generating nucleic acid sequences based on reinforcement learning and context-free grammar according to claim 1, characterized in that: Before applying this method to generate nucleic acid sequences, the agent’s neural network needs to be trained; Training refers to using a reinforcement learning algorithm and deep learning technology to update the parameters of the agent's neural network through the value of the reward function calculated in step S7, thereby obtaining a better agent; when applying this method to generate nucleic acid sequences, there is no need to use the value of the reward function to guide the update of the agent.

4. The method for generating nucleic acid sequences based on reinforcement learning and context-free grammar according to claim 1, wherein: The encoding and processing described in step S3 specifically includes the following steps: S31, passing the current context-free grammar tree and the constraint information on the nucleic acid secondary structure and base ratio to the neural network as input; S32. Convert the context-free grammar tree from a tree structure to a sequence structure, so as to meet the input form requirements of the neural network; S33. Use a neural network based on the self-attention mechanism to encode the grammar tree of the input sequence. The self-attention mechanism allows the network to focus on information at different positions in the input to better capture long-range dependencies in the sequence. S34. After processing, the neural network generates an encoding representation of each terminal and non-terminal symbol in the sequence; S35. The output encoding representation is used as the output of the intelligent agent for subsequent decision-making steps.

5. The method for generating nucleic acid sequences based on reinforcement learning and context-free grammar according to claim 1, characterized in that: The optional non-terminal symbol list described in step S4 is a list formed by all non-terminal symbols that serve as leaf nodes in the context-free grammar tree; these non-terminal symbols need to grow new terminal symbols or non-terminal symbol elements on the grammar tree according to the production rules, thereby completing the generation of the nucleic acid sequence.

6. The method for generating nucleic acid sequences based on reinforcement learning and context-free grammar according to claim 1, wherein: The updating of the grammar tree of the nucleic acid sequence in step S6 specifically includes the following steps: S61. Find the node corresponding to the non-terminal symbol in the grammar tree according to the selected non-terminal symbol and production rule; S62: Based on the production, take the node found in step S61 as the parent node and add the terminal or non-terminal elements generated by the production to the child nodes of the parent node to update the grammar tree; S63. Add the new non-terminal symbol generated in step S62 to the non-terminal symbol list of the grammar tree.

Images