Reverse biosynthesis prediction method based on progressive training and reinforcement learning
By employing progressive training and reinforcement learning methods, and utilizing a large-scale unlabeled molecular corpus and enzyme-catalyzed reaction dataset, the model parameters were optimized, solving the problem of scarce enzyme-catalyzed reaction data and achieving accurate prediction of the reverse biosynthesis of target products.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-25
- Publication Date
- 2026-03-20
AI Technical Summary
Existing technologies cannot effectively resolve the contradiction between the scarcity of enzyme-catalyzed reaction data and the need for models to learn complex biochemical knowledge, resulting in insufficient accuracy and generalization ability in reverse biosynthesis prediction.
We employ progressive training and reinforcement learning methods, using a large-scale unlabeled molecular corpus for pre-training, and progressive training and reinforcement learning on chemical reaction datasets and enzyme-catalyzed reaction datasets to optimize model parameters and generate reverse biosynthetic pathways containing predicted reactants and their enzyme numbers.
Without relying on a large amount of enzyme-catalyzed reaction data, the model achieves accurate prediction of the reactants and catalytic enzymes of the target product, thus improving the generalization performance and prediction accuracy of the model in biosynthetic tasks.
Smart Images
Figure CN121709055A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of biosynthesis, and in particular to a retro-biosynthesis prediction method based on progressive training and reinforcement learning. BACKGROUND
[0002] Retro-biosynthesis refers to a process of predicting precursor molecules and required enzymes of a target compound under the constraint of enzyme-catalyzed reactions, which is crucial for green chemistry and sustainable manufacturing. Enzymatic reactions have inherent complexity, often involving large biological molecules and significant structural changes. Annotated enzymatic reaction data is extremely scarce, and the largest enzymatic reaction dataset (about 60,000 reactions) is much smaller than the standard chemical reaction dataset (about 1 million reactions).
[0003] In related technologies, the methods of retro-biosynthesis mainly include template-based, semi-template-based and template-free methods. The template-based method relies on rules extracted from known reactions, but the size of the enzyme reaction database is much smaller than the general chemical reaction library, resulting in a serious lack of template coverage, and it is difficult to discover novel reaction paths and has poor generalization ability. Template-free methods are usually formulated as sequence-to-sequence learning tasks, which have greater flexibility and generalization ability. However, when directly applied to biological catalytic reactions, the performance of these methods will decrease significantly due to the above-mentioned data scarcity and reaction complexity. Training the model directly on limited enzymatic reaction data can easily lead to overfitting and make it difficult to learn robust chemical principles. Existing multi-task learning methods (training chemical and enzymatic reactions simultaneously) often cause the model to pay attention to scattered things, making it difficult to accurately grasp the subtle differences of enzymatic reactions. In addition, although traditional supervised learning can minimize Token-level loss, it cannot guarantee the chemical reasonableness or correctness of the entire reaction sequence generated. The prediction result may have a very small Token-level error, but it represents an invalid molecule or an incorrect reaction.
[0004] In summary, the related technology has failed to effectively solve the contradiction between the scarcity of enzymatic reaction data and the need for the model to learn complex biological chemistry knowledge, and there is a need for a method that can accurately predict the biological precursors (reactants) and catalytic enzymes of the target product without relying on a large amount of enzymatic reaction data. SUMMARY
[0005] The purpose of the present application is to provide a retro-biosynthesis prediction method based on progressive training and reinforcement learning, which can accurately predict the reactants and catalytic enzymes of the target product without relying on a large amount of enzymatic reaction data.
[0006] To achieve the above-mentioned purpose, the present application provides the following solutions: In a first aspect, the application provides a method for predicting reverse biosynthesis based on progressive training and reinforcement learning, comprising: obtaining reaction data and preprocessing to obtain an enhanced data set, the enhanced data set comprising a chemical reaction data set and an enzymatic reaction data set; obtaining a large-scale unlabeled molecular corpus, and pre-training a to-be-trained model according to the molecular corpus to obtain an initial model that masters grammatical rules and molecular structure rules; performing supervised reverse synthesis training and reverse biosynthesis fine-tuning on the initial model according to the chemical reaction data set and the enzymatic reaction data set in sequence to obtain a strategy model; post-training the strategy model according to a group relative strategy optimization reinforcement learning method, and optimizing the parameters of the strategy model by optimizing a composite reward function to obtain a final model; and predicting an input target product according to the final model, and outputting at least one reverse biosynthesis path containing predicted reactants and corresponding enzyme numbers.
[0007] Exemplarily, the reaction data is preprocessed to obtain an enhanced data set, comprising: selecting different atoms as root atoms for product molecules to generate normalized strings; obtaining atom mapping information based on enzymatic reactions or chemical reactions; traversing the normalized strings according to the atom mapping information to generate strings of corresponding reactants, thereby generating a plurality of enhanced samples to obtain the enhanced data set.
[0008] Exemplarily, the to-be-trained model is pre-trained according to the molecular corpus, comprising: performing mask language model training on the to-be-trained model according to a large-scale unlabeled molecular corpus, wherein the to-be-trained model is a sequence-to-sequence model; the mask language model training comprises: selecting a predetermined proportion of characters in an input sequence for mask processing, and predicting the original characters of the masked positions according to the decoder of the to-be-trained model based on the sequence after mask processing.
[0009] Exemplarily, the initial model is subjected to supervised reverse synthesis training, comprising: extracting the string of the product in the chemical reaction data set as the input of the initial model; taking the reactant string corresponding to the product in the chemical reaction data set as the training target, optimizing the model parameters of the initial model, making the initial model learn the chemical transformation mapping from the product to the reactant, and obtaining the general chemical reverse synthesis prediction ability.
[0010] Exemplarily, the reverse biosynthesis fine-tuning is performed to obtain a strategy model, specifically comprising: taking a product string of each reaction in the enzyme reaction data set as input data based on a model for obtaining general chemical reverse synthesis prediction ability; taking a sequence formed by a corresponding reactant string and an EC number of an enzyme in the enzyme reaction data set as a training target, the training target being constructed to generate a complete reactant string first and then generate the EC number of the enzyme.
[0011] Exemplarily, the strategy model is post-trained according to a group relative strategy optimization reinforcement learning method, and parameters of the strategy model are optimized by optimizing a composite reward function to obtain a final model, comprising: generating a group of candidate prediction sequences by using the strategy model through bundle search based on a target product in the training data; calculating a reward value of each candidate prediction sequence through the composite reward function; and updating the parameters of the strategy model through a policy gradient method combined with the reward value.
[0012] Exemplarily, the updating of the parameters of the strategy model through the policy gradient method combined with the reward value comprises: calculating a relative advantage score of each candidate prediction sequence in the group based on the reward value, the relative advantage score being used to quantify the pros and cons of the candidate prediction sequence relative to the average performance in the group; and updating the parameters of the strategy model through a policy gradient ascending direction according to the relative advantage score; wherein, when updating the strategy model, a KL divergence is introduced as a constraint term to limit the deviation between the updated strategy model and a reference model as a benchmark.
[0013] Exemplarily, the composite reward function comprises a format reward, a correctness reward and a ranking reward, and an expression of the composite function is as follows:
[0014] wherein, represents the format reward, represents the correctness reward, represents the ranking reward, , , represents a weight coefficient of a corresponding item; the format reward is used to evaluate the format validity of the candidate prediction sequence; the correctness reward is obtained by comparing a prediction result with a true label, and is used to evaluate the accuracy of the prediction result; and the ranking reward is used to optimize the sorting of the candidate prediction sequence with a high score in the output.
[0015] Exemplarily, the target product is predicted according to the final model, comprising: inputting a string of the target product into the final model, and generating a plurality of prediction sequences with the highest probability as reverse biosynthesis path outputs by using a bundle search algorithm.
[0016] In a second aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the inverse biosynthesis prediction method based on progressive training and reinforcement learning of any one of the above aspects when executing the computer program.
[0017] According to the specific embodiments provided in the present application, the following technical effects are disclosed: The present application provides an inverse biosynthesis prediction method based on progressive training and reinforcement learning. By obtaining a large-scale unlabeled molecular corpus for model pre-training, the model can first master the grammar and molecular structure rules before encountering specific reaction tasks, reducing the burden of learning basic rules from limited reaction data. Through the progressive process of inverse synthesis training and synthesis fine-tuning, the model first learns the universal inverse synthesis transformation rules in general chemical reactions, and then fine-tunes through enzyme reaction data to master the biological catalytic specificity, improving the generalization performance and prediction accuracy of the model in the biosynthesis task. Through the process of post-training and optimization of strategy model parameters, the output is comprehensively evaluated and optimized from multiple dimensions, thereby ensuring that the inverse biosynthesis path generated by the final model has higher overall rationality and practicality. The final prediction method can output at least one inverse biosynthesis path containing predicted reactants and their corresponding enzyme numbers according to the input target product. Without relying on a large amount of enzyme reaction data, the method can accurately predict the reactants and catalytic enzymes of the target product. BRIEF DESCRIPTION OF DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0019] Figure 1 The flowchart of the inverse biosynthesis prediction method based on progressive training and reinforcement learning in the embodiments of the present application.
[0020] Figure 2 The network structure and training process schematic diagram in the embodiments of the present application.
[0021] Figure 3 The reinforcement learning process schematic diagram of the model in the embodiments of the present application. DETAILED DESCRIPTION
[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0024] like Figure 1 As shown in the embodiments of this application, a method for predicting inverse biosynthesis based on progressive training and reinforcement learning is provided. The method includes the following steps: S110. Obtain reaction data and preprocess it to obtain an enhanced dataset, which includes chemical reaction dataset and enzyme-catalyzed reaction dataset.
[0025] S120. Obtain a large-scale unlabeled molecular corpus and pre-train the model to be trained based on the molecular corpus to obtain an initial model that has mastered the grammatical rules and molecular structure rules.
[0026] S130. Based on the chemical reaction dataset and the enzyme-catalyzed reaction dataset, the initial model is sequentially trained in supervised retrosynthesis and fine-tuned in retrobiosynthesis to obtain the strategy model.
[0027] S140. The policy model is post-trained using a reinforcement learning method based on group-based policy optimization, and the parameters of the policy model are optimized by optimizing the composite reward function to obtain the final model.
[0028] S150. Based on the final model, predict the target product of the input and output at least one reverse biosynthetic pathway containing the predicted reactants and their corresponding enzyme numbers.
[0029] The prediction method (R) in the embodiments of this application 2Bio), which adopts a template-free sequence-to-sequence generation framework, takes a simplified molecular input line entry system (SMILES) string of a target product molecule as input, and outputs a sequence containing a SMILES string of a reactant and an enzyme commission number (EC number) of a corresponding enzyme. The method improves the performance of the model through a three-stage training process: first, SMILES syntax pre-training, second, progressive training from general chemistry to specific biological catalysis (including reverse synthesis training and reverse biosynthesis fine-tuning), and finally, post-training based on a reinforcement learning method.
[0030] The reverse biosynthesis prediction method based on progressive training and reinforcement learning in the embodiments of the present application pre-trains the model by obtaining a large-scale unlabeled molecular corpus, so that the model can first master the grammar and molecular structure rules before encountering specific reaction tasks, thereby reducing the burden of learning basic rules from limited reaction data. Through the progressive process of reverse synthesis training and synthesis fine-tuning, the model first learns the general reverse synthesis transformation rules in general chemical reactions, and then fine-tunes the enzyme-catalyzed reaction data to master the biological catalytic specificity, thereby improving the generalization performance and prediction accuracy of the model in the biosynthesis task. Through the post-training process and optimization of the strategy model parameters, the output is comprehensively evaluated and optimized from multiple dimensions, thereby ensuring that the reverse biosynthesis path generated by the final model has higher overall rationality and practicality. The final prediction method can output at least one reverse biosynthesis path containing predicted reactants and corresponding enzyme numbers according to the input target product, and can accurately predict the biological precursors (reactants) and catalytic enzymes of the target product without relying on a large amount of enzyme-catalyzed reaction data.
[0031] Exemplarily, in the step S110, the reaction data is pre-processed to obtain an enhanced data set, including the following steps: S111, selecting different atoms as root atoms for product molecules to generate normalized SMILES strings.
[0032] S112, obtaining atom mapping information based on enzyme-catalyzed reactions or chemical reactions.
[0033] S113, traversing the normalized SMILES string according to the atom mapping information to generate a SMILES string of a corresponding reactant, thereby generating a plurality of enhanced samples to obtain an enhanced data set.
[0034] Specifically, before training begins, data augmentation is required to expand the dataset. First, large-scale chemical reaction datasets (such as USPTO-FULL) and enzyme-catalyzed reaction datasets (such as ECREACT) are acquired. Molecules are represented using SMILES. A "root-aligned" data augmentation strategy is employed, rewriting the SMILES strings of products and reactants through atom mapping, traversing from the corresponding "root" atom to reduce edit distance between strings and help the model focus on the core chemical transformations. Finally, through this traversal rewriting process, multiple augmented samples are obtained. For each reaction in the dataset, multiple (e.g., 5) different augmented versions can be obtained through the above data augmentation process, thus achieving data augmentation.
[0035] For example, step S120 above, the process of pre-training the model to be trained based on the molecular corpus, includes the following steps: S121. Based on a large-scale unlabeled molecular corpus, perform masked language model training on the model to be trained, where the model to be trained is a sequence-to-sequence model.
[0036] S122. Masking language model training includes: selecting a predetermined proportion of characters in the input sequence for masking processing, and using the decoder of the model to be trained to predict the original characters at the masked positions based on the masked sequence.
[0037] The above process is the pre-training process of the model to be trained, which is the first step in model training. In this embodiment, a sequence-to-sequence model (such as the T5-Base model) is used as the backbone network, and SMILES grammar pre-training is performed using masked language modeling. During training, a preset proportion (such as 15%) of characters (tokens) in the SMILES sequence are randomly selected for masking. The decoder uses causal masking to predict the mask, which involves predicting the original token at the masked position based on the noise sequence, aiming to enable the model to master the molecular structural rules and SMILES grammar. In addition, in some embodiments, the strategy adopted is: training for 10 epochs, with 80% probability of replacement (MASK), 10% random replacement, and 10% remaining unchanged. This step uses a large-scale unlabeled molecular corpus for model pre-training, allowing the model to master the grammar and molecular structural rules before encountering specific reaction tasks, reducing the burden of learning basic rules from limited reaction data. After pre-training, an initial model that has mastered the grammar rules and molecular structure rules was obtained.
[0038] Next, the second stage of model training, namely step S130 above, is carried out. This step is a progressive training process from general chemistry to specialized biocatalysis. It includes two stages. The first is general chemical reaction training, i.e., retrosynthetic training. In the above steps, the initial model is subjected to supervised retrosynthetic training, including the following steps: S131. Extract the strings of products from the chemical reaction dataset as input to the initial model.
[0039] S132. Using the reactant strings corresponding to the products in the chemical reaction dataset as training targets, optimize the model parameters of the initial model so that the initial model learns the chemical transformation mapping from products to reactants, thereby obtaining a general chemical retrosynthesis prediction capability.
[0040] Specifically, using the initial model obtained in step S120 as a foundation, supervised training is performed on a general chemical reaction dataset (approximately 1 million reactions) to train a sequence-to-sequence retrosynthesis task. The input is an aligned product SMILES string, and the output is an aligned reactant SMILES string. This allows the initial model to learn the chemical transformation mapping from products to reactants. In some embodiments, the training data is augmented 5-fold using a root alignment strategy, and training is performed for 20 epochs. This step enables the model to acquire general chemical retrosynthesis prediction capabilities.
[0041] The second sub-step in step S130 is reverse biosynthesis fine-tuning. This involves fine-tuning the reverse biosynthesis to obtain the strategy model, specifically including the following steps: S133. Based on a model that can predict chemical retrosynthesis in a general way, the product strings of each reaction in the enzyme-catalyzed reaction dataset are used as input data.
[0042] S134. The sequence formed by the corresponding reactant string and the EC number of the enzyme in the enzyme-catalyzed reaction dataset is used as the training target. The training target is constructed as follows: first generate the complete reactant string, and then generate the EC number of the enzyme.
[0043] Using the model obtained in step S132 above as a foundation and initializing the model parameters, fine-tuning was performed on an enzyme-catalyzed reaction dataset. The task was to predict the EC numbers of biological reactants and enzymes. The training data was also augmented 5-fold using a root alignment strategy. Training was performed for 30 epochs. The output format was constructed as a complete reactant string plus the enzyme's EC number. This training process enabled the model to predict enzymes based on complete chemical transformations (the EC number reflects relevant information about the enzyme).
[0044] like Figure 2 The diagram shown is a schematic representation of the network structure and training process in an embodiment of this application, illustrating the method of the present invention (R). 2The three-stage progressive training process used by -Bio Figure 2 The left side shows the first stage (Stage-1): SMILES syntax pre-training. In this stage, an encoder receives a molecular SMILES string processed with a random mask as input, and the decoder's task is to reconstruct the complete and correct molecular SMILES string. This process aims to enable the model to master the basic structural representation rules of chemical molecules. Figure 2 The right side shows the second stage (Stage-2): progressive training from general chemistry to specific biocatalysis. This stage is further divided into two sub-stages. Sub-stage 1 (Phase-1) is pre-training on chemical reactions: the model takes the product (SMILES) strings from general chemical reactions as input and learns to output the corresponding reactant (SMILES) strings. This step enables the model to acquire general chemical retrosynthetic analysis capabilities. Sub-stage 2 (Phase-1) is fine-tuning for retro-biosynthesis: the model input is still the product (SMILES) strings, but the output target is expanded to a combination of reactant (SMILES) strings and enzyme EC numbers. This completes the transfer and adaptation from general chemistry knowledge to biocatalysis-specific knowledge.
[0045] Figure 2 The lower left section shows the third stage (Stage-3): Reinforcement Learning for Reaction Plausibility. This stage uses the previously obtained policy model as its core and further optimizes it through reinforcement learning. The policy model generates multiple candidate outputs (i.e., different inverse synthesis path predictions) for the same input, and these outputs will enter the subsequent reward evaluation and model update cycle.
[0046] Through the above training process, the goal of transferring knowledge from the data-rich field of general chemistry to the data-scarce field of biocatalysis is achieved. The next step is the third stage of training, namely, reinforcement learning for reaction plausibility post-training. As described in step S140 above, the policy model is post-trained using a reinforcement learning method for group-relative policy optimization, and the parameters of the policy model are optimized by optimizing the composite reward function to obtain the final model. This specifically includes the following steps: S141. Based on the target product in the training data, a set of candidate prediction sequences is generated by using a policy model through bundle search.
[0047] S142. Calculate the reward value for each candidate prediction sequence using a composite reward function.
[0048] S143. Update the parameters of the policy model by combining the reward value with the policy gradient method.
[0049] Specifically, this step introduces a reinforcement learning phase based on Group Relative Policy Optimization (GRPO) to directly optimize the overall quality and ranking of the generated candidate prediction sequences. For a given input product, a group of candidate sequences is generated using the current policy model via beam search. A reward value for each candidate prediction sequence is calculated using a composite reward function, and a score is then assigned based on the reward value. The composite reward function includes a format reward, a correctness reward, and a ranking reward, and its expression is shown below:
[0050] in, Indicates a formatted reward. Rewards for correctness Indicates ranking rewards. , , The weight coefficients of the corresponding items are represented; the format reward is used to evaluate the format validity of the candidate prediction sequence; the correctness reward is obtained by comparing the prediction results with the true labels and is used to evaluate the accuracy of the prediction results; the ranking reward is used to optimize the sorting of high-scoring candidate prediction sequences in the output.
[0051] In some embodiments, the reward weight is set to The specific reward process is as follows: For format rewards, the generated SMILES string and EC number are checked for validity. If both are valid, the reward is +1.0. If the SMILES string is valid but the EC number is invalid, the reward is -0.3. If the EC number is valid but the SMILES string is invalid, the reward is -0.7. If both are invalid, the reward is -1.0.
[0052] For the correctness bonus, the generated regularized reaction precursor (reactant) and EC number are compared with the real label. If both are correct, the bonus is +1.0; if only the reaction precursor is correct, the bonus is +0.7; if only the EC number is correct, the bonus is +0.3; if neither is correct, the bonus is -1.0.
[0053] For ranking rewards, the ideal ranking of each candidate is calculated based on the scores of the first two items and compared with its actual ranking in the cluster search. The reward value is calculated based on the degree of misalignment and position weight.
[0054] In step S143 above, the parameters of the policy model are updated using the policy gradient method combined with the reward value. This includes: calculating the relative advantage score of each candidate prediction sequence within the group based on the reward value. The relative advantage score quantifies the performance of the candidate prediction sequence relative to the group's average performance. Based on the relative advantage score, the parameters of the policy model are updated according to the policy gradient ascent direction. Specifically, when updating the policy model, KL (Kullback–Leibler Divergence) divergence is introduced as a constraint term to limit the deviation between the updated policy model and the reference model used as a benchmark.
[0055] For example, for each product in the training data, the old policy model is used ( Generate G=20 candidate prediction sequences through beam search. Then, calculate the reward for each candidate prediction sequence according to the reward function described above, and use the policy model and reference model ( The output probabilities of the above candidate prediction sequences are calculated, and the parameters of the policy model are updated using the GRPO algorithm. After every K=3 updates to the policy model, the old policy model is updated using its parameters. A total of 160,000 training steps are performed.
[0056] It's important to note that the above process does not require training a separate commenting network (such as a Critic model) to evaluate the value of states or actions, thus simplifying the training architecture and reducing computational complexity. By having the policy model generate a set of candidate sequences for the same input and directly calculating the reward for each candidate prediction sequence using a composite reward function, the policy model parameters are updated by comparing the relative advantages among the candidates. This mechanism, based on intra-group relative comparison, avoids the stability issues of dual-model collaborative training in traditional frameworks, achieving more efficient and stable policy optimization.
[0057] like Figure 3 The diagram shown is a schematic representation of the reinforcement learning process of the model in this embodiment of the application, which explains in detail... Figure 2 The third stage involves a reinforcement learning loop based on Group Relative Policy Optimization (GRPO).
[0058] The process begins with the SMILES string of the target molecule (e.g., CC(=O)C, acetone molecule). Three model instances participate in the training: an old policy model responsible for generating candidate policies, a policy model to be updated, and a reference policy model serving as the update baseline, demonstrating the framework of the GRPO algorithm.
[0059] Figure 3 The diagram illustrates the process of generating and calculating candidate prediction sequences: the old policy model generates a set of candidate sequences (e.g., CC(=O)C plus EC number 1111, etc.) for the same target molecule through bundle search. In order to calculate the updated gradient, it is necessary to obtain the generation probabilities of these candidate sequences from the policy model and the reference model. The teacher forcing path in the diagram shows how the model calculates the probability of generating each character (Token, such as C(1)) step by step, conditioned on the ground truth of the candidate prediction sequences.
[0060] The generated candidate sequences are fed into the reward function for calculation, yielding a reward value (reward1, reward2) for each candidate. These reward values then enter the group computation module, whose core purpose is to calculate the relative advantage of each candidate prediction sequence. For example, relative advantage (A1, A2, AG) is derived by comparing the rewards of candidate prediction sequences within the group, quantifying the performance of a single candidate relative to the group average. This is represented in the diagram as probability decrease / increase: for candidate prediction sequences with positive advantage (good performance), the probability of the model generating that sequence in the future should increase, and vice versa; this reflects the policy gradient of this method. Simultaneously, the update process is constrained by KL divergence. KL divergence measures the difference between the output probability distribution of the updated policy model and the reference model, which serves as a stability benchmark. This constraint is to prevent a single update from deviating too far, leading to training failure. Figure 3 The optimization and update every K steps in the text indicate that the above steps are a cyclical iterative process. The policy model parameters are updated periodically, and every K steps (3 in some embodiments) the updated model parameters are synchronized to the old policy model to start a new round of optimization.
[0061] In step S150 above, predicting the input target product based on the final model includes: inputting the string of the target product into the final model, and using a bundle search algorithm to generate several predicted sequences with the highest probability as the output of the reverse biosynthetic pathway. For example, for a new target product molecule, using the final model trained in 3.3, the top-K (when K is 3, it means the top 3) most likely reverse biosynthetic pathways (including predicted reactants and EC numbers) are generated through bundle search.
[0062] In some embodiments, after training through the above steps, the method further includes: using Template-Matching Assessment (TMA) as a supplementary indicator to assess the rationality of the chemical mechanism of the reaction by comparing whether the template of the predicted result matches a template library of known enzyme classes.
[0063] This application presents a reverse biosynthesis prediction method based on progressive training and reinforcement learning. By acquiring a large-scale unlabeled molecular corpus for model pre-training, the model masters grammatical and molecular structural rules before encountering specific reaction tasks, reducing the burden of learning basic rules from limited reaction data. Through this progressive process of reverse biosynthesis training and fine-tuning, the model first learns universal reverse biosynthesis transformation rules in general chemical reactions, and then fine-tunes using enzyme-catalyzed reaction data to master biocatalytic specificity, improving the model's generalization performance and prediction accuracy in biosynthesis tasks. Through post-training and optimization of strategy model parameters, the output is comprehensively evaluated and optimized from multiple dimensions, ensuring that the final reverse biosynthesis pathway generated by the model has higher overall rationality and practicality. The resulting prediction method can output at least one reverse biosynthesis pathway containing the predicted reactants and their corresponding enzyme numbers based on the input target product, achieving accurate prediction of the target product's biological precursors (reactants) and catalytic enzymes without relying on a large amount of enzyme-catalyzed reaction data. Furthermore, this method introduces the GRPO reinforcement learning algorithm as a post-training stage, which directly optimizes the effectiveness, accuracy, and ranking of responses without relying on an additional Critic model.
[0064] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0065] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0066] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0067] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).
[0069] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0070] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0071] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for predicting inverse biosynthesis based on progressive training and reinforcement learning, characterized in that, The inverse biosynthesis prediction method based on progressive training and reinforcement learning includes: The reaction data is acquired and preprocessed to obtain an enhanced dataset, which includes a chemical reaction dataset and an enzyme-catalyzed reaction dataset. A large-scale unlabeled molecular corpus is obtained, and the model to be trained is pre-trained based on the molecular corpus to obtain an initial model that has mastered the grammatical rules and molecular structure rules. Based on the chemical reaction dataset and the enzyme-catalyzed reaction dataset, the initial model is sequentially trained in supervised retrosynthesis and fine-tuned in retrobiosynthesis to obtain the strategy model. The policy model is post-trained using a reinforcement learning method based on group-relative policy optimization, and the parameters of the policy model are optimized by optimizing the composite reward function to obtain the final model. Based on the final model, the target product is predicted, and at least one reverse biosynthetic pathway containing the predicted reactant and its corresponding enzyme number is output.
2. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 1, characterized in that, The reaction data were preprocessed to obtain an augmented dataset, including: Different atoms are selected as root atoms for the product molecules to generate normalized strings; Obtain atomic mapping information based on enzyme-catalyzed or chemical reactions; Based on the atomic mapping information, the normalized string is traversed to generate the corresponding reactant string, thereby generating multiple enhanced samples to obtain the enhanced dataset.
3. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 1, characterized in that, Pre-training the model to be trained based on the molecular corpus includes: Based on a large-scale unlabeled molecular corpus, a masked language model is trained on the model to be trained, wherein the model to be trained is a sequence-to-sequence model; The masked language model training includes: selecting a predetermined proportion of characters in the input sequence for masking processing, and using the decoder of the model to be trained to predict the original character at the masked position based on the masked sequence.
4. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 1, characterized in that, Supervised inverse synthesis training of the initial model includes: Extract the strings of the products from the chemical reaction dataset as input to the initial model; Using the reactant strings corresponding to the products in the chemical reaction dataset as training targets, the model parameters of the initial model are optimized, enabling the initial model to learn the chemical transformation mapping from products to reactants, thereby obtaining a general chemical retrosynthetic prediction capability.
5. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 4, characterized in that, The process of fine-tuning reverse biosynthesis to obtain a strategy model specifically includes: Based on a model that can predict chemical retrosynthesis in a general way, the product strings of each reaction in the enzyme-catalyzed reaction dataset are used as input data. The sequence formed by the reactant string and the enzyme's EC number in the enzyme-catalyzed reaction dataset is used as the training target. The training target is constructed as follows: first, generate the complete reactant string, and then generate the enzyme's EC number.
6. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 1, characterized in that, The policy model is post-trained using a reinforcement learning method based on group-based policy optimization, and the parameters of the policy model are optimized by optimizing the composite reward function to obtain the final model, including: Based on the target product in the training data, a set of candidate prediction sequences is generated by the strategy model through bundle search. The reward value for each candidate prediction sequence is calculated using a composite reward function; The parameters of the policy model are updated by combining the reward value with the policy gradient method.
7. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 6, characterized in that, The parameters of the policy model are updated by combining the reward value with the policy gradient method, including: The relative advantage score of each candidate prediction sequence within the group is calculated based on the reward value. The relative advantage score is used to quantify the superiority or inferiority of the candidate prediction sequence relative to the average performance within the group. Based on the relative advantage score, the parameters of the policy model are updated via the policy gradient ascent direction; wherein, when updating the policy model, KL divergence is introduced as a constraint term to limit the deviation between the updated policy model and the reference model used as a benchmark.
8. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 6, characterized in that, The composite reward function includes a format reward, a correctness reward, and a ranking reward, and the expression of the composite function is as follows: ； in, Indicates a formatted reward. Rewards for correctness Indicates ranking rewards. , , The weight coefficients of the corresponding items are represented; the format reward is used to evaluate the format validity of the candidate prediction sequence; the correctness reward is obtained by comparing the prediction results with the true labels and is used to evaluate the accuracy of the prediction results; the ranking reward is used to optimize the sorting of high-scoring candidate prediction sequences in the output.
9. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 1, characterized in that, The final model is used to predict the target product, including: inputting the string of the target product into the final model and using a cluster search algorithm to generate several predicted sequences with the highest probability as the output of the reverse biosynthesis path.
10. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the inverse biosynthesis prediction method based on progressive training and reinforcement learning as described in any one of claims 1-9.