A method for predicting reverse biosynthesis based on progressive training and reinforcement learning

Abstract

The application discloses a reverse biosynthesis prediction method based on progressive training and reinforcement learning, relates to the technical field of biosynthesis, and comprises the following steps: acquiring reaction data and preprocessing the reaction data to obtain an enhanced data set; pretraining a to-be-trained model according to a molecular corpus to obtain an initial model that masters grammatical rules and molecular structure rules; sequentially performing supervised reverse synthesis training and reverse biosynthesis fine-tuning on the initial model to obtain a strategy model; post-training the strategy model according to a group relative strategy optimization reinforcement learning method, and optimizing parameters of the strategy model by optimizing a composite reward function to obtain a final model; and predicting an input target product according to the final model, and outputting at least one reverse biosynthesis path containing predicted reactants and corresponding enzyme numbers. The application can accurately predict reactants and catalytic enzymes of a target product.

Description

Technical Field

[0001] This application relates to the technical field of biosynthesis, and in particular to a reverse biosynthesis prediction method based on progressive training and reinforcement learning. Background Technology

[0002] Retro-biosynthesis refers to the process of predicting the precursor molecules and required enzymes of a target compound under the constraints of an enzyme-catalyzed reaction, and is crucial for green chemistry and sustainable manufacturing. Enzymatic reactions are inherently complex, often involving large biomolecules and significant structural changes. Annotated enzyme-catalyzed reaction data are extremely scarce; the largest current dataset of enzyme-catalyzed reactions (approximately 60,000 reactions) is far smaller than standard chemical reaction datasets (approximately 1 million reactions).

[0003] In related technologies, reverse biosynthesis methods mainly include template-based, semi-template-based, and template-free methods. Template-based methods rely on rules extracted from known reactions, but the size of enzyme reaction databases is much smaller than that of general chemical reaction libraries, resulting in severely insufficient template coverage, an inability to discover novel reaction pathways, and poor generalization ability. Template-free methods are usually formulated as sequence-to-sequence learning tasks, offering greater flexibility and generalization ability. However, when directly applied to biocatalytic reactions, the performance of these methods deteriorates significantly due to the aforementioned data scarcity and reaction complexity. Training models directly on limited enzyme-catalyzed reaction data easily leads to overfitting and makes it difficult to learn robust chemical principles. Existing multi-task learning methods (training both chemical and enzyme-catalyzed reactions simultaneously) often result in model attention being scattered, failing to accurately grasp the subtle differences in enzyme-catalyzed reactions. Furthermore, while traditional supervised learning can minimize token-level losses, it cannot guarantee the chemical rationality or correctness of the generated entire reaction sequence. Predictions may have small errors at the token level, but may represent invalid molecules or incorrect reactions.

[0004] In summary, the existing technologies have failed to effectively resolve the contradiction between the scarcity of enzyme-catalyzed reaction data and the need for models to learn complex biochemical knowledge. There is a need for a method that can accurately predict the biological precursors (reactants) and catalytic enzymes of target products without relying on a large amount of enzyme-catalyzed reaction data. Summary of the Invention

[0005] The purpose of this application is to provide a reverse biosynthesis prediction method based on progressive training and reinforcement learning, which can accurately predict the reactants and catalytic enzymes of the target product without relying on a large amount of enzyme-catalyzed reaction data.

[0006] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a method for predicting inverse biosynthesis based on progressive training and reinforcement learning, comprising: acquiring reaction data and preprocessing it to obtain an augmented dataset, the augmented dataset including a chemical reaction dataset and an enzyme-catalyzed reaction dataset; acquiring a large-scale unlabeled molecular corpus and pre-training a model to be trained based on the molecular corpus to obtain an initial model that has mastered grammatical rules and molecular structure rules; performing supervised inverse biosynthesis training and inverse biosynthesis fine-tuning on the initial model sequentially based on the chemical reaction dataset and the enzyme-catalyzed reaction dataset to obtain a policy model; post-training the policy model using a reinforcement learning method for group-relative policy optimization, and optimizing the parameters of the policy model by optimizing a composite reward function to obtain a final model; predicting the input target product based on the final model, and outputting at least one inverse biosynthesis path containing the predicted reactant and its corresponding enzyme number.

[0007] For example, preprocessing reaction data to obtain an enhanced dataset includes: selecting different atoms as root atoms for product molecules to generate normalized strings; obtaining atom mapping information based on enzyme-catalyzed or chemical reactions; traversing the normalized strings according to the atom mapping information to generate strings corresponding to the reactants, thereby generating multiple enhanced samples to obtain the enhanced dataset.

[0008] For example, pre-training the model to be trained based on the molecular corpus includes: training the model to be trained using a masked language model based on a large-scale unlabeled molecular corpus, wherein the model to be trained is a sequence-to-sequence model; the masked language model training includes: selecting a predetermined proportion of characters in the input sequence for masking processing, and predicting the original character at the masked position based on the masked sequence using the decoder of the model to be trained.

[0009] For example, supervised retrosynthesis training of the initial model includes: extracting the strings of products from the chemical reaction dataset as input to the initial model; using the reactant strings corresponding to the products in the chemical reaction dataset as training targets, optimizing the model parameters of the initial model, so that the initial model learns the chemical transformation mapping from products to reactants, thereby obtaining a general chemical retrosynthesis prediction capability.

[0010] For example, the step of fine-tuning the reverse biosynthesis to obtain the strategy model specifically includes: based on a model that can obtain general chemical retrosynthesis prediction capabilities, using the product string of each reaction in the enzyme-catalyzed reaction dataset as input data; using the sequence formed by the corresponding reactant string and the EC number of the enzyme in the enzyme-catalyzed reaction dataset as the training target, wherein the training target is constructed as follows: first generating the complete reactant string, and then generating the EC number of the enzyme.

[0011] For example, post-training the policy model according to a reinforcement learning method for group-relative policy optimization, and optimizing the parameters of the policy model by optimizing a composite reward function to obtain a final model, includes: generating a set of candidate prediction sequences using the policy model through beam search based on the target products in the training data; calculating the reward value of each candidate prediction sequence through a composite reward function; and updating the parameters of the policy model by combining the reward values ​​through a policy gradient method.

[0012] For example, updating the parameters of the policy model using the policy gradient method in conjunction with the reward value includes: calculating the relative advantage score of each candidate prediction sequence within the group based on the reward value, wherein the relative advantage score is used to quantify the superiority or inferiority of the candidate prediction sequence relative to the average performance within the group; updating the parameters of the policy model according to the relative advantage score via the policy gradient ascent direction; wherein, when updating the policy model, KL divergence is introduced as a constraint term to limit the deviation between the updated policy model and the reference model used as a benchmark.

[0013] For example, the composite reward function includes a format reward, a correctness reward, and a ranking reward, and the expression of the composite function is as follows:

[0014] in, Indicates a formatted reward. Rewards for correctness Indicates ranking rewards. , , The weight coefficients of the corresponding items are represented; the format reward is used to evaluate the format validity of the candidate prediction sequence; the correctness reward is obtained by comparing the prediction results with the true labels and is used to evaluate the accuracy of the prediction results; the ranking reward is used to optimize the sorting of high-scoring candidate prediction sequences in the output.

[0015] For example, predicting the input target product based on the final model includes: inputting the string of the target product into the final model, and using a cluster search algorithm to generate several predicted sequences with the highest probability as the output of the reverse biosynthesis path.

[0016] In a second aspect, this application provides a computer device, the computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the reverse biosynthesis prediction method based on progressive training and reinforcement learning as described above.

[0017] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a retrosynthetic prediction method based on progressive training and reinforcement learning. By acquiring a large-scale unlabeled molecular corpus for model pre-training, the model masters the grammatical and molecular structural rules before encountering specific reaction tasks, reducing the burden of learning basic rules from limited reaction data. Through a progressive process of retrosynthetic training and synthetic fine-tuning, the model first learns universal retrosynthetic transformation rules in general chemical reactions, and then fine-tunes using enzyme-catalyzed reaction data to master biocatalytic specificity, improving the model's generalization performance and prediction accuracy in biosynthetic tasks. Through post-training and optimization of strategy model parameters, the output is comprehensively evaluated and optimized from multiple dimensions, ensuring that the retrosynthetic pathway generated by the final model has higher overall rationality and practicality. The resulting prediction method can output at least one retrosynthetic pathway containing the predicted reactants and their corresponding enzyme numbers based on the input target product, achieving accurate prediction of the reactants and catalytic enzymes of the target product without relying on a large amount of enzyme-catalyzed reaction data. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the inverse biosynthesis prediction method based on progressive training and reinforcement learning in the embodiments of this application.

[0020] Figure 2 This is a schematic diagram of the network structure and training process in the embodiments of this application.

[0021] Figure 3 This is a schematic diagram of the reinforcement learning process of the model in the embodiments of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] like Figure 1 As shown in the embodiments of this application, a method for predicting inverse biosynthesis based on progressive training and reinforcement learning is provided. The method includes the following steps: S110. Obtain reaction data and preprocess it to obtain an enhanced dataset, which includes chemical reaction dataset and enzyme-catalyzed reaction dataset.

[0025] S120. Obtain a large-scale unlabeled molecular corpus and pre-train the model to be trained based on the molecular corpus to obtain an initial model that has mastered the grammatical rules and molecular structure rules.

[0026] S130. Based on the chemical reaction dataset and the enzyme-catalyzed reaction dataset, the initial model is sequentially trained in supervised retrosynthesis and fine-tuned in retrobiosynthesis to obtain the strategy model.

[0027] S140. The policy model is post-trained using a reinforcement learning method based on group-based policy optimization, and the parameters of the policy model are optimized by optimizing the composite reward function to obtain the final model.

[0028] S150. Based on the final model, predict the target product of the input and output at least one reverse biosynthetic pathway containing the predicted reactants and their corresponding enzyme numbers.

[0029] The prediction method (R) in the embodiments of this application 2The method employs a template-free sequence-to-sequence generation framework. The input is a simplified molecular input line entry system (SMILES) string representing the target product molecule, and the output is a sequence containing the reactant SMILES string and the corresponding enzyme's Enzyme Commission number (EC number). This method improves model performance through a three-stage training process: first, SMILES syntax pre-training; second, progressive training from general chemistry to specific biocatalysis (including retrosynthetic training and fine-tuning); and finally, post-training based on reinforcement learning.

[0030] This application presents a reverse biosynthesis prediction method based on progressive training and reinforcement learning. By acquiring a large-scale unlabeled molecular corpus for model pre-training, the model masters grammatical and molecular structural rules before encountering specific reaction tasks, reducing the burden of learning basic rules from limited reaction data. Through this progressive process of reverse biosynthesis training and fine-tuning, the model first learns universal reverse biosynthesis transformation rules in general chemical reactions, and then fine-tunes using enzyme-catalyzed reaction data to master biocatalytic specificity, improving the model's generalization performance and prediction accuracy in biosynthesis tasks. Through post-training and optimization of strategy model parameters, the output is comprehensively evaluated and optimized from multiple dimensions, ensuring that the final reverse biosynthesis pathway generated by the model has higher overall rationality and practicality. The resulting prediction method can output at least one reverse biosynthesis pathway containing the predicted reactants and their corresponding enzyme numbers based on the input target product, achieving accurate prediction of the target product's biological precursors (reactants) and catalytic enzymes without relying on a large amount of enzyme-catalyzed reaction data.

[0031] For example, in step S110 above, preprocessing the reaction data to obtain an enhanced dataset includes the following steps: S111: Select different atoms as root atoms for the product molecules to generate a normalized SMILES string.

[0032] S112. Obtain atomic mapping information based on enzyme-catalyzed reactions or chemical reactions.

[0033] S113. Based on the atomic mapping information, traverse the normalized SMILES string to generate the SMILES string of the corresponding reactant, thereby generating multiple enhanced samples to obtain an enhanced dataset.

[0034] Specifically, before training begins, data augmentation is required to expand the dataset. First, large-scale chemical reaction datasets (such as USPTO-FULL) and enzyme-catalyzed reaction datasets (such as ECREACT) are acquired. Molecules are represented using SMILES. A "root-aligned" data augmentation strategy is employed, rewriting the SMILES strings of products and reactants through atom mapping, traversing from the corresponding "root" atom to reduce edit distance between strings and help the model focus on the core chemical transformations. Finally, through this traversal rewriting process, multiple augmented samples are obtained. For each reaction in the dataset, multiple (e.g., 5) different augmented versions can be obtained through the above data augmentation process, thus achieving data augmentation.

[0035] For example, step S120 above, the process of pre-training the model to be trained based on the molecular corpus, includes the following steps: S121. Based on a large-scale unlabeled molecular corpus, perform masked language model training on the model to be trained, where the model to be trained is a sequence-to-sequence model.

[0036] S122. Masking language model training includes: selecting a predetermined proportion of characters in the input sequence for masking processing, and using the decoder of the model to be trained to predict the original characters at the masked positions based on the masked sequence.

[0037] The above process is the pre-training process of the model to be trained, which is the first step in model training. In this embodiment, a sequence-to-sequence model (such as the T5-Base model) is used as the backbone network, and SMILES grammar pre-training is performed using masked language modeling. During training, a preset proportion (such as 15%) of characters (tokens) in the SMILES sequence are randomly selected for masking. The decoder uses causal masking to predict the mask, which involves predicting the original token at the masked position based on the noise sequence, aiming to enable the model to master the molecular structural rules and SMILES grammar. In addition, in some embodiments, the strategy adopted is: training for 10 epochs, with 80% probability of replacement (MASK), 10% random replacement, and 10% remaining unchanged. This step uses a large-scale unlabeled molecular corpus for model pre-training, allowing the model to master the grammar and molecular structural rules before encountering specific reaction tasks, reducing the burden of learning basic rules from limited reaction data. After pre-training, an initial model that has mastered the grammar rules and molecular structure rules was obtained.

[0038] Next, the second stage of model training, namely step S130 above, is carried out. This step is a progressive training process from general chemistry to specialized biocatalysis. It includes two stages. The first is general chemical reaction training, i.e., retrosynthetic training. In the above steps, the initial model is subjected to supervised retrosynthetic training, including the following steps: S131. Extract the strings of products from the chemical reaction dataset as input to the initial model.

[0039] S132. Using the reactant strings corresponding to the products in the chemical reaction dataset as training targets, optimize the model parameters of the initial model so that the initial model learns the chemical transformation mapping from products to reactants, thereby obtaining a general chemical retrosynthesis prediction capability.

[0040] Specifically, using the initial model obtained in step S120 as a foundation, supervised training is performed on a general chemical reaction dataset (approximately 1 million reactions) to train a sequence-to-sequence retrosynthesis task. The input is an aligned product SMILES string, and the output is an aligned reactant SMILES string. This allows the initial model to learn the chemical transformation mapping from products to reactants. In some embodiments, the training data is augmented 5-fold using a root alignment strategy, and training is performed for 20 epochs. This step enables the model to acquire general chemical retrosynthesis prediction capabilities.

[0041] The second sub-step in step S130 is reverse biosynthesis fine-tuning. This involves fine-tuning the reverse biosynthesis to obtain the strategy model, specifically including the following steps: S133. Based on a model that can predict chemical retrosynthesis in a general way, the product strings of each reaction in the enzyme-catalyzed reaction dataset are used as input data.

[0042] S134. The sequence formed by the corresponding reactant string and the EC number of the enzyme in the enzyme-catalyzed reaction dataset is used as the training target. The training target is constructed as follows: first generate the complete reactant string, and then generate the EC number of the enzyme.

[0043] Using the model obtained in step S132 above as a foundation and initializing the model parameters, fine-tuning was performed on an enzyme-catalyzed reaction dataset. The task was to predict the EC numbers of biological reactants and enzymes. The training data was also augmented 5-fold using a root alignment strategy. Training was performed for 30 epochs. The output format was constructed as a complete reactant string plus the enzyme's EC number. This training process enabled the model to predict enzymes based on complete chemical transformations (the EC number reflects relevant information about the enzyme).

[0044] like Figure 2 The diagram shown is a schematic representation of the network structure and training process in an embodiment of this application, illustrating the method of the present invention (R). 2The three-stage progressive training process used by -Bio Figure 2 The left side shows the first stage (Stage-1): SMILES syntax pre-training. In this stage, an encoder receives a molecular SMILES string processed with a random mask as input, and the decoder's task is to reconstruct the complete and correct molecular SMILES string. This process aims to enable the model to master the basic structural representation rules of chemical molecules. Figure 2 The right side shows the second stage (Stage-2): progressive training from general chemistry to specific biocatalysis. This stage is further divided into two sub-stages. Sub-stage 1 (Phase-1) is pre-training on chemical reactions: the model takes the product (SMILES) strings from general chemical reactions as input and learns to output the corresponding reactant (SMILES) strings. This step enables the model to acquire general chemical retrosynthetic analysis capabilities. Sub-stage 2 (Phase-1) is fine-tuning for retro-biosynthesis: the model input is still the product (SMILES) strings, but the output target is expanded to a combination of reactant (SMILES) strings and enzyme EC numbers. This completes the transfer and adaptation from general chemistry knowledge to biocatalysis-specific knowledge.

[0045] Figure 2 The lower left section shows the third stage (Stage-3): Reinforcement Learning for Reaction Plausibility. This stage uses the previously obtained policy model as its core and further optimizes it through reinforcement learning. The policy model generates multiple candidate outputs (i.e., different inverse synthesis path predictions) for the same input, and these outputs will enter the subsequent reward evaluation and model update cycle.

[0046] Through the above training process, the goal of transferring knowledge from the data-rich field of general chemistry to the data-scarce field of biocatalysis is achieved. The next step is the third stage of training, namely, reinforcement learning for reaction plausibility post-training. As described in step S140 above, the policy model is post-trained using a reinforcement learning method for group-relative policy optimization, and the parameters of the policy model are optimized by optimizing the composite reward function to obtain the final model. This specifically includes the following steps: S141. Based on the target product in the training data, a set of candidate prediction sequences is generated by using a policy model through bundle search.

[0047] S142. Calculate the reward value for each candidate prediction sequence using a composite reward function.

[0048] S143. Update the parameters of the policy model by combining the reward value with the policy gradient method.

[0049] Specifically, this step introduces a reinforcement learning phase based on Group Relative Policy Optimization (GRPO) to directly optimize the overall quality and ranking of the generated candidate prediction sequences. For a given input product, a group of candidate sequences is generated using the current policy model via beam search. A reward value for each candidate prediction sequence is calculated using a composite reward function, and a score is then assigned based on the reward value. The composite reward function includes a format reward, a correctness reward, and a ranking reward, and its expression is shown below:

[0050] in, Indicates a formatted reward. Rewards for correctness Indicates ranking rewards. , , The weight coefficients of the corresponding items are represented; the format reward is used to evaluate the format validity of the candidate prediction sequence; the correctness reward is obtained by comparing the prediction results with the true labels and is used to evaluate the accuracy of the prediction results; the ranking reward is used to optimize the sorting of high-scoring candidate prediction sequences in the output.

[0051] In some embodiments, the reward weight is set to The specific reward process is as follows: For format rewards, the generated SMILES string and EC number are checked for validity. If both are valid, the reward is +1.0. If the SMILES string is valid but the EC number is invalid, the reward is -0.3. If the EC number is valid but the SMILES string is invalid, the reward is -0.7. If both are invalid, the reward is -1.0.

[0052] For the correctness bonus, the generated regularized reaction precursor (reactant) and EC number are compared with the real label. If both are correct, the bonus is +1.0; if only the reaction precursor is correct, the bonus is +0.7; if only the EC number is correct, the bonus is +0.3; if neither is correct, the bonus is -1.0.

[0053] For ranking rewards, the ideal ranking of each candidate is calculated based on the scores of the first two items and compared with its actual ranking in the cluster search. The reward value is calculated based on the degree of misalignment and position weight.

[0054] In step S143 above, the parameters of the policy model are updated using the policy gradient method combined with the reward value. This includes: calculating the relative advantage score of each candidate prediction sequence within the group based on the reward value. The relative advantage score quantifies the performance of the candidate prediction sequence relative to the group's average performance. Based on the relative advantage score, the parameters of the policy model are updated according to the policy gradient ascent direction. Specifically, when updating the policy model, KL (Kullback–Leibler Divergence) divergence is introduced as a constraint term to limit the deviation between the updated policy model and the reference model used as a benchmark.

[0055] For example, for each product in the training data, the old policy model is used ( Generate G=20 candidate prediction sequences through beam search. Then, calculate the reward for each candidate prediction sequence according to the reward function described above, and use the policy model and reference model ( The output probabilities of the above candidate prediction sequences are calculated, and the parameters of the policy model are updated using the GRPO algorithm. After every K=3 updates to the policy model, the old policy model is updated using its parameters. A total of 160,000 training steps are performed.

[0056] It's important to note that the above process does not require training a separate commenting network (such as a Critic model) to evaluate the value of states or actions, thus simplifying the training architecture and reducing computational complexity. By having the policy model generate a set of candidate sequences for the same input and directly calculating the reward for each candidate prediction sequence using a composite reward function, the policy model parameters are updated by comparing the relative advantages among the candidates. This mechanism, based on intra-group relative comparison, avoids the stability issues of dual-model collaborative training in traditional frameworks, achieving more efficient and stable policy optimization.

[0057] like Figure 3 The diagram shown is a schematic representation of the reinforcement learning process of the model in this embodiment of the application, which explains in detail... Figure 2 The third stage involves a reinforcement learning loop based on Group Relative Policy Optimization (GRPO).

[0058] The process begins with the SMILES string of the target molecule (e.g., CC(=O)C, acetone molecule). Three model instances participate in the training: an old policy model responsible for generating candidate policies, a policy model to be updated, and a reference policy model serving as the update baseline, demonstrating the framework of the GRPO algorithm.

[0059] Figure 3 The diagram illustrates the process of generating and calculating candidate prediction sequences: the old policy model generates a set of candidate sequences (e.g., CC(=O)C plus EC number 1111, etc.) for the same target molecule through bundle search. In order to calculate the updated gradient, it is necessary to obtain the generation probabilities of these candidate sequences from the policy model and the reference model. The teacher forcing path in the diagram shows how the model calculates the probability of generating each character (Token, such as C(1)) step by step, conditioned on the ground truth of the candidate prediction sequences.

[0060] The generated candidate sequences are fed into the reward function for calculation, yielding a reward value (reward1, reward2) for each candidate. These reward values ​​then enter the group computation module, whose core purpose is to calculate the relative advantage of each candidate prediction sequence. For example, relative advantage (A1, A2, AG) is derived by comparing the rewards of candidate prediction sequences within the group, quantifying the performance of a single candidate relative to the group average. This is represented in the diagram as probability decrease / increase: for candidate prediction sequences with positive advantage (good performance), the probability of the model generating that sequence in the future should increase, and vice versa; this reflects the policy gradient of this method. Simultaneously, the update process is constrained by KL divergence. KL divergence measures the difference between the output probability distribution of the updated policy model and the reference model, which serves as a stability benchmark. This constraint is to prevent a single update from deviating too far, leading to training failure. Figure 3 The optimization and update every K steps in the text indicate that the above steps are a cyclical iterative process. The policy model parameters are updated periodically, and every K steps (3 in some embodiments) the updated model parameters are synchronized to the old policy model to start a new round of optimization.

[0061] In step S150 above, predicting the input target product based on the final model includes: inputting the string of the target product into the final model, and using a bundle search algorithm to generate several predicted sequences with the highest probability as the output of the reverse biosynthetic pathway. For example, for a new target product molecule, using the final model trained in 3.3, the top-K (when K is 3, it means the top 3) most likely reverse biosynthetic pathways (including predicted reactants and EC numbers) are generated through bundle search.

[0062] In some embodiments, after training through the above steps, the method further includes: using Template-Matching Assessment (TMA) as a supplementary indicator to assess the rationality of the chemical mechanism of the reaction by comparing whether the template of the predicted result matches a template library of known enzyme classes.

[0063] This application presents a reverse biosynthesis prediction method based on progressive training and reinforcement learning. By acquiring a large-scale unlabeled molecular corpus for model pre-training, the model masters grammatical and molecular structural rules before encountering specific reaction tasks, reducing the burden of learning basic rules from limited reaction data. Through this progressive process of reverse biosynthesis training and fine-tuning, the model first learns universal reverse biosynthesis transformation rules in general chemical reactions, and then fine-tunes using enzyme-catalyzed reaction data to master biocatalytic specificity, improving the model's generalization performance and prediction accuracy in biosynthesis tasks. Through post-training and optimization of strategy model parameters, the output is comprehensively evaluated and optimized from multiple dimensions, ensuring that the final reverse biosynthesis pathway generated by the model has higher overall rationality and practicality. The resulting prediction method can output at least one reverse biosynthesis pathway containing the predicted reactants and their corresponding enzyme numbers based on the input target product, achieving accurate prediction of the target product's biological precursors (reactants) and catalytic enzymes without relying on a large amount of enzyme-catalyzed reaction data. Furthermore, this method introduces the GRPO reinforcement learning algorithm as a post-training stage, which directly optimizes the effectiveness, accuracy, and ranking of responses without relying on an additional Critic model.

[0064] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0065] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0066] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0067] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0069] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchain. The processors involved in the embodiments provided in this application may be, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc.

[0070] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for predicting inverse biosynthesis based on progressive training and reinforcement learning, characterized in that, The inverse biosynthesis prediction method based on progressive training and reinforcement learning includes: The reaction data is acquired and preprocessed to obtain an enhanced dataset, which includes a chemical reaction dataset and an enzyme-catalyzed reaction dataset. A large-scale unlabeled molecular corpus is obtained, and the model to be trained is pre-trained based on the molecular corpus to obtain an initial model that has mastered the grammatical rules and molecular structure rules. Based on the chemical reaction dataset and the enzyme-catalyzed reaction dataset, the initial model is sequentially trained in supervised retrosynthesis and fine-tuned in retrobiosynthesis to obtain the strategy model. The policy model is post-trained using a reinforcement learning method based on group-relative policy optimization, and the parameters of the policy model are optimized by optimizing the composite reward function to obtain the final model. Based on the final model, the target product is predicted, and at least one reverse biosynthetic pathway containing the predicted reactant and its corresponding enzyme number is output. The process involves sequentially performing supervised retrosynthetic training and reverse biosynthetic fine-tuning on the initial model to obtain a strategy model. Specifically, this includes: extracting the product strings from the chemical reaction dataset as input to the initial model; using the reactant strings corresponding to the products in the chemical reaction dataset as the training target to optimize the model parameters of the initial model, enabling the initial model to learn the chemical transformation mapping from products to reactants, thereby obtaining a general chemical retrosynthetic prediction capability; based on the model that has obtained the general chemical retrosynthetic prediction capability, using the product strings of each reaction in the enzyme-catalyzed reaction dataset as input data; and using the sequence formed by the reactant strings corresponding to the enzymes in the enzyme-catalyzed reaction dataset and the enzyme's EC number as the training target. The training target is constructed as follows: first generating the complete reactant string, and then generating the enzyme's EC number. The composite reward function includes a format reward, a correctness reward, and a ranking reward, and its expression is shown below: ； in, Indicates a formatted reward. Rewards for correctness Indicates ranking rewards. , , The weight coefficients of the corresponding items are represented; the format reward is used to evaluate the format validity of the candidate prediction sequence; the correctness reward is obtained by comparing the prediction results with the true labels and is used to evaluate the accuracy of the prediction results; the ranking reward is used to optimize the sorting of high-scoring candidate prediction sequences in the output.

2. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 1, characterized in that, The reaction data were preprocessed to obtain an augmented dataset, including: Different atoms are selected as root atoms for the product molecules to generate normalized strings; Obtain atomic mapping information based on enzyme-catalyzed or chemical reactions; Based on the atomic mapping information, the normalized string is traversed to generate the corresponding reactant string, thereby generating multiple enhanced samples to obtain the enhanced dataset.

3. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 1, characterized in that, Pre-training the model to be trained based on the molecular corpus includes: Based on a large-scale unlabeled molecular corpus, a masked language model is trained on the model to be trained, wherein the model to be trained is a sequence-to-sequence model; The masked language model training includes: selecting a predetermined proportion of characters in the input sequence for masking processing, and using the decoder of the model to be trained to predict the original character at the masked position based on the masked sequence.

4. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 1, characterized in that, The policy model is post-trained using a reinforcement learning method based on group-based policy optimization, and the parameters of the policy model are optimized by optimizing the composite reward function to obtain the final model, including: Based on the target product in the training data, a set of candidate prediction sequences is generated by the strategy model through bundle search. The reward value for each candidate prediction sequence is calculated using a composite reward function; The parameters of the policy model are updated by combining the reward value with the policy gradient method.

5. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 4, characterized in that, The parameters of the policy model are updated by combining the reward value with the policy gradient method, including: The relative advantage score of each candidate prediction sequence within the group is calculated based on the reward value. The relative advantage score is used to quantify the superiority or inferiority of the candidate prediction sequence relative to the average performance within the group. Based on the relative advantage score, the parameters of the policy model are updated via the policy gradient ascent direction; wherein, when updating the policy model, KL divergence is introduced as a constraint term to limit the deviation between the updated policy model and the reference model used as a benchmark.

6. The inverse biosynthesis prediction method based on progressive training and reinforcement learning according to claim 1, characterized in that, The final model is used to predict the target product, including: inputting the string of the target product into the final model and using a cluster search algorithm to generate several predicted sequences with the highest probability as the output of the reverse biosynthesis path.

7. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the inverse biosynthesis prediction method based on progressive training and reinforcement learning as described in any one of claims 1-6.

Images