Marine compound molecule generation method based on deep learning and chemical reaction rules

Abstract

The application belongs to the technical field of computer drug screening, and discloses a marine compound molecule generation method based on deep learning and chemical reaction rules, which comprises three stages of data preprocessing, model pre-training and reinforcement learning fine-tuning; the data preprocessing stage forms a data set, which is used for model pre-training; then, the reinforcement learning method is used to fine-tune the generation behavior of the pre-trained model; finally, the optimized marine compound molecule is generated; through the application, the chemical structure and characteristics of the marine compound can be maximally learned, and the diversity and synthesizability of the synthesized molecule are fully considered.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer drug screening, and particularly relates to a marine compound molecule generation method based on deep learning and chemical reaction rules. BACKGROUND

[0002] With the development of biotechnology and artificial intelligence technology in recent years, new research methods have been provided for marine drug development. How to use emerging algorithm technology to simulate and solve many problems in marine drug research, such as few marine compounds, complex structure, and difficult synthesis, has become a difficulty and hotspot in current marine drug research, and is an important trend in future marine drug research.

[0003] Molecule generation needs to generate new or modified molecular structures according to expected properties. Deep learning models can generate expected molecules directly without using explicit inverse mapping functions based on deep learning models with strong molecular representation learning capabilities. The molecule generation model learns the joint probability distribution. The advantage of applying the generation model is that they can be used to infer the distribution of real data in a supervised or unsupervised manner. This way of learning data can be used for data simulation or new data synthesis.

[0004] The mainstream molecule generation model at the present stage is divided into the following four categories: 1) model based on autoencoder; 2) model based on generative adversarial network; 3) model based on recurrent neural network; 4) hybrid model combining deep generation model and reinforcement learning.

[0005] Meanwhile, the Transformer model has also been verified to be used in the research of molecule generation in drug discovery. MolGPT proposed by Viraj uses part of the decoder of the Transformer to generate molecules and achieves good results. Meanwhile, Yang artificially constructs the required fragment-molecule pairs as training data by using the cutting algorithm. The cutting method using the molecular pair matching algorithm (MMPs, Matched Molecular Pairs) disassembles a molecule into two terminal fragments and a connection segment, and adds some reasonable screening conditions, so that the generated fragment-molecule pairs are more reasonable. Their purpose is to realize that the model can generate a large number of required and structurally diverse drug-like molecules under the condition of given starting terminal fragments and self-defined connection segment constraints.

[0006] The idea of using reinforcement learning to enhance the model is also gradually popular, and researchers have proposed some hybrid models that combine deep generative models with reinforcement learning for molecular generation. Reinforcement learning is a type of artificial intelligence algorithm used to solve dynamic decision-making problems. The basic idea of using reinforcement learning in molecular generation is to guide or constrain the entire molecular generation process through reinforcement learning to obtain the desired properties. In this dynamic process, a generative model for molecular generation is usually designed, and a prediction model is used for molecular evaluation. The evaluation results are fed back to the system as rewards or punishments, guiding the generator to generate the desired molecules in an interactive manner. There are two typical modes of combining deep generative models with reinforcement learning, namely GAN-based models including ORGAN and MolGAN, and ANC-based (Adversarial Neural Computer) models. However, this higher model complexity and model training method still has problems.

[0007] Due to the problems of few samples, difficult experiments, long cycle, low efficiency, high cost, etc. in marine biological research and development, the number of marine compounds is not as dominant as terrestrial compounds, and how to expand the marine compound library with existing marine compounds has become a problem to be solved.

[0008] In terms of Transformer models, the existing technology cannot directly use skeletons and modifications to link fragments to generate molecules, and needs to use seq2seq models to realize compound generation, and cannot fully consider the constraints of chemical reaction conditions in the generation process, and cannot make the synthetic ability of the generated molecules meet the expected effect.

[0009] In terms of reinforcement learning, the training process is not stable, and due to the single reward mechanism design, the model is prone to pattern collapse, causing the molecules generated by reinforcement learning to repeat when optimizing the desired properties, resulting in the problem of insufficient diversity and novelty of the generated molecules. SUMMARY

[0010] In view of the deficiencies of the prior art, the present application provides a marine compound generation method based on deep learning and chemical reaction rules, which solves the problems of few marine compounds, low synthetic ability, unstable training process of Transformer for structure generation and reinforcement learning, and single reward mechanism design.

[0011] To solve the above technical problems, the technical scheme adopted by the present application is:

[0012] The marine compound molecule generation method based on deep learning and chemical reaction rules comprises three stages of data preprocessing, model pre-training and reinforcement learning fine-tuning, the data preprocessing stage forms a data set for model pre-training, and then the generation behavior of the pre-trained model is fine-tuned using a reinforcement learning method, and finally an optimized marine compound molecule is generated; specifically comprising the following steps:

[0013] S1: Establishing a corresponding chemical reaction rule library on the marine compound data set, cutting the molecule using the corresponding chemical reaction rule to form the representation of the skeleton and the molecular modification, and forming the data set;

[0014] S2: The data set and chemical reaction rule of step S1 are input into the pre-trained model for pre-training: the pre-trained model adopts the GPT part of the Transformer model, first trains the model to learn the SMILES syntax, so that the model can generate the molecular skeleton; then further train the model to be able to propose molecular modification for the skeleton to generate complete compounds and perform iterative learning to optimize the parameters;

[0015] The GPT comprises a plurality of decoder blocks, each decoder block is composed of a mask self-attention layer and a feedforward neural network layer, the mask self-attention layer is based on a self-attention mechanism, uses a mask module to assign an infinite or infinitesimal value to a TOKEN value that is not expected to be focused, forcing the model to focus only on the process that has occurred during the training process, and not on the process that will occur in the future, focusing on the prior likelihood of the data; the feedforward neural network adopts a ReLU activation function as a nonlinear factor;

[0016] S3: The pre-trained model of step S2 is initialized as a Prior prior model and an Agent model;

[0017] S4: Input the target skeleton as the molecular structure benchmark;

[0018] S5: Use the Agent model to generate new molecules by sampling the skeleton modification method;

[0019] S6: The generated new molecules are evaluated in multiple ways, respectively through a chemical reaction filter and a diversity filter, and molecules that do not meet the expectations are punished; then use the evaluation of the molecules, including target activity scoring, novelty evaluation and chemical property evaluation, to generate a reward score, if the reward reaches the expectation, execute S8; if it does not reach the expectation, execute S7;

[0020] S7: The source sequence and the new molecule generated by the Agent model are sent to the Prior prior model to obtain the prior likelihood using the teacher-forcing algorithm. The prior likelihood and the reward value obtained in S6 are combined to form an enhanced likelihood. A penalty matrix is formed for molecules that do not meet the expectations through the diversity filter and the chemical reaction filter. The parameters of the Agent model are updated according to the enhanced likelihood and the penalty matrix, and the execution of S4 is returned.

[0021] S8: The generated molecules are evaluated according to the reward score generated in step S6 and the prior likelihood generated in step S7, and the corresponding likelihood is calculated to update the network weight parameters of the Agent model.

[0022] S9: The generated molecules are sorted according to the chemical reaction filter score, and the molecules are stored according to the similarity threshold to generate a high-score molecule library.

[0023] Further, the marine compound dataset used in step S1 is an open source marine compound set. The chemical reaction rule library is established on this dataset and the compound molecules are divided according to this library.

[0024] Further, the molecular generation process of the pre-trained model in step S2 is regarded as a continuous conditional likelihood maximization problem. The output of the model represents a probability distribution in the token space, where the word vector space contains all the tokens in the training set. The token head and tail of the molecular skeleton and the previously generated molecule modification are added with "START" and "END" marks to expand the word vector. The conditional likelihood to be maximized is written as formula (2):

[0025]

[0026] In the formula, θ is the learnable policy parameter, X i , (i = 1,..., T) is a random variable corresponding to the token, and x i is observed or the token generated by the model in the previous generation. S and s respectively refer to the random variables corresponding to the input skeleton and the specified skeleton itself. In this work, the skeleton is input into the prior, so its distribution is determined. T is another random variable that determines the length of the string. In the generation process, the model does not sample its distribution, and when the sampled token is "END", the generation process ends.

[0027] After generating the Prior prior model and stabilizing the generated molecules, the model is initialized as an Agent model, where the Agent model is an instantiated copy of the Prior prior model. The Agent model performs the reinforcement learning process.

[0028] Further, the reinforcement learning in step S5 uses an Agent model, at reinforcement learning time step t, the state space S is defined as t and the corresponding action space A t (s t ), the corresponding reward r t : = R(a t ), the space state contains all the information of the previous proposed modifications and the rewards assigned to these modifications, that is:

[0029]

[0030] The Agent model is according to the policy π θ of the policy parameter θ, a random action is sampled, which is a modification suggestion for the molecular skeleton, the purpose is to find the value of θ which is the optimal policy π θ , and maximize the cumulative reward in the whole reinforcement learning process, E is the likelihood under this variable space, R(A) is the reward score of the current generated compound A:

[0031]

[0032] The objective function of reinforcement learning is shown in formula (5):

[0033]

[0034] In formula (5), the behavior of the Agent depends on the size of the expected value, and the gradient ascent method is used to maximize the objective, the gradient at the next time step can be written as:

[0035]

[0036] In formula (6), represents the policy gradient when the policy θ is used in the next time step t of reinforcement learning, is used to represent the dynamic change of the policy θ.

[0037] Further, in step S6, the following reward strategy is used:

[0038] R(a) = log π a (a) - log π θ (a) (7)

[0039] Where In the formula is the logarithm of the prior likelihood, and is the prior likelihood generated by inputting the molecular sequence generated by the Agent model and the source sequence into the Prior prior model, that is: σ is a learnable hyper-parameter; S(a) is a score between 0 and 1 generated by the scoring function to quantify the effectiveness of the generated compound; logπ θ (a) is the logarithm of the prior likelihood of a certain set of network parameters;

[0040] More specifically, S(a) itself is composed of multiple weighted elements, which are added or multiplied, and the final score is normalized to a value between 0 and 1. This scoring function supports a series of components, including molecular weight (MW), topological polar surface area (TPSA), QSAR activity prediction, and docking scoring. Among them, for molecular diversity and reactions, filters are added to further limit the relevant output space and promote diversity.

[0041] Further, in step S7,

[0042] The diversity filter assesses the repeatability of the generated molecules at the same time and penalizes the behavior of generating the same compound multiple times by halving the score of the repeated compound, preventing the Agent model from repeatedly generating high-scoring molecules;

[0043] The chemical reaction filter assesses whether the newly generated molecule can be generated by applying a specified reaction to the new molecule. During the initial preparation of the data set, the molecules represented by the standard SMLIES have been cut using a reaction-based cutting method, which helps the trained model to learn the reaction knowledge. The chemical reaction filter penalizes the behavior of generating compounds that do not meet the expected requirements.

[0044] Further, the reward score in step S8 is calculated by the trained quantitative structure-activity relationship model and the similarity and synthetic molecule index using the RDkit library. The SMILES one-dimensional molecular descriptor is used as input, and the reward function of the molecule is finally output.

[0045] Compared with the prior art, the present application has the following advantages:

[0046] The present application designs a marine compound molecule generation model based on the GPT model as the benchmark model, and uses a marine compound molecule set as the training set, so that the generated molecules have the characteristics of marine compounds. During the training process, the model can learn the chemical structure knowledge of marine compounds. The self-attention mechanism used in the model ensures that the model calculates the weights of the key areas that have connections between SMILES words, ensuring the diversity and efficiency of the generated molecules.

[0047] The pre-training model can stably generate effective molecules, and the optimized reinforcement learning method is used to fine-tune the generation behavior of the model, so that the molecules generated by the model can maximize the chemical structure and characteristics of marine compounds, fully considering the diversity and synthesizability of the synthesized molecules; and the generated molecules can also be molecules with specified target protein activity, promoting the formation of the key marine drug molecule library in drug discovery research. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art.

[0049] Figure 1 The method flow chart for model training of the present application;

[0050] Figure 2 The pre-training model structure diagram of the present application;

[0051] Figure 3 The compound data cutting method flow chart according to the chemical reaction rule in the embodiment;

[0052] Figure 4 The molecular skeleton structure diagram in the embodiment;

[0053] Figure 5 The molecular generation situation diagram in the embodiment (the number is the synthesizability score, greater than 0 means synthesizable, and the highest score is 1). DETAILED DESCRIPTION

[0054] The present application will be further described below in combination with the drawings and specific embodiments.

[0055] Embodiment 1

[0056] In combination with Figure 1 the present embodiment provides a marine compound molecule generation method based on deep learning and chemical reaction rules, including data preprocessing, model pre-training and reinforcement learning fine-tuning three stages, the data preprocessing stage forms a data set, which is used for model pre-training, then the reinforcement learning method is used to fine-tune the generation behavior of the pre-training model, and finally the optimized marine compound molecules are generated. Specifically, the following steps are included:

[0057] S1: Establishing a corresponding chemical reaction rule library on the marine compound data set, cutting the molecules using the corresponding chemical reaction rules, forming the representation of the skeleton and molecular modification, and forming the data set.

[0058] The marine compound dataset used in step S1 is an open-source marine compound dataset (such as MC3D). A chemical reaction rule base is built on this dataset, and compound molecules are segmented according to this base. First, the marine compound database (MC3D) is cleaned to remove molecules that are obviously not drug-like and molecules that are not of great learning significance to the model. Then, the molecules represented by SMILES are divided into skeleton and modification representations according to chemical reaction rules to form a training set, which is used in the pre-trained model.

[0059] S2: Input the dataset and chemical reaction rules from step S1 as training conditions into the pre-trained model for pre-training: The pre-trained model structure is as follows Figure 2 As shown, the pre-trained model uses the GPT part of the Transformer model. First, the model is trained to learn the SMILES syntax, enabling it to generate molecular skeletons. Then, the model is further trained to propose the most suitable molecular modifications for the skeleton, thereby generating complete compounds, and iterative learning is performed to optimize the parameters.

[0060] The GPT comprises multiple stacked decoder blocks, each consisting of a masked self-attention layer and a feedforward neural network layer. The masked self-attention layer is based on a self-attention mechanism and uses a mask module to assign infinite or infinitesimal TOKEN values ​​that are not intended to be focused on, forcing the model to focus only on the processes that have already occurred during training, rather than on the processes that will occur in the future, thus focusing on the prior likelihood of the data. The feedforward neural network uses the ReLU activation function as a nonlinear factor.

[0061] The molecular generation process of the pre-trained model in step S2 can be regarded as a continuous conditional likelihood maximization problem. The output of the model represents the probability distribution on the token space, where the word vector space contains all the word tokens in the training set. The word vectors are expanded by adding "START" and "END" markers to the beginning and end of the molecular skeleton and the previously generated molecular modified tokens. The conditional likelihood to be maximized can be written as formula (2):

[0062]

[0063] In the formula, θ is a learnable parameter, and X... i (i = 1, ..., T) are random variables corresponding to the token, and x i These are either observed tokens or tokens previously generated by the model. S and s refer to the random variables corresponding to the input skeleton and the specified skeleton itself, respectively. In this work, the skeleton is input prior, so its distribution is deterministic. T is another random variable that determines the string length; its distribution is not sampled by the model during generation. The generation process ends when the sampled token is "END".

[0064] After generating the Prior prior model and stabilizing the molecule generation, the model is initialized as the Agent model, which is an instantiated copy of the Prior prior model. The Agent model then undergoes a reinforcement learning process.

[0065] S3: Initialize the pre-trained model from step S2 into a Prior model and an Agent model.

[0066] S4: Input the target skeleton as the molecular structure benchmark.

[0067] S5: Use the Agent model to sample and generate new molecules through skeleton modification.

[0068] In step S5, an agent model is used for reinforcement learning. At time step t of the reinforcement learning, the state space S is defined. t and the corresponding motion space A t (s t ), and the corresponding reward r t :=R(a t The spatial state contains all previously suggested modifications and all information regarding the rewards assigned to those modifications, namely:

[0069]

[0070] In the formula, the reward function remains constant throughout the process. At each step, the agent model adjusts the policy π based on the parameter θ. θ A random action is selected, which proposes a modification to the molecular skeleton. The goal is to identify strategy π. θ The optimal value of θ is determined, and the cumulative reward is maximized throughout the reinforcement learning process. E is the likelihood in this variable space, and R(A) is the reward score for the currently generated compound A.

[0071]

[0072] The expected value is greedily maximized at each time step. Therefore, the objective function of reinforcement learning is as follows:

[0073]

[0074] In equation (5), the agent's behavior depends on the magnitude of the expected value, and gradient ascent is used to maximize the objective. The gradient at the next time step can be written as:

[0075]

[0076] In equation (6), This represents the policy gradient at the next time step t when the reinforcement learning adopts the policy θ. This is used to represent the dynamic changes of policy θ. Formula (6) is the basis of reinforcement learning algorithms. If the model training objective is to maximize the cumulative reward over N training phases, the above formula helps to add the cumulative value in all time steps, which is sufficient to calculate the gradient of the log probability to obtain the update step of gradient ascent.

[0077] S6: Perform multiple evaluations on the generated new molecules, using both chemical reaction filters and diversity filters, and penalize molecules that do not meet expectations; then use molecule evaluation, including targeting activity scoring, novelty assessment, and chemical property assessment, to generate a reward score. If the reward meets expectations, proceed to S8; otherwise, proceed to S7.

[0078] The goal of molecular generation is to produce a large number of possible molecules, which means that a certain degree of diversity is needed to facilitate the exploration of chemical space. To meet the above requirements and prevent the pattern from collapsing into a single, high-molecular-weight molecule, this invention adopts the following reward strategy:

[0079] R(a) = logπ a (a)-logπ θ (a) (7)

[0080] in In the formula It is the logarithm of the prior likelihood, which is the prior likelihood generated by inputting the molecular sequence and source sequence generated by the Agent model into the Prior prior model, i.e.: σ is a learnable hyperparameter; S(a) is a score between 0 and 1 generated by the scoring function, used to quantify the effectiveness of compound generation; logπ θ (a) is the logarithm of the prior likelihood of a certain set of network parameters.

[0081] More specifically, S(a) itself consists of multiple weighted elements, which are added or multiplied, and the final score is standardized to a value between 0 and 1. This scoring function supports a range of components, including molecular weight (MW), topological polar surface area (TPSA), QSAR activity prediction, and docking scoring. For molecular diversity and reactions, filters are added to further limit the relevant output space and promote diversity.

[0082] S7: Input the source sequence and the new molecule generated by the Agent model into the Prior prior model and use the teacher-forcing algorithm to obtain the prior likelihood. Combine the prior likelihood and the reward value obtained in S6 to form the enhanced likelihood. Then, use the diversity filter and chemical reaction filter to form a penalty matrix for molecules that do not meet the expectations. Update the parameters of the Agent model according to the enhanced likelihood and the penalty matrix, and return to execute S4.

[0083] In step 7, a diversity filter is added to evaluate the repeatability of molecules generated within the same time period and to penalize the behavior of generating the same compound multiple times. Specifically, the score of the repeated compound is halved to prevent the agent model from repeatedly generating high-scoring molecules and avoid model collapse.

[0084] Adding a chemical reaction filter assesses whether a new molecule can be generated by applying a specified reaction to it. In the early stages of dataset preparation, molecules represented by standard SMLIES were already segmented using a reaction-based segmentation method, which helps the training model learn reaction knowledge. The reaction filter penalizes the generation behavior of compounds that do not meet expectations. This consideration of known chemical reactions introduces positive induction bias, resulting in more feasible results. These compounds can be synthesized through chemical routes obtained through fully explored and more readily available methods, thus improving the syntheticability of compounds.

[0085] S8: Evaluate the generated molecules based on the reward score generated in step S6 and the prior likelihood generated in step S7, calculate the corresponding likelihood, and update the network weight parameters of the Agent model.

[0086] The reward score mentioned in step S8 is calculated using a trained quantitative structure-activity relationship (QSAR) model (reward function component, which is not a design feature of this invention and will not be elaborated further) and similarity and syntheticity molecular indices using the RDkit library. The SMILES one-dimensional molecular descriptor is used as input, and the reward function of the molecule is finally output.

[0087] S9: Sort the generated molecules according to the chemical reaction filter score, and store the molecules according to the similarity threshold to generate a high-resolution molecular library that can synthesize marine compounds.

[0088] Example 2

[0089] This embodiment illustrates the method for generating marine compound molecules according to the present invention with specific examples.

[0090] First, the marine compound database (MC3D) is cleaned to remove molecules that are clearly not drug-like and those that have little learning value for the model. Then, data augmentation algorithms are used, such as... Figure 3The SMILES representation of molecules is divided into skeleton and modified representations according to chemical reaction rules, forming a training set and used in the pre-trained model.

[0091] The specific process of pre-training the model is as follows: Figure 1 As shown, the processed training set and chemical reaction rules are input into the pre-trained model as training conditions. A "START" start marker is provided to the model. The model is trained using the Teacher-forcing method. During the training process, the pre-trained model can propose possible modifications to the skeleton to generate new molecules and perform iterative learning to optimize parameters. When the model generates an "END" end marker, molecule generation ends.

[0092] Pre-trained model structure as follows Figure 2 As shown, it is essentially a mini version of the Transformer, based on GPT with further parameter compression. This includes stacked decoder blocks, each consisting of a masked self-attention layer and a feedforward fully connected layer. The input to each masked self-attention layer is a 256-bit feature vector, output from the previous fully connected layer. The output is a 1024-bit vector, activated by the GELU function, and then fed into the fully connected layer to map the 1024-bit vector back to a 256-bit vector before being fed into the next decoder block. This process continues across eight decoder blocks, ultimately generating a possible molecular modification. Training samples undergo embedding before being input into the model. Distinguishing markers are used to differentiate between the backbone, modifications, and chemical reaction rules, mapping these to 256-bit feature vectors. These are then combined with 256-bit position vectors to distinguish word positions, generating a 256-bit aggregated information vector that is input into the model for training.

[0093] The key design element of the GPT structure lies in its masked self-attention layer. The self-attention mechanism calculates the relevant attention score through dot product. It involves three sets of vectors: Query (Q), Key (K), and Value (V). The Q vector is dot-producted with the K vector, then normalized using a softmax function to obtain a set of weight vectors that sum to 1. The V vector is then multiplied by each of these weight vectors and summed to obtain the corresponding attention score, as shown in the following formula:

[0094]

[0095] Where Q, K, and V are the Query, Key, and Value vectors, respectively, and d k is the dimension of the Query and Key vectors, used here to prevent the dot product from being too large, and T is the transpose of the weight matrix.

[0096] To make the attention more stable in predicting future labels, a masked self-attention mechanism is introduced. During training, the model improves the stability of predicting future labels by masking them.

[0097] The training process consisted of ten iterations, and the Adam optimizer was used for optimization. The experimental results were evaluated using the following criteria (these are existing criteria, and are only briefly introduced here):

[0098] Validity: The validity of the generated molecules. RDkit is used to check the validity of the generated molecules to measure the model’s understanding of the SMILES syntax.

[0099] Validity = |C s | / n s Validity∈[0,1] (9)

[0100] In the formula, C s It is the set of generated molecules, n s It represents the number of molecules produced.

[0101] Uniqueness: The uniqueness among the generated effective molecules, determining the distribution of repetitive generated molecules in the model generation.

[0102] Uniqueness = |set(C s )| / n s Uniqueness∈[0,1] (10)

[0103] Novelty: The percentage of molecules that are effectively and uniquely generated.

[0104] Novelty = 1 - |C s ∩D| / |C s |Novelty∈[0,1] (11)

[0105] In the formula, D is the training set.

[0106] Internal Diversity (IntDiv) p ): Used to check whether mode collapse has occurred, that is, whether the generated molecules have shifted toward the same dominant structure, resulting in the generation of a large number of molecules with the same structure.

[0107]

[0108] This patent references the MOLGPT model by Bajal et al. as a pre-trained model. Table 1 shows a comparison of the model's performance.

[0109] Table 1: Performance Comparison of Models

[0110]

[0111] After the pre-trained model is trained, it is initialized as a Prior model and an Agent model for reinforcement learning. The input is the backbone of the active molecule of the target DRD2 [*:0]N1CCN(CC1)CCCCN[*:1] as shown. Figure 4 The diagram shown serves as the baseline skeleton for this embodiment, where [*:1] and [*:0] are bond connection points capable of undergoing chemical reactions.

[0112] The molecular skeleton is input into the pre-trained model, which then undergoes reinforcement learning to generate molecules. The resulting compound library is as follows: Figure 5 As shown.

[0113] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should be protected by the present invention.

Claims

1. A method for generating marine compound molecules based on deep learning and chemical reaction rules, characterized in that, It includes three stages: data preprocessing, model pretraining, and reinforcement learning fine-tuning. The data preprocessing stage forms a dataset for model pretraining. Then, reinforcement learning methods are used to fine-tune the generation behavior of the pretrained model, and finally, optimized marine compound molecules are generated. Specifically, the following steps are included: S1: Establish a corresponding chemical reaction rule library on the marine compound dataset, use the corresponding chemical reaction rules to cut molecules, form the skeleton and molecular modification characterization, and form a dataset; S2: Input the dataset and chemical reaction rules from step S1 as training conditions into the pre-trained model for pre-training: The pre-trained model uses the GPT part of the Transformer model. First, train the model to learn the SMILES syntax so that the model can generate molecular skeletons; then train the model further so that it can propose molecular modifications for the skeleton to generate complete compounds and perform iterative learning to optimize parameters. The GPT includes multiple decoder blocks, each consisting of a masked self-attention layer and a feedforward neural network layer. The masked self-attention layer is based on a self-attention mechanism and uses a mask module to assign infinite or infinitesimal TOKEN values ​​that are not intended to be focused on, forcing the model to focus only on the processes that have already occurred during training, rather than on the processes that will occur in the future, thus focusing on the prior likelihood of the data. The feedforward neural network uses the ReLU activation function as a nonlinear factor. S3: Initialize the pre-trained model from step S2 into the Prior model and the Agent model; S4: Input the target framework as the molecular structure benchmark; S5: Using the target skeleton input in step S4 as a baseline, new molecules are generated by sampling through skeleton modification using the Agent model; S6: Perform multiple evaluations on the generated new molecules, using both chemical reaction filters and diversity filters, and penalize molecules that do not meet expectations; then use molecule evaluation, including targeting activity scoring, novelty assessment, and chemical property assessment, to generate a reward score. If the reward meets expectations, proceed to S8; otherwise, proceed to S7. S7: Input the source sequence and the new molecule generated by the Agent model into the Prior prior model and use the teacher-forcing algorithm to obtain the prior likelihood. Combine the prior likelihood and the reward value obtained in S6 to form the enhanced likelihood. Then, use the diversity filter and chemical reaction filter to form a penalty matrix for molecules that do not meet the expectations. Update the parameters of the Agent model according to the enhanced likelihood and the penalty matrix, and return to execute S4. S8: Evaluate the generated molecules based on the reward score generated in step S6 and the prior likelihood generated in step S7, calculate the corresponding likelihood, and update the network weight parameters of the Agent model. S9: Sort the generated molecules according to the chemical reaction filter score, and store the molecules according to the similarity threshold to generate a high-scoring molecule library.

2. The method for generating marine compound molecules based on deep learning and chemical reaction rules according to claim 1, characterized in that, The marine compound dataset used in step S1 is an open-source marine compound dataset. A chemical reaction rule library is built on this dataset and compound molecules are segmented according to this library.

3. The method for generating marine compound molecules based on deep learning and chemical reaction rules according to claim 1, characterized in that, In step S2, the molecular generation process of the pre-trained model is regarded as a continuous conditional likelihood maximization problem. The output of the model represents the probability distribution on the token space, where the word vector space contains all the word tokens in the training set. The word vectors are expanded by adding "START" and "END" markers to the beginning and end of the molecular skeleton and the previously generated molecular modified tokens. The conditional likelihood to be maximized is written as formula (2): In the formula, θ is the learnable policy parameter, and X... i It is a random variable corresponding to the token, i = 1,...,T, and x i The tokens are either observed or generated by the model previously. S and s refer to the random variables corresponding to the input skeleton and the specified skeleton itself, respectively. In this work, the skeleton is input prior, so its distribution is deterministic. T is another random variable that determines the string length. During the generation process, the model does not sample its distribution, and the generation process ends when the sampled token is "END". After generating the Prior model and stabilizing the generation of molecules, the model is initialized as the Agent model, which is an instantiated copy of the Prior model. The Agent model then undergoes a reinforcement learning process.

4. The method for generating marine compound molecules based on deep learning and chemical reaction rules according to claim 1, characterized in that, In step S5, an agent model is used for reinforcement learning. At time step t of the reinforcement learning, the state space S is defined. t and the corresponding motion space A t (s t ), and the corresponding reward r t ∶=R(a t The spatial state contains all previously suggested modifications and all information regarding the rewards assigned to those modifications, namely: The agent model uses policy π based on policy parameter θ. θ A random action is selected, which proposes a modification to the molecular skeleton. The goal is to identify strategy π. θ The optimal value of θ is determined, and the cumulative reward is maximized throughout the reinforcement learning process. E is the likelihood in this variable space, and R(A) is the reward score for the currently generated compound A. The objective function for reinforcement learning is shown in equation (5): In equation (5), the agent's behavior depends on the magnitude of the expected value, and gradient ascent is used to maximize the objective. The gradient at the next time step can be written as: In equation (6), This represents the policy gradient at the next time step t when the reinforcement learning adopts the policy θ. Used to represent the dynamic changes of policy θ.

5. The method for generating marine compound molecules based on deep learning and chemical reaction rules according to claim 1, characterized in that, In step S6, the following reward strategy is adopted: R(a)=logπ a (a)-logπ θ (a) (7) in In the formula It is the logarithm of the prior likelihood, which is the prior likelihood generated by inputting the molecular sequence and source sequence generated by the Agent model into the Prior prior model, i.e.: σ is a learnable hyperparameter; S(a) is a score between 0 and 1 generated by the scoring function, used to quantify the effectiveness of compound generation; logπ θ (a) is the logarithm of the prior likelihood of a certain set of network parameters; More specifically, S(a) consists of multiple weighted elements that are added or multiplied together, and the final score is standardized to a value between 0 and 1. This scoring function supports a range of components, including molecular weight (MW), topological polar surface area (TPSA), QSAR activity prediction, and docking scoring. For molecular diversity and reaction, filters are added to further limit the relevant output space and promote diversity.

6. The method for generating marine compound molecules based on deep learning and chemical reaction rules according to claim 1, characterized in that, In step S7, Add a diversity filter to evaluate the repeatability of molecules generated in the same time period and penalize the behavior of generating the same compound multiple times. Specifically, the score of repeated compounds is halved to prevent the agent model from repeatedly generating high-scoring molecules. A chemical reaction filter is added to evaluate whether a new molecule can be generated by applying a specified reaction to the newly generated molecule. In the early stages of dataset preparation, molecules represented by standard SMLIES have been cut using a reaction-based cutting method, which helps the training model learn reaction knowledge. The chemical reaction filter penalizes the generation behavior of compounds that do not meet expectations.

7. The method for generating marine compound molecules based on deep learning and chemical reaction rules according to claim 1, characterized in that, The reward score mentioned in step S8 is calculated using a trained quantitative structure-activity relationship model and similarity and syntheticity molecular indices using the RDkit library, with the SMILES one-dimensional molecular descriptor as input, and finally output as the reward function of the molecule.

Images