A molecular optimization method, device, equipment, medium and product

Abstract

The present application discloses a molecular optimization method, apparatus, equipment, medium and product, relating to the field of molecular design. The method comprises: according to a first training set, with the goal of minimizing a first loss function, using contrastive learning to train a conditional transformer model to obtain a generative model; the first loss function comprises a contrastive loss function, a KL loss function and a maximum likelihood estimation function; an optimized molecule is obtained according to the generative model, and based on a knowledge distillation method, an RNN model is trained according to the optimized molecule to obtain a distillation model; based on the distillation model, a reinforcement learning method is used to train an intelligent agent model to obtain a molecular optimization model, and the molecular optimization model is used to optimize the molecule to be optimized; the reward function in the reinforcement learning method comprises binding affinity. The present application can solve the problem of exposure bias, so that the optimized molecule finally generated meets the requirements.

Description

Technical Field

[0001] The present application relates to the field of molecular design, and in particular to a molecular optimization method, device, equipment, medium and product. Background Art

[0002] Drug discovery often consumes enormous resources and time. Therefore, the use of computer technology to aid in the discovery and design of drug molecules has significant practical value. Molecular optimization is a key step in the drug design (molecule generation) process. The goal of molecular optimization is to improve the desired properties of drug candidates by chemically modifying molecules, maximizing their properties while maintaining similarity to the input molecule. However, current molecular optimization methods are generally subject to exposure bias, resulting in poor quality of the resulting optimized molecules. Summary of the Invention

[0003] The purpose of this application is to provide a molecular optimization method, device, equipment, medium and product that can solve the problem of exposure deviation and make the quality of the optimized molecules finally generated better.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a molecular optimization method, comprising:

[0006] Obtaining a first training set, the first training set comprising a plurality of sample molecules; wherein the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range;

[0007] According to the first training set, the conditional transformer model is trained using contrastive learning with the goal of minimizing a first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function, and a maximum likelihood estimation function;

[0008] The optimized molecule is obtained according to the generative model, and the RNN model is trained based on the optimized molecule based on the knowledge distillation method to obtain a distillation model;

[0009] The agent model is trained using a reinforcement learning method according to the distillation model to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

[0010] Optionally, based on the first training set, with the goal of minimizing the first loss function, contrastive learning is used to train the conditional transformer model to obtain a generative model, specifically including:

[0011] Input each sample molecule in the first training set into the conditional transformer model to obtain the predicted probability distribution corresponding to each sample molecule;

[0012] Input each sample molecule in the first training set into the encoder of the conditional transformer model to obtain the hidden state corresponding to each sample molecule;

[0013] For any sample molecule in the first training set, negatively perturb the predicted probability distribution corresponding to the sample molecule according to the maximum likelihood estimation function to obtain the probability distribution of the negative molecule corresponding to the sample molecule;

[0014] Performing a positive perturbation on the predicted probability distribution corresponding to the sample molecule according to a positive contrast loss function to obtain a probability distribution of an intermediate positive molecule corresponding to the sample molecule;

[0015] Performing a positive perturbation on the probability distribution of the intermediate positive molecules corresponding to the sample molecules according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules;

[0016] Calculate a first loss function value according to the predicted probability distribution corresponding to each sample molecule, the hidden state corresponding to each sample molecule, the probability distribution of the intermediate positive molecule corresponding to each sample molecule, the probability distribution of the negative molecule corresponding to each sample molecule, and the probability distribution of the positive molecule corresponding to each sample molecule;

[0017] The conditional transformer model is trained according to the first loss function value to obtain a generative model.

[0018] Optionally, the predicted probability distribution corresponding to the sample molecule is negatively perturbed according to the maximum likelihood estimation function to obtain the probability distribution of the negative molecule corresponding to the sample molecule:

[0019] According to the formula The probability distribution of negative molecules corresponding to the sample molecules is obtained, where represents the probability distribution of negative molecules corresponding to the i-th sample molecule, S i Represents the predicted probability distribution corresponding to the i-th sample molecule, λ represents the negative weight, and g represents the back propagation S of the maximum likelihood estimation function i The gradient operation of ,||||2 represents the L2 norm.

[0020] Optionally, the predicted probability distribution corresponding to the sample molecule is positively perturbed according to the positive contrast loss function to obtain the probability distribution of the intermediate positive molecule corresponding to the sample molecule, specifically:

[0021] According to the formula The probability distribution of the intermediate positive molecules corresponding to the sample molecules is obtained, where represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, μ represents the positive weight, f + Represents back propagation S of the positive contrast loss function i The gradient operation of ,||||2 represents the L2 norm.

[0022] Optionally, the probability distribution of the intermediate positive molecules corresponding to the sample molecules is positively perturbed according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules, specifically:

[0023] According to the formula The probability distribution of positive molecules corresponding to the sample molecules is obtained, where represents the probability distribution of the positive molecule corresponding to the i-th sample molecule, represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, μ represents the positive weight, and f ++ Represents the back propagation S of the KL loss function i The gradient operation of ,||||2 represents the L2 norm.

[0024] Optionally, the first loss function is:

[0025] L=max θ L MLE (θ)-L KL (θ)+L cont- (θ)+L cont+ (θ), where L represents the first loss function, max θ L MLE (θ) represents the L MLE (θ) takes the maximum value, L MLE (θ) represents the maximum likelihood estimation function, L KL (θ) represents the KL loss function, L cont- (θ) represents the negative contrast loss function, L cont+ (θ) represents the contrast loss function with the probability distribution of positive molecules, and θ represents the parameters of the conditional transformer model;

[0026] S i represents the predicted probability distribution corresponding to the i-th sample molecule, and N represents the total number of sample molecules in the first training set;

[0027] Indicates the projection of the hidden state corresponding to the i-th sample molecule, Represents the projection of the predicted probability distribution corresponding to the i-th sample molecule, τ represents the hyperparameter, Indicates U i To project, S j represents the predicted probability distribution corresponding to the jth sample molecule, ∪ represents the union, represents the probability distribution of the negative molecule corresponding to the i-th sample molecule, exp() represents the exponential function with e as the base, and cos() represents the cosine function;

[0028] Among them, D KL represents KL divergence, softmax() represents activation function, W represents weight matrix; b represents bias vector, μ represents positive weight, f + Represents the back propagation S of the positive contrast loss function i Gradient operation, || ||2 represents the L2 norm, Indicates S i and KL divergence calculation between;

[0029] It represents the projection of the set consisting of the probability distribution of the i-th sample molecule and the positive molecule corresponding to the i-th sample molecule.

[0030] In a second aspect, the present application provides a molecular optimization device, comprising:

[0031] An acquisition module is configured to acquire a first training set, wherein the first training set includes a plurality of sample molecules; the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range;

[0032] A contrastive learning module is configured to train the conditional transformer model using contrastive learning based on the first training set with the goal of minimizing a first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function, and a maximum likelihood estimation function;

[0033] The knowledge distillation module is used to obtain optimized molecules based on the generative model and train the RNN model based on the optimized molecules based on the knowledge distillation method to obtain a distilled model;

[0034] A reinforcement learning module is used to train the intelligent agent model using a reinforcement learning method based on the distillation model to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

[0035] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any one of the above-described molecular optimization methods.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any one of the above-described molecular optimization methods.

[0037] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements any one of the above-described molecular optimization methods.

[0038] According to the specific embodiments provided in this application, this application has the following technical effects:

[0039] This application provides a molecular optimization method, apparatus, device, medium, and product. The reason for the exposure bias problem in the conditional Transformer model in related technologies is that the inputs in the inference and training stages are different. During the training stage, each word input comes from a real sample; while during the inference stage, the current input uses the output of the previous word. Therefore, to avoid the exposure bias problem, this application introduces a contrastive learning method to solve the problem of different inputs in the inference and training stages, achieving a solution to the exposure bias problem, so that the resulting optimized molecules meet the requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 This is a schematic diagram of a molecular optimization method in one embodiment of the present application;

[0042] Figure 2 A schematic diagram of a molecular optimization method provided in one embodiment of the present application;

[0043] Figure 3 Flowchart for training the conditional transformer model;

[0044] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0046] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0047] In an exemplary embodiment, a molecular optimization method is provided, such as Figure 1 and Figure 2 As shown, the following steps are included, wherein:

[0048] Step 201: Obtain a first training set, comprising a plurality of sample molecules; each sample molecule in the first training set has a protein ratio within a first preset range (optimal protein ratio) and a binding affinity within a second preset range (optimal binding affinity). Related techniques optimize molecules based on their chemical properties. This application selects a training set based on preset protein ratio and binding affinity requirements, optimizing the molecules based on these two aspects.

[0049] Step 202: Based on the first training set, with the goal of minimizing the first loss function, use contrastive learning to train the conditional transformer model (structured as Figure 3 As shown) is trained to obtain a generation model; the first loss function includes a contrast loss function, a KL loss function and a maximum likelihood estimation function.

[0050] Step 203: Obtain an optimized molecule according to the generative model, and train the RNN model according to the optimized molecule based on the knowledge distillation method to obtain a distillation model.

[0051] Step 204: The agent model is trained using a reinforcement learning method according to the distillation model to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

[0052] In an exemplary embodiment, based on the first training set, with the goal of minimizing the first loss function, contrastive learning is used to train the conditional transformer model to obtain a generative model, specifically including:

[0053] Each sample molecule in the first training set is input into the conditional transformer model to obtain the predicted probability distribution corresponding to each sample molecule.

[0054] Each sample molecule in the first training set is input into the encoder of the conditional transformer model to obtain the hidden state corresponding to each sample molecule.

[0055] For any sample molecule in the first training set, negative perturbation is performed on the predicted probability distribution corresponding to the sample molecule according to the maximum likelihood estimation function to obtain the probability distribution of the negative molecule corresponding to the sample molecule.

[0056] The predicted probability distribution corresponding to the sample molecule is positively perturbed according to the positive contrast loss function to obtain the probability distribution of the intermediate positive molecule corresponding to the sample molecule.

[0057] The probability distribution of the intermediate positive molecules corresponding to the sample molecules is positively perturbed according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules.

[0058] The first loss function value is calculated according to the predicted probability distribution corresponding to each sample molecule, the hidden state corresponding to each sample molecule, the probability distribution of the intermediate positive molecules corresponding to each sample molecule, the probability distribution of the negative molecules corresponding to each sample molecule, and the probability distribution of the positive molecules corresponding to each sample molecule.

[0059] The conditional transformer model is trained according to the first loss function value to obtain a generative model.

[0060] More specifically, molecule optimization can be formulated as a seq2seq problem, where the optimization from the starting molecule to the target molecule can be understood as the translation from the initial molecule to the optimized molecule. This task is usually accomplished using a transformer (encoder-decoder architecture). i Perform encoding and decoding operations, output the predicted probability distribution corresponding to the molecule, and obtain the optimized molecule corresponding to the molecule based on the predicted probability distribution. i Convert to high-level representation H i Then it is input into the decoder, which autoregressively generates a label S of the predicted molecule each time i The operation process of the conditional Transformer (encoder-decoder architecture) can be defined as H i =f(x i ) and S i =g(H i ),in, and They represent the hidden state of an input molecule of length L and the predicted probability distribution of an output molecule of length T. f() and g() represent the encoder and decoder, respectively.

[0061] Through the decoder g(), the autoregressive decoding process can be obtained, which is recorded as p θ is the encoder-decoder model, θ is the model parameter, let S i =p θ (S i |x i ) means to convert x i The predicted probability distribution S obtained by inputting the conditional Transformer model i Given a dataset of N molecules This application uses maximum likelihood estimation To train the conditional transformer model, S i Indicates that the i-th sample molecule x i Input the conditional transformer model to obtain the predicted probability distribution corresponding to the i-th sample molecule, where N is the total number of sample molecules in the first training set. By backpropagating the sum of the loss in this step and the contrastive loss, the conditional transformer model can output the probability distribution of the optimized molecule. By mapping this distribution to the vocabulary, the corresponding optimized molecule can be generated.

[0062] like Figure 3 As shown, the molecules generated by the conditional transformer model use positive and negative perturbations to generate the probability distribution of positive and negative molecules, thereby simulating the situation in the inference stage. The following are the details of the probability distribution of generated positive and negative molecules:

[0063] 1.1 Probability distribution of generating negative molecules

[0064] (1) According to the formula Probability distribution for generating negative molecules.

[0065] in, represents the probability distribution of negative molecules corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, λ represents the negative weight, is a negative disturbance, L MLE ·||·||2 Back propagation S i The gradient of , which means back propagating the maximum likelihood estimation function S i The gradient operation of ,||||2 represents the L2 norm.

[0066] (2) The negative contrast loss L is obtained by comparing the probability distribution of negative molecules obtained by negative perturbation with the optimized molecules generated by tranformer cont- To guide the transformer model training, the contrast loss is expressed as:

[0067]

[0068] in, and S j represents the predicted probability distribution corresponding to the jth sample molecule, ∪ represents the union, S j ={S j :j≠i} and τ is a hyperparameter used to adjust the degree of similarity score smoothing. It is a projection function including a linear layer and an average pooling layer. S j is a random numerator in the batch that acts as a negative numerator, exp() represents an exponential function with e as the base, and cos() represents a cosine function. L cont- Minimization of (θ) will force the probability distribution of negative numerators to Far from the probability distribution of real molecules.

[0069] 2.1 Generate a probability distribution of positive molecules and obtain a probability distribution of positive molecules that is far away from the probability distribution of real molecules.

[0070] The generation process of the probability distribution of positive molecules is divided into two perturbations, which produce a probability distribution of positive molecules that is far away from the probability distribution of real molecules.

[0071] (1) Learn the representation of the real molecule by comparing the molecular pairs and the unpaired molecules, thereby obtaining the first gradient perturbation. Similarly, the positive contrast loss function L cont Defined as:

[0072]

[0073] Among them, L cont (θ) represents the positive contrast loss function, N represents the total number of sample molecules in the first training set, Indicates the projection of the hidden state corresponding to the i-th sample molecule, It represents the projection of the predicted probability distribution corresponding to the i-th sample molecule, τ represents the hyperparameter used to adjust the smoothness of the similarity score, Indicates S i Projection, S i Represents the predicted probability distribution corresponding to the i-th sample molecule, exp() represents the exponential function with e as the base, cos() represents the cosine function, L cont Back propagation S of (θ) i The gradient perturbation of is a positive perturbation. Therefore, the first positive perturbation can be defined as

[0074] (2) The first positive perturbation is used to generate the probability distribution of the intermediate temporary positive molecules. The probability distribution of the intermediate temporary positive molecules can be obtained according to the formula calculate, represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, μ represents the positive weight, f + Represents back propagation S of the positive contrast loss function i Gradient operation, || ||2 represents the L2 norm, since is the probability distribution of the middle positive molecule, so after the softmax layer The distribution of should be similar to the probability distribution of the real molecules. Next, this application uses KL loss to minimize the distribution between the probability distribution of the intermediate positive molecules and the probability distribution of the real molecules, defined as:

[0075]

[0076] in, D KL represents the KL divergence, which is used to measure the difference between two probability distributions, softmax() represents the activation function, W represents the weight matrix, which is usually used to map the input features or hidden states to another space, such as the probability distribution of the predicted category; b represents the bias vector, which is used to adjust the probability output of each category, S i represents the predicted probability distribution corresponding to the i-th sample molecule, μ represents the positive weight, f + Represents the back propagation S of the positive contrast loss function i Gradient operation, || ||2 represents the L2 norm, || represents the probability distribution S i and The KL divergence between the two is calculated by minimizing the KL loss to achieve the similarity between the probability distribution of positive molecules and the true molecular distribution.

[0077] (3) Since the KL loss maximizes the similarity between the probability distribution of the intermediate positive molecules and the probability distribution of the real molecules, the gradient perturbation obtained by the KL loss can also be used as the second positive perturbation to generate the probability distribution of the final positive molecules. The probability distribution of the positive molecules obtained by the second positive perturbation can be calculated according to the formula Calculated, represents the probability distribution of the positive molecule corresponding to the i-th sample molecule, represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, μ represents the positive weight, f ++ Represents the back propagation S of the KL loss function i The gradient operation of , || ||2 represents the L2 norm.

[0078] (4) The contrast loss with the probability distribution of positive molecules can be written as:

[0079]

[0080] in, It represents the projection of the set consisting of the probability distribution of the i-th sample molecule and the positive molecule corresponding to the i-th sample molecule.

[0081] (5) Finally, the parameters of the encoder-decoder can be trained by minimizing the following objective.

[0082] L=max θ L MLE (θ)-L KL (θ)+L cont -(θ)+L cont+ (θ)

[0083] Among them, L represents the first loss function, max θ L MLE (θ) represents the L MLE (θ) takes the maximum value, L MLE (θ) represents the maximum likelihood estimation function, L KL (θ) represents the KL loss function, L cont- (θ) represents the negative contrast loss function, L cont+ (θ) represents the contrast loss function with positive molecules, and θ represents the parameters of the conditional transformer model. The seq2seq model is trained using different negative and positive samples through contrastive learning, thereby alleviating the "exposure bias" problem.

[0084] In an exemplary embodiment, an optimized molecule is obtained according to a generative model, and a RNN model is trained based on the optimized molecule to obtain a distilled model based on a knowledge distillation method, specifically including:

[0085] Each sample molecule in the pre-training training set (the first training set) is fed into the generative model to generate a probability distribution corresponding to each sample molecule in the first training set. Specifically, the generative model generates a probability distribution of one million expected molecules given the target protein information. By mapping this distribution to a vocabulary, the corresponding optimized molecular composition RNN model dataset is generated.

[0086] The optimized molecules in the RNN model dataset are used as the input of the RNN model, and the RNN model is trained using maximum likelihood estimation (π μ ) to obtain the distillation model, μ is the model parameter, and the RNN model outputs a probability distribution. By mapping this distribution to the vocabulary, the corresponding optimized molecules can be generated. Given a dataset with N molecules The maximum likelihood estimation formula is: π μ (s' i |x' i ) represents the probability distribution obtained by inputting the i-th optimized molecule in the RNN model dataset into the RNN model; inputting the optimized molecule from the RNN model dataset into the RNN model outputs the probability distribution corresponding to this optimized molecule. Under this training, the distilled model cannot learn the precise likelihood function from the previous model, and due to the randomness in the decoding part of the generation process, the generated molecule set cannot be used to infer a precise likelihood function. Therefore, the distilled model provides a less concentrated chemical space for the reinforcement learning algorithm in the next stage to explore, which increases the diversity of the generated optimized molecules.

[0087] In another exemplary embodiment of the present application, the RNN model is an RNN model with three layers of gated recurrent units.

[0088] In another exemplary embodiment of the present application, a molecular optimization model is obtained by training the agent model using a reinforcement learning method based on the distillation model. This is a well-known process, and the general steps are as follows:

[0089] Step 1: Initialize the agent model

[0090] This application uses the reinforcement learning algorithm in REINVENT (Reinforcement Learning Inspired NovelVacant Engine for Targeted Design, a new vacancy engine for targeted design based on reinforcement learning) to fine-tune the distillation model and build a customized reward function for multiple goals commonly required for molecule generation in drug design. The reward functions used in this application include drug similarity score QED(m), synthetic accessibility score SAS(m) and binding affinity DOCK_score(m), and the specific form is S(A)=QED(m)+SAS(m)+DOCK_score(m). The calculation methods of QED(m), SAS(m) and DOCK_score(m) are all well known.

[0091] Step 2: The sample molecules in the first training set are input into the generation model and the distillation model respectively to obtain the probability distribution. Then, by mapping the distribution to the vocabulary, the corresponding optimized molecules can be generated to form the training set, and the RNN model is used to guide the intelligent agent model (this model uses the same RNN architecture as the distillation model) for training.

[0092] First, the intelligent agent model still generates molecules in the smiles format character by character. In this process, the generation of each token (in the SMILES string format molecule, the token represents a single character of the SMILES) is regarded as an action A under the reinforcement learning framework. After training, the intelligent agent model finally learns the conditional probability p(A|state) of the action taken under the state state containing the decoded partial sequence (referring to the incomplete molecular SMILES that has been generated). This application specifies the appropriate reward S(A) = QED(m) + SAS(m) + DOCK_score(m) for the action A in the environment based on indicators such as QED (drug similarity score), SAS (synthetic accessibility score), and binding affinity (referring to the affinity between a molecule (such as a drug molecule) and its target protein or biological target. This application scores the docking pose of the target protein and molecule by calling the smina toolkit, denoted as DOCK_score(m), where m is the incomplete molecular SMILES composed of the above action A (i.e., token).

[0093] Finally, the loss function for reinforcement learning training is defined as follows: This application freezes the distillation model (considered as the teacher model in this section) and initializes the parameters of the agent model to the parameters of the distillation model.

[0094] This application first generates a SMILES string from the agent model (generates molecules) and records the likelihood as logp agent (A). The generated SMILES string is then input into the distillation model to obtain the estimated likelihood log p middle (A). The score S(A) of each generated molecule is obtained by weighting the reward function with the coefficient. The enhanced log-likelihood is the weighted sum of the molecule score and the estimated likelihood of the generated molecule by the distillation model (teacher model), defined as logp aug (A) = log p middle (A)+αS(A). The loss function of this part is: Loss=[logp aug (A)-logp agent (A)] 2 . By backpropagating this loss, the agent model can be trained.

[0095] Step 3: Introduce experience buffer sampling to improve agent model training stability and data utilization

[0096] This application introduces an experience buffer in this section to improve data utilization, reduce the impact of data noise and improve training stability. When the number of experience tuples in the experience buffer is greater than N, this application randomly samples experience tuples from it. The experience tuple includes a SMILES string (generated molecule) sampled from the agent model, a generated molecule score and a corresponding log-likelihood function. The corresponding loss and log-likelihood obtained according to the training process of the above-mentioned agent model are spliced ​​into the overall loss and agent log-likelihood.

[0097] This application divides the molecular optimization training process into three stages:

[0098] Pre-training stage: A generative model was trained to solve the multi-objective optimization problem. This application regards the process of molecular optimization as a translation problem. The input and output of the conditional transformer model are molecules (strings) in the format of smalls. Due to the exposure bias problem of the transformer model, specifically, the input of the decoder in the training stage is the word of the real molecule, and the input of the decoder in the inference stage is the estimated word of the previous inference, and there is inconsistency between the two. This application introduces contrastive learning to avoid the exposure bias problem, generates positive and negative molecules through positive and negative perturbations, and uses the contrast loss between positive and negative molecules and generated molecules to guide model training.

[0099] Knowledge Distillation: Use the knowledge extraction model RNN to reduce the complexity of the transformer model in the previous stage so that it can be effectively fine-tuned through reinforcement learning in the next stage. Specifically, this application uses the generative model trained in the first stage to generate a certain number of optimized molecules as the training set for the RNN model in the current stage.

[0100] Reinforcement Learning: The distillation model is used as a teacher model to guide the training of the agent model in this phase. The distillation model is frozen, and the agent model is initialized using its parameters. Specifically, the optimized numerator generated by the agent model is input into the distillation model to obtain two corresponding log-likelihoods. The score of the optimized numerator is calculated based on the reward function, and the weighted sum of the optimized numerator score and the log-likelihood of the distillation model is used to obtain the enhanced log-likelihood. The difference between the log-likelihood of the agent model and the enhanced log-likelihood is used as the loss for backpropagation training of the agent model.

[0101] The following limitations exist in related molecular optimization technologies:

[0102] 1. Drug design is essentially a multi-optimization constrained process, and the use of early machine learning models to achieve molecular optimization cannot meet its multi-objective requirements.

[0103] 2. The transformer is trained by predicting masked words (these words are represented by a token), but during testing, this token is not present. This is known as an exposure bias problem. In simple terms, text generation is inconsistent between training and inference.

[0104] 3. Reinforcement learning requires a large number of optimization steps to learn knowledge through reward training, which makes reinforcement learning very inefficient.

[0105] This application is based on the generative model of conditional transformers, which reasonably solves the problem that early machine learning models cannot meet multi-objective optimization. At the same time, based on the ideas of combining contrastive learning, knowledge distillation, and reinforcement learning, this application proposes a multi-stage training method to improve the efficiency and performance of molecular optimization, and uses contrastive learning methods to solve the problem of exposure bias, so that the final generated optimized molecules meet the requirements.

[0106] Based on the same inventive concept, the present application also provides a molecular optimization device for implementing the aforementioned molecular optimization method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more molecular optimization device embodiments provided below can be found in the above-mentioned limitations of the molecular optimization method and will not be repeated here.

[0107] In an exemplary embodiment, a molecular optimization apparatus is provided, comprising:

[0108] The acquisition module is used to acquire a first training set, which includes multiple sample molecules; the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range.

[0109] A contrastive learning module is used to train the conditional transformer model using contrastive learning based on the first training set and with the goal of minimizing a first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function, and a maximum likelihood estimation function.

[0110] The knowledge distillation module is used to obtain optimized molecules according to the generative model, and train the RNN model according to the optimized molecules based on the knowledge distillation method to obtain a distilled model.

[0111] A reinforcement learning module is used to train the intelligent agent model using a reinforcement learning method based on the distillation model to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

[0112] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store molecular optimization data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a molecular optimization method is implemented.

[0113] Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned method embodiments when executing the computer program.

[0114] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above-mentioned method embodiments when executed by a processor.

[0115] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the above method embodiments are implemented.

[0116] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0117] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0118] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0119] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A molecular optimization method, characterized in that: The molecular optimization method comprises: Obtaining a first training set, the first training set comprising a plurality of sample molecules; wherein the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range; According to the first training set, the conditional transformer model is trained using contrastive learning with the goal of minimizing a first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function, and a maximum likelihood estimation function; The optimized molecule is obtained according to the generative model, and the RNN model is trained based on the optimized molecule based on the knowledge distillation method to obtain a distillation model; The agent model is trained using a reinforcement learning method according to the distillation model to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity; Based on the first training set, with the goal of minimizing the first loss function, contrastive learning is used to train the conditional transformer model to obtain a generative model, specifically including: Input each sample molecule in the first training set into the conditional transformer model to obtain the predicted probability distribution corresponding to each sample molecule; Input each sample molecule in the first training set into the encoder of the conditional transformer model to obtain the hidden state corresponding to each sample molecule; For any sample molecule in the first training set, negatively perturb the predicted probability distribution corresponding to the sample molecule according to the maximum likelihood estimation function to obtain the probability distribution of the negative molecule corresponding to the sample molecule; Performing a positive perturbation on the predicted probability distribution corresponding to the sample molecule according to a positive contrast loss function to obtain a probability distribution of an intermediate positive molecule corresponding to the sample molecule; Performing a positive perturbation on the probability distribution of the intermediate positive molecules corresponding to the sample molecules according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules; Calculate a first loss function value according to the predicted probability distribution corresponding to each sample molecule, the hidden state corresponding to each sample molecule, the probability distribution of the intermediate positive molecule corresponding to each sample molecule, the probability distribution of the negative molecule corresponding to each sample molecule, and the probability distribution of the positive molecule corresponding to each sample molecule; The conditional transformer model is trained according to the first loss function value to obtain a generative model.

2. The molecular optimization method according to claim 1, characterized in that The predicted probability distribution corresponding to the sample molecule is negatively perturbed according to the maximum likelihood estimation function to obtain the probability distribution of the negative molecule corresponding to the sample molecule: According to the formula The probability distribution of negative molecules corresponding to the sample molecules is obtained, where represents the probability distribution of negative molecules corresponding to the i-th sample molecule, S i Represents the predicted probability distribution corresponding to the i-th sample molecule, λ represents the negative weight, and g represents the back propagation S of the maximum likelihood estimation function i The gradient operation of , || ||2 represents the L2 norm.

3. The molecular optimization method according to claim 1, characterized in that The predicted probability distribution corresponding to the sample molecule is positively perturbed according to the positive contrast loss function to obtain the probability distribution of the intermediate positive molecule corresponding to the sample molecule, specifically: According to the formula The probability distribution of the intermediate positive molecules corresponding to the sample molecules is obtained, where represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, μ represents the positive weight, f + Represents back propagation S of the positive contrast loss function i The gradient operation of , || ||2 represents the L2 norm.

4. The molecular optimization method according to claim 1, characterized in that The probability distribution of the intermediate positive molecules corresponding to the sample molecules is positively perturbed according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules, specifically: According to the formula The probability distribution of positive molecules corresponding to the sample molecules is obtained, where represents the probability distribution of the positive molecule corresponding to the i-th sample molecule, represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, μ represents the positive weight, and f ++ Represents the back propagation S of the KL loss function i The gradient operation of ,||||2 represents the L2 norm.

5. The molecular optimization method according to claim 1, characterized in that The first loss function is: L=max θ L MLE (θ)-L KL (θ)+L cont- (θ)+L cont+ (θ), where L represents the first loss function, max θ L MLE (θ) represents the L MLE (θ) takes the maximum value, L MLE (θ) represents the maximum likelihood estimation function, L KL (θ) represents the KL loss function, L cont- (θ) represents the negative contrast loss function, L cont+ (θ) represents the contrast loss function with the probability distribution of positive molecules, and θ represents the parameters of the conditional transformer model; S i represents the predicted probability distribution corresponding to the i-th sample molecule, and N represents the total number of sample molecules in the first training set; Indicates the projection of the hidden state corresponding to the i-th sample molecule, Represents the projection of the predicted probability distribution corresponding to the i-th sample molecule, τ represents the hyperparameter, Indicates U i To project, S j represents the predicted probability distribution corresponding to the jth sample molecule, ∪ represents the union, represents the probability distribution of the negative molecule corresponding to the i-th sample molecule, exp() represents the exponential function with e as the base, and cos() represents the cosine function; Among them, D KL represents KL divergence, softmax() represents activation function, W represents weight matrix; b represents bias vector, μ represents positive weight, f + Represents the back propagation S of the positive contrast loss function i Gradient operation, || ||2 represents the L2 norm, Indicates S i and KL divergence calculation between; It represents the projection of the set consisting of the probability distribution of the i-th sample molecule and the positive molecule corresponding to the i-th sample molecule.

6. A molecular optimization device, characterized in that: The molecular optimization device comprises: An acquisition module is configured to acquire a first training set, wherein the first training set includes a plurality of sample molecules; the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range; A contrastive learning module is configured to train the conditional transformer model using contrastive learning based on the first training set with the goal of minimizing a first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function, and a maximum likelihood estimation function; The knowledge distillation module is used to obtain optimized molecules based on the generative model and train the RNN model based on the optimized molecules based on the knowledge distillation method to obtain a distilled model; A reinforcement learning module, configured to train the agent model using a reinforcement learning method based on the distillation model to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity; Based on the first training set, with the goal of minimizing the first loss function, contrastive learning is used to train the conditional transformer model to obtain a generative model, specifically including: Input each sample molecule in the first training set into the conditional transformer model to obtain the predicted probability distribution corresponding to each sample molecule; Input each sample molecule in the first training set into the encoder of the conditional transformer model to obtain the hidden state corresponding to each sample molecule; For any sample molecule in the first training set, negatively perturb the predicted probability distribution corresponding to the sample molecule according to the maximum likelihood estimation function to obtain the probability distribution of the negative molecule corresponding to the sample molecule; Performing a positive perturbation on the predicted probability distribution corresponding to the sample molecule according to a positive contrast loss function to obtain a probability distribution of an intermediate positive molecule corresponding to the sample molecule; Performing a positive perturbation on the probability distribution of the intermediate positive molecules corresponding to the sample molecules according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules; Calculate a first loss function value according to the predicted probability distribution corresponding to each sample molecule, the hidden state corresponding to each sample molecule, the probability distribution of the intermediate positive molecule corresponding to each sample molecule, the probability distribution of the negative molecule corresponding to each sample molecule, and the probability distribution of the positive molecule corresponding to each sample molecule; The conditional transformer model is trained according to the first loss function value to obtain a generative model.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the molecular optimization method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the molecular optimization method according to any one of claims 1 to 5 is implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the molecular optimization method according to any one of claims 1 to 5 is implemented.

Images