Molecular optimization method, device, equipment, medium and product

Abstract

The invention discloses a molecular optimization method and device, equipment, a medium and a product, and relates to the field of molecular design, and the method comprises the steps: training a conditional transformer model through comparative learning according to a first training set with a minimum first loss function as a target, and obtaining a generative model; the first loss function comprises a comparison loss function, a KL loss function and a maximum likelihood estimation function; obtaining optimized molecules according to the generation model, and training an RNN model according to the optimized molecules based on a knowledge distillation method to obtain a distillation model; according to the distillation model, adopting a reinforcement learning method to train the agent model to obtain a molecular optimization model, wherein the molecular optimization model is used for optimizing the to-be-optimized molecules; a reward function in the reinforcement learning method comprises binding affinity, the problem of exposure deviation can be solved, and finally generated optimized molecules meet requirements.

Description

Technical Field

[0001] The present application relates to the field of molecular design, and in particular to a molecular optimization method, device, equipment, medium and product. Background Art

[0002] Drug discovery often consumes a huge amount of resources and time. Therefore, the use of computer technology to assist in the discovery and design of drug molecules has important practical value. Molecular optimization is a key step in the drug design (molecule generation) process. The purpose of "molecular optimization" is to improve the desired properties of candidate drugs by chemically modifying molecules, maximizing drug properties while maintaining similarity to the input molecules. However, current molecular optimization methods generally have exposure bias problems, resulting in poor quality of the optimized molecules generated. Summary of the invention

[0003] The purpose of this application is to provide a molecular optimization method, device, equipment, medium and product, which can solve the problem of exposure deviation and make the quality of the optimized molecules finally generated better.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a molecular optimization method, comprising:

[0006] Acquire a first training set, wherein the first training set includes a plurality of sample molecules; the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range;

[0007] According to the first training set, the conditional transformer model is trained by contrastive learning with the goal of minimizing the first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function and a maximum likelihood estimation function;

[0008] The optimized molecule is obtained according to the generative model, and the RNN model is trained according to the optimized molecule based on the knowledge distillation method to obtain the distillation model;

[0009] According to the distillation model, the agent model is trained by a reinforcement learning method to obtain a molecular optimization model, and the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

[0010] Optionally, according to the first training set, with the goal of minimizing the first loss function, contrastive learning is used to train the conditional transformer model to obtain a generative model, specifically including:

[0011] Input each sample molecule in the first training set into the conditional transformer model to obtain the predicted probability distribution corresponding to each sample molecule;

[0012] Input each sample molecule in the first training set into the encoder of the conditional transformer model to obtain the hidden state corresponding to each sample molecule;

[0013] For any sample molecule in the first training set, negatively perturb the predicted probability distribution corresponding to the sample molecule according to the maximum likelihood estimation function to obtain the probability distribution of the negative molecule corresponding to the sample molecule;

[0014] Performing positive perturbations on the predicted probability distribution corresponding to the sample molecule according to the positive contrast loss function to obtain the probability distribution of the intermediate positive molecule corresponding to the sample molecule;

[0015] Performing a positive perturbation on the probability distribution of the intermediate positive molecules corresponding to the sample molecules according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules;

[0016] Calculate a first loss function value according to the predicted probability distribution corresponding to each sample molecule, the hidden state corresponding to each sample molecule, the probability distribution of the intermediate positive molecule corresponding to each sample molecule, the probability distribution of the negative molecule corresponding to each sample molecule, and the probability distribution of the positive molecule corresponding to each sample molecule;

[0017] The conditional transformer model is trained according to the first loss function value to obtain a generation model.

[0018] Optionally, negatively perturbing the predicted probability distribution corresponding to the sample molecule according to the maximum likelihood estimation function to obtain the probability distribution of negative molecules corresponding to the sample molecule is specifically:

[0019] According to the formula The probability distribution of negative molecules corresponding to the sample molecules is obtained, where: represents the probability distribution of negative molecules corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, λ represents the negative weight, and g represents the back propagation of the maximum likelihood estimation function S i The gradient operation of , ||||2 represents the L2 norm.

[0020] Optionally, the predicted probability distribution corresponding to the sample molecule is positively perturbed according to the positive contrast loss function to obtain the probability distribution of the intermediate positive molecule corresponding to the sample molecule, specifically:

[0021] According to the formula The probability distribution of the intermediate positive molecules corresponding to the sample molecules is obtained, where: represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, μ represents the positive weight, and f + Represents the back propagation of the positive contrast loss function S i The gradient operation of , ||||2 represents the L2 norm.

[0022] Optionally, the probability distribution of the intermediate positive molecules corresponding to the sample molecules is positively perturbed according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules, specifically:

[0023] According to the formula The probability distribution of the positive molecules corresponding to the sample molecules is obtained, where: represents the probability distribution of the positive molecule corresponding to the i-th sample molecule, represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, μ represents the positive weight, and f ++ Represents the back propagation of the KL loss function S i The gradient operation of , ||||2 represents the L2 norm.

[0024] Optionally, the first loss function is:

[0025] L=max θ L MLE (θ)-L KL (θ)+L cont- (θ)+L cont+ (θ), where L represents the first loss function, max θ L MLE (θ) represents the MLE (θ) takes the maximum value, L MLE (θ) represents the maximum likelihood estimation function, L KL (θ) represents the KL loss function, L cont- (θ) represents the negative contrast loss function, L cont+ (θ) represents the contrast loss function with the probability distribution of positive molecules, and θ represents the parameters of the conditional transformer model;

[0026] S i represents the predicted probability distribution corresponding to the i-th sample molecule, and N represents the total number of sample molecules in the first training set;

[0027] It represents the projection of the hidden state corresponding to the i-th sample molecule. represents the projection of the predicted probability distribution corresponding to the i-th sample molecule, τ represents the hyperparameter, Indicates U i To project, S j represents the predicted probability distribution corresponding to the jth sample molecule, ∪ represents the union, represents the probability distribution of negative molecules corresponding to the i-th sample molecule, exp() represents the exponential function with e as the base, and cos() represents the cosine function;

[0028] Among them, D KL represents KL divergence, softmax() represents activation function, W represents weight matrix; b represents bias vector, μ represents positive weight, f + Represents the back propagation of the positive contrast loss function S i Gradient operation, || ||2 represents the L2 norm, Indicates S i and KL divergence calculation between;

[0029] It represents the projection of the set consisting of the probability distribution of the i-th sample molecule and the positive molecule corresponding to the i-th sample molecule.

[0030] In a second aspect, the present application provides a molecular optimization device, comprising:

[0031] An acquisition module is used to acquire a first training set, wherein the first training set includes a plurality of sample molecules; the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range;

[0032] A contrastive learning module, used to train the conditional transformer model using contrastive learning based on the first training set and with the goal of minimizing a first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function, and a maximum likelihood estimation function;

[0033] The knowledge distillation module is used to obtain the optimized molecules according to the generation model, and train the RNN model according to the optimized molecules based on the knowledge distillation method to obtain the distillation model;

[0034] A reinforcement learning module is used to train the intelligent agent model using a reinforcement learning method according to the distillation model to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

[0035] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the molecular optimization methods described above.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the molecular optimization methods described above.

[0037] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements any of the above-mentioned molecular optimization methods when executed by a processor.

[0038] According to the specific embodiments provided in this application, this application has the following technical effects:

[0039] The present application provides a molecular optimization method, device, equipment, medium and product. In the related art, the reason why the conditional Transformer model has the exposure bias problem is that the inputs in the inference and training stages are different. In the training stage, each word input comes from a real sample; while in the inference stage, the current input uses the output of the previous word. Therefore, in order to avoid the exposure bias problem, the present application introduces a contrastive learning method, which solves the problem of different inputs in the inference and training stages, and solves the exposure bias problem, so that the optimized molecules finally generated meet the requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 This is a schematic diagram of a molecular optimization method in one embodiment of the present application;

[0042] Figure 2 A schematic diagram of a molecular optimization method provided in one embodiment of the present application;

[0043] Figure 3 Flowchart for training the conditional transformer model;

[0044] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0047] In an exemplary embodiment, a molecular optimization method is provided, such as Figure 1 and Figure 2 As shown, the following steps are included, wherein:

[0048] Step 201: Obtain a first training set, which includes multiple sample molecules; the protein ratio corresponding to each sample molecule in the first training set is within a first preset range (the protein ratio is the best), and the binding affinity is within a second preset range (the binding affinity is the best). In the related art, molecules are optimized in terms of chemical properties. When selecting a training set, the present application selects a training set according to the preset protein ratio and binding affinity requirements, and optimizes the molecules from these two aspects.

[0049] Step 202: Based on the first training set, with the goal of minimizing the first loss function, a conditional transformer model (structured as Figure 3 As shown) is trained to obtain a generation model; the first loss function includes a contrast loss function, a KL loss function and a maximum likelihood estimation function.

[0050] Step 203: Obtain optimized molecules according to the generated model, and train the RNN model according to the optimized molecules based on the knowledge distillation method to obtain a distillation model.

[0051] Step 204: According to the distillation model, the agent model is trained using a reinforcement learning method to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

[0052] In an exemplary embodiment, according to the first training set, with the goal of minimizing the first loss function, contrastive learning is used to train the conditional transformer model to obtain a generative model, specifically including:

[0053] Each sample molecule in the first training set is input into the conditional transformer model to obtain the predicted probability distribution corresponding to each sample molecule.

[0054] Each sample molecule in the first training set is input into the encoder of the conditional transformer model to obtain the hidden state corresponding to each sample molecule.

[0055] For any sample molecule in the first training set, negative perturbations are performed on the predicted probability distribution corresponding to the sample molecule according to the maximum likelihood estimation function to obtain the probability distribution of negative molecules corresponding to the sample molecule.

[0056] The predicted probability distribution corresponding to the sample molecule is positively perturbed according to the positive contrast loss function to obtain the probability distribution of the intermediate positive molecule corresponding to the sample molecule.

[0057] The probability distribution of the intermediate positive molecules corresponding to the sample molecules is positively perturbed according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules.

[0058] The first loss function value is calculated according to the predicted probability distribution corresponding to each sample molecule, the hidden state corresponding to each sample molecule, the probability distribution of the intermediate positive molecules corresponding to each sample molecule, the probability distribution of the negative molecules corresponding to each sample molecule, and the probability distribution of the positive molecules corresponding to each sample molecule.

[0059] The conditional transformer model is trained according to the first loss function value to obtain a generation model.

[0060] More specifically, molecule optimization can be formulated as a seq2seq problem, where the optimization from the starting molecule to the target molecule can be understood as the translation from the initial molecule to the optimized molecule. This task is usually done using a transformer (encoder-decoder architecture). i Perform encoding and decoding operations, output the predicted probability distribution corresponding to the molecule, and obtain the optimized molecule corresponding to the molecule based on the predicted probability distribution. i Convert to high-level representation H i Then it is input into the decoder, which autoregressively generates a label S of the predicted molecule each time i The operation process of the conditional Transformer (encoder-decoder architecture) can be defined as H i =f(x i ) and S i =g(H i ),in, and They represent the hidden state of the input molecule of length L and the predicted probability distribution of the output molecule of length T. f() and g() represent the encoder and decoder, respectively.

[0061] Through the decoder g(), the autoregressive decoding process can be obtained, which is recorded as p θ is the encoder-decoder model, θ is the model parameter, let S i =p θ (S i |x i ) means to convert x i The predicted probability distribution S obtained by inputting into the conditional Transformer model i Given a dataset with N molecules This application uses maximum likelihood estimation To train the conditional transformer model, S i Indicates that the i-th sample molecule x i The predicted probability distribution corresponding to the i-th sample molecule obtained by inputting the conditional transformer model, N represents the total number of sample molecules in the first training set. By backpropagating the sum of the loss of this step and the contrast loss, the conditional transformer model can output the probability distribution of the optimized molecule, and by mapping this distribution to the vocabulary, the corresponding optimized molecule can be generated.

[0062] like Figure 3 As shown, positive and negative perturbations are used to generate the probability distribution of positive and negative molecules for the molecules generated by the conditional transformer model to simulate the situation in the inference stage. The following are the details of the probability distribution of positive and negative molecules:

[0063] 1.1 Probability distribution of generating negative molecules

[0064] (1) According to the formula Probability distribution for generating negative molecules.

[0065] in, represents the probability distribution of negative molecules corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, λ represents the negative weight, is a negative disturbance, L MLE ·||·||2 Back propagation S i The gradient of , which means back propagating the maximum likelihood estimation function S i The gradient operation of , ||||2 represents the L2 norm.

[0066] (2) The negative contrast loss L is obtained by comparing the probability distribution of negative molecules obtained by negative perturbation with the optimized molecules generated by tranformer cont- To guide the transformer model training, the contrast loss is expressed as:

[0067]

[0068] in, and S j represents the predicted probability distribution corresponding to the jth sample molecule, ∪ represents the union, S j ={S j :j≠i} and τ is a hyperparameter used to adjust the degree of similarity score smoothing. is a projection function consisting of a linear layer and an average pooling layer. S j is a random numerator in the batch that acts as a negative numerator, exp() represents an exponential function with e as the base, and cos() represents a cosine function. L cont- Minimization of (θ) will force the probability distribution of negative numerators to Far from the probability distribution of real molecules.

[0069] 2.1 Generate a probability distribution of positive molecules and obtain a probability distribution of positive molecules that is far from the probability distribution of real molecules.

[0070] The generation process of the probability distribution of positive molecules is divided into two perturbations, which produce the probability distribution of positive molecules far away from the probability distribution of real molecules.

[0071] (1) Learn the representation of the true molecule by comparing the molecular pairs and the unpaired molecules, thereby obtaining the first gradient perturbation. Similarly, the positive contrast loss function L cont Defined as:

[0072]

[0073] Among them, L cont (θ) represents the positive contrast loss function, N represents the total number of sample molecules in the first training set, It represents the projection of the hidden state corresponding to the i-th sample molecule. represents the projection of the predicted probability distribution corresponding to the i-th sample molecule, τ represents a hyperparameter used to adjust the degree of smoothing of the similarity score, Indicates S i Projection, S i represents the predicted probability distribution corresponding to the i-th sample molecule, exp() represents the exponential function with e as the base, cos() represents the cosine function, L cont Back propagation S of (θ) i The gradient perturbation of is a positive perturbation. Therefore, the first positive perturbation can be defined as

[0074] (2) The first positive perturbation is used to generate the probability distribution of the intermediate temporary positive molecules. The probability distribution of the intermediate temporary positive molecules can be obtained according to the formula calculate, represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, μ represents the positive weight, and f + Represents the back propagation of the positive contrast loss function S i Gradient operation, || ||2 represents the L2 norm, since is the probability distribution of the middle positive molecule, so after the softmax layer The distribution of should be similar to the probability distribution of the real molecules. Next, this application uses KL loss to minimize the distribution between the probability distribution of the intermediate positive molecules and the probability distribution of the real molecules, defined as:

[0075]

[0076] in, D KL represents KL divergence, which is used to measure the difference between two probability distributions. Softmax() represents the activation function. W represents the weight matrix, which is usually used to map input features or hidden states to another space, such as the probability distribution of the predicted category. b represents the bias vector, which is used to adjust the probability output of each category. S i represents the predicted probability distribution corresponding to the i-th sample molecule, μ represents the positive weight, and f + Represents the back propagation of the positive contrast loss function S i The gradient operation, || ||2 represents the L2 norm, and || represents the probability distribution S i and The KL divergence between is calculated by minimizing the KL loss to achieve the similarity between the probability distribution of positive molecules and the true molecular distribution.

[0077] (3) Since the KL loss maximizes the similarity between the probability distribution of the intermediate positive molecules and the probability distribution of the real molecules, the gradient perturbation obtained by the KL loss can also be used as the second positive perturbation to generate the probability distribution of the final positive molecules. The probability distribution of the positive molecules obtained by the second positive perturbation can be obtained according to the formula Calculated, represents the probability distribution of the positive molecule corresponding to the i-th sample molecule, represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, μ represents the positive weight, f ++ Represents the back propagation of the KL loss function S i The gradient operation of , || ||2 represents the L2 norm.

[0078] (4) The contrast loss with the probability distribution of positive molecules can be written as:

[0079]

[0080] in, It represents the projection of the set consisting of the probability distribution of the i-th sample molecule and the positive molecule corresponding to the i-th sample molecule.

[0081] (5)Finally, the parameters of the encoder-decoder can be trained by minimizing the following objective:

[0082] L=max θ L MLE (θ)-L KL (θ)+L cont -(θ)+L cont+ (θ)

[0083] Among them, L represents the first loss function, max θ L MLE (θ) represents the MLE (θ) takes the maximum value, L MLE (θ) represents the maximum likelihood estimation function, L KL (θ) represents the KL loss function, L cont- (θ) represents the negative contrast loss function, L cont+ (θ) represents the contrast loss function with positive molecules, and θ represents the parameters of the conditional transformer model. The seq2seq model is trained using different negative and positive samples through contrastive learning, thereby alleviating the “exposure bias” problem.

[0084] In an exemplary embodiment, an optimized molecule is obtained according to the generative model, and a RNN model is trained according to the optimized molecule based on a knowledge distillation method to obtain a distillation model, specifically including:

[0085] Each sample molecule in the training set (first training set) of the pre-training stage is input into the generative model to obtain the probability distribution corresponding to each sample molecule in the first training set. Specifically, the generative model is used to generate the probability distribution of one million expected molecules given the target protein information, and the corresponding optimized molecular composition RNN model data set can be generated by mapping the distribution to the vocabulary.

[0086] The optimized molecules in the RNN model dataset are used as the input of the RNN model, and the RNN model is trained using maximum likelihood estimation (π μ ) to get the distillation model, μ is the model parameter, and the RNN model outputs the probability distribution. By mapping the distribution to the vocabulary, the corresponding optimized molecules can be generated. Given a data set with N molecules The maximum likelihood estimation formula is: π μ (s' i |x' i ) represents the probability distribution obtained by inputting the i-th optimized molecule in the RNN model dataset into the RNN model; the optimized molecule of the RNN model dataset is input into the RNN model, and the probability distribution corresponding to this optimized molecule is output. Under this training, the distillation model cannot learn the exact likelihood function from the previous model, and due to the randomness in the decoding part of the generation process, the generated molecule set cannot be used to infer the exact likelihood function. Therefore, the distilled model provides a less concentrated chemical space for the reinforcement learning algorithm in the next stage to explore, which improves the diversity of the generated optimized molecules.

[0087] In another exemplary embodiment of the present application, the RNN model is an RNN model with three layers of gated recurrent units.

[0088] In another exemplary embodiment of the present application, the agent model is trained by a reinforcement learning method according to the distillation model to obtain a molecular optimization model, which is a well-known process. The general steps are:

[0089] Step 1: Initialize the agent model

[0090] This application uses the reinforcement learning algorithm in REINVENT (Reinforcement Learning Inspired NovelVacant Engine for Targeted Design, a targeted design novel vacancy engine based on reinforcement learning) to fine-tune the distillation model, and builds a customized reward function for multiple goals commonly required for molecule generation in drug design. The reward functions used in this application include the drug similarity score QED(m), the synthetic accessibility score SAS(m), and the binding affinity DOCK_score(m), and the specific form is S(A)=QED(m)+SAS(m)+DOCK_score(m). The calculation methods of QED(m), SAS(m), and DOCK_score(m) are all well known.

[0091] Step 2: Input the sample molecules in the first training set into the generation model and the distillation model respectively to obtain the probability distribution. Then, by mapping the distribution to the vocabulary, the corresponding optimized molecules can be generated to form the training set, and the RNN model is used to guide the intelligent agent model (this model uses the same RNN architecture as the distillation model) for training.

[0092] First, the agent model still generates smiles format molecules character by character. In this process, the generation of each token (in the SMILES string format molecule, the token represents a single character of the SMILES) is regarded as an action A under the reinforcement learning framework. After training, the agent model finally learns the conditional probability p(A|state) of the action taken under the state state containing the decoded partial sequence (referring to the incomplete molecular SMILES that has been generated). This application specifies the appropriate reward S(A) = QED(m) + SAS(m) + DOCK_score(m) for the action A in the environment based on indicators such as QED (drug similarity score), SAS (synthetic accessibility score), binding affinity (referring to the affinity between a molecule (such as a drug molecule) and its target protein or biological target, which is obtained by calling the smina toolkit to score the docking posture of the targeted protein and molecule, denoted as DOCK_score(m), where m is the incomplete molecular SMILES composed of the above action A (i.e., token)).

[0093] Finally, the loss function for reinforcement learning training is defined as follows: This application freezes the distillation model (regarded as the teacher model in this section) and initializes the parameters of the agent model to the parameters of the distillation model.

[0094] This application first generates a SMILES string (generated molecule) from the agent model and records the likelihood as logp agent (A). The generated SMILES string is then input into the distillation model to obtain the estimated likelihood log p middle (A). The score S(A) of each generated molecule is obtained by weighting the reward function with the coefficient. The enhanced log-likelihood is the weighted sum of the molecule score and the estimated likelihood of the generated molecule by the distillation model (teacher model), defined as logp aug (A) = log p middle (A)+αS(A). The loss function of this part is: Loss=[logp aug (A)-logp agent (A)] 2 . By back-propagating the loss, the agent model can be trained.

[0095] Step 3: Introduce experience buffer sampling to improve the stability of agent model training and data utilization

[0096] The present application introduces an experience buffer in this section to improve data utilization, reduce the impact of data noise and improve training stability. When the number of experience tuples in the experience buffer is greater than N, the present application randomly samples experience tuples therefrom. The experience tuple includes a SMILES string (generated molecule), a generated molecule score and a corresponding log-likelihood function sampled from the agent model. The corresponding loss and log-likelihood obtained according to the training process of the above-mentioned agent model are spliced ​​into the overall loss and agent log-likelihood.

[0097] This application divides the molecular optimization training process into three stages:

[0098] Pre-training stage: A generative model was trained to solve the multi-objective optimization problem. This application regards the process of molecular optimization as a translation problem. The input and output of the conditional transformer model are molecules (strings) in smalls format. Due to the exposure bias problem of the transformer model, specifically, the input of the decoder in the training stage is the word of the real molecule, and the input of the decoder in the reasoning stage is the estimated word of the previous inference, and there is an inconsistency between the two. This application introduces contrastive learning to avoid the exposure bias problem, generates positive and negative molecules through positive and negative perturbations, and uses the contrast loss between positive and negative molecules and generated molecules to guide model training.

[0099] Knowledge distillation stage: Use the knowledge extraction model RNN to reduce the complexity of the transformer model in the previous stage so that it can be effectively fine-tuned through reinforcement learning in the next stage. Specifically, this application uses the generative model trained in the first stage to generate a certain number of optimized molecules as the training set of the RNN model in the current stage.

[0100] Reinforcement learning stage: The distillation model is used as a teacher model to guide the training of the agent model in this stage, the distillation model is frozen, and the agent model is initialized using the parameters of the distillation model. Specifically, the optimized molecule generated by the agent model is input into the distillation model to obtain two corresponding log-likelihoods. The score of the optimized molecule is calculated according to the reward function, and the weighted sum of the optimized molecule score and the log-likelihood of the distillation model is obtained to obtain the enhanced log-likelihood. The difference between the log-likelihood of the agent model and the enhanced log-likelihood is used as the loss back-propagation to train the agent model.

[0101] The following limitations exist in related molecular optimization techniques:

[0102] 1. Drug design is essentially a multi-optimization constraint process, and the use of early machine learning models to achieve molecular optimization cannot meet its multi-objective requirements.

[0103] 2. The transformer task is trained by predicting masked words (these words are represented by a token), but when testing, this token does not exist. This is the exposure bias problem. In simple terms, text generation is inconsistent during training and inference.

[0104] 3. Reinforcement learning requires a large number of optimization steps to learn knowledge through reward training, which makes reinforcement learning very inefficient.

[0105] This application is based on the generative model of conditional transformers, which reasonably solves the problem that early machine learning models cannot meet multi-objective optimization. At the same time, based on the ideas of combining contrastive learning, knowledge distillation, and reinforcement learning, this application proposes a multi-stage training method to improve the efficiency and performance of molecular optimization, and uses contrastive learning methods to solve the problem of exposure bias, so that the optimized molecules finally generated meet the requirements.

[0106] Based on the same inventive concept, the embodiment of the present application also provides a molecular optimization device for implementing the molecular optimization method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more molecular optimization device embodiments provided below can refer to the limitations of the molecular optimization method above, and will not be repeated here.

[0107] In an exemplary embodiment, a molecular optimization apparatus is provided, comprising:

[0108] The acquisition module is used to acquire a first training set, wherein the first training set includes a plurality of sample molecules; the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range.

[0109] The contrastive learning module is used to train the conditional transformer model using contrastive learning based on the first training set and with the goal of minimizing the first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function and a maximum likelihood estimation function.

[0110] The knowledge distillation module is used to obtain optimized molecules according to the generation model, and train the RNN model according to the optimized molecules based on the knowledge distillation method to obtain a distillation model.

[0111] A reinforcement learning module is used to train the intelligent agent model using a reinforcement learning method according to the distillation model to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

[0112] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store molecular optimization data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a molecular optimization method is implemented.

[0113] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned method embodiments when executing the computer program.

[0114] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0115] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0116] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0117] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0118] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0119] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A molecular optimization method, characterized in that: The molecular optimization method comprises: Acquire a first training set, wherein the first training set includes a plurality of sample molecules; the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range; According to the first training set, the conditional transformer model is trained by contrastive learning with the goal of minimizing the first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function and a maximum likelihood estimation function; The optimized molecule is obtained according to the generative model, and the RNN model is trained according to the optimized molecule based on the knowledge distillation method to obtain the distillation model; According to the distillation model, the agent model is trained by a reinforcement learning method to obtain a molecular optimization model, and the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

2. The molecular optimization method according to claim 1, characterized in that: According to the first training set, the conditional transformer model is trained by contrastive learning with the goal of minimizing the first loss function to obtain a generative model, which specifically includes: Input each sample molecule in the first training set into the conditional transformer model to obtain the predicted probability distribution corresponding to each sample molecule; Input each sample molecule in the first training set into the encoder of the conditional transformer model to obtain the hidden state corresponding to each sample molecule; For any sample molecule in the first training set, negatively perturb the predicted probability distribution corresponding to the sample molecule according to the maximum likelihood estimation function to obtain the probability distribution of the negative molecule corresponding to the sample molecule; Performing positive perturbations on the predicted probability distribution corresponding to the sample molecule according to the positive contrast loss function to obtain the probability distribution of the intermediate positive molecule corresponding to the sample molecule; Performing a positive perturbation on the probability distribution of the intermediate positive molecules corresponding to the sample molecules according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules; Calculate a first loss function value according to the predicted probability distribution corresponding to each sample molecule, the hidden state corresponding to each sample molecule, the probability distribution of the intermediate positive molecule corresponding to each sample molecule, the probability distribution of the negative molecule corresponding to each sample molecule, and the probability distribution of the positive molecule corresponding to each sample molecule; The conditional transformer model is trained according to the first loss function value to obtain a generation model.

3. The molecular optimization method according to claim 2, characterized in that: The predicted probability distribution corresponding to the sample molecule is negatively disturbed according to the maximum likelihood estimation function to obtain the probability distribution of the negative molecule corresponding to the sample molecule: According to the formula The probability distribution of negative molecules corresponding to the sample molecules is obtained, where: represents the probability distribution of negative molecules corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, λ represents the negative weight, and g represents the back propagation of the maximum likelihood estimation function S i The gradient operation of , || ||2 represents the L2 norm.

4. The molecular optimization method according to claim 2, characterized in that: The predicted probability distribution corresponding to the sample molecule is positively perturbed according to the positive contrast loss function to obtain the probability distribution of the intermediate positive molecule corresponding to the sample molecule, specifically: According to the formula The probability distribution of the intermediate positive molecules corresponding to the sample molecules is obtained, where: represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, S i represents the predicted probability distribution corresponding to the i-th sample molecule, μ represents the positive weight, and f + Represents the back propagation of the positive contrast loss function S i The gradient operation of , || ||2 represents the L2 norm.

5. The molecular optimization method according to claim 2, characterized in that: The probability distribution of the intermediate positive molecules corresponding to the sample molecules is positively perturbed according to the KL loss function to obtain the probability distribution of the positive molecules corresponding to the sample molecules, specifically: According to the formula The probability distribution of the positive molecules corresponding to the sample molecules is obtained, where: represents the probability distribution of the positive molecule corresponding to the i-th sample molecule, represents the probability distribution of the intermediate positive molecule corresponding to the i-th sample molecule, μ represents the positive weight, and f ++ Represents the back propagation of the KL loss function S i The gradient operation of , || ||2 represents the L2 norm.

6. The molecular optimization method according to claim 2, characterized in that: The first loss function is: L=max θ L MLE (θ)-L KL (θ)+L cont- (θ)+L cont+ (θ), where L represents the first loss function, max θ L MLE Indicates L MLE (θ) takes the maximum value, L MLE (θ) represents the maximum likelihood estimation function, L KL (θ) represents the KL loss function, L cont- (θ) represents the negative contrast loss function, L cont+ (θ) represents the contrast loss function with the probability distribution of positive molecules, and θ represents the parameters of the conditional transformer model; S i represents the predicted probability distribution corresponding to the i-th sample molecule, and N represents the total number of sample molecules in the first training set; It represents the projection of the hidden state corresponding to the i-th sample molecule. represents the projection of the predicted probability distribution corresponding to the i-th sample molecule, τ represents the hyperparameter, Indicates U i To project, S j represents the predicted probability distribution corresponding to the jth sample molecule, ∪ represents the union, represents the probability distribution of negative molecules corresponding to the i-th sample molecule, exp() represents the exponential function with e as the base, and cos() represents the cosine function; Among them, D KL represents KL divergence, softmax() represents activation function, W represents weight matrix; b represents bias vector, μ represents positive weight, f + Represents the back propagation of the positive contrast loss function S i Gradient operation, || ||2 represents the L2 norm, Indicates S i and KL divergence calculation between; It represents the projection of the set consisting of the probability distribution of the i-th sample molecule and the positive molecule corresponding to the i-th sample molecule.

7. A molecular optimization device, characterized in that: The molecular optimization device comprises: An acquisition module is used to acquire a first training set, wherein the first training set includes a plurality of sample molecules; the protein ratio corresponding to each sample molecule in the first training set is within a first preset range, and the binding affinity is within a second preset range; A contrastive learning module, used to train the conditional transformer model using contrastive learning based on the first training set and with the goal of minimizing a first loss function to obtain a generative model; the first loss function includes a contrastive loss function, a KL loss function, and a maximum likelihood estimation function; The knowledge distillation module is used to obtain the optimized molecules according to the generation model, and train the RNN model according to the optimized molecules based on the knowledge distillation method to obtain the distillation model; A reinforcement learning module is used to train the intelligent agent model using a reinforcement learning method according to the distillation model to obtain a molecular optimization model, wherein the molecular optimization model is used to optimize the molecules to be optimized; the reward function in the reinforcement learning method includes binding affinity.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the molecular optimization method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the molecular optimization method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the molecular optimization method according to any one of claims 1 to 6 is implemented.

Images