Optimization method of fragment connection small molecule compound based on deep reinforcement learning and curriculum learning
Through deep reinforcement learning and course learning, the generation of small-molecule compounds connected by fragments is solved, and the problem of difficult to take into account both the legitimacy, authenticity and desired attributes in drug molecule design in the prior art is achieved, and efficient generation of drug molecules with complex attributes is achieved.
Patent Information
- Application Number
- CN202410983424.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-22
AI Technical Summary
Existing deep learning models are difficult to balance legitimacy, authenticity and desired attributes in drug molecule design, especially when the target attributes are complex, it is difficult to calculate and generate effective drug molecules.
Using a method based on deep reinforcement learning and course learning, the generation of small molecule compounds of fragment connection is optimized by establishing a supervised training set and a priori model based on graph neural networks, combining reinforcement learning and course learning frameworks to optimize the generation of fragment-connected small molecule compounds to ensure the legitimacy of the generated molecules. , authenticity and expected attributes.
It effectively solves the problems of ineffective molecules and non-real structures, reduces the computational cost of generating complex attribute drug molecules, and improves the generation ability of the model and the drug properties and synthesis feasibility of drug molecules.
Smart Images

Figure CN119069034A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of artificial intelligence and drug molecule optimization, and more specifically, to an optimization method for fragment-connected small molecule compounds based on deep reinforcement learning and curriculum learning. Background Art
[0002] Fragment-Based Drug Discovery (FBDD) starts with the screening of low molecular weight fragments and then proceeds to molecular optimization. It has attracted widespread attention in the pharmaceutical industry due to its low screening cost and ability to efficiently identify lead compounds. In FBDD, a useful molecular optimization strategy is fragment connection, which connects two molecular fragments that bind to different but adjacent sub-pockets through appropriate connectors. Traditional computational connector design methods are based on fragment database searches combined with physical simulations to prioritize subsets of linked compounds. These methods are inherently limited by the small chemical space of the fragment database used and the computational complexity of physical simulations. Recently, the development of generative methods based on deep learning has enabled the exploration of larger chemical spaces for connector design in a data-driven manner.
[0003] In deep learning-based model approaches, various generative architectures, such as recurrent neural networks (RNNs), transformer neural networks, and variable autoencoders (VAEs), have been proposed to generate desired molecules for de novo drug design and lead compound optimization, and these generative strategies have also been effectively used for linker design in FBDD. Yang et al. proposed a SMILES-based model, SyntaLinker, which treats fragment linking as a sentence completion task through a conditional transformer neural network and generates complete molecules with linker length constraints; Imrie et al. developed a graph-based model, DeLinker, which explicitly incorporates 3D structural information through the relative orientation and distance between two fragments during linker design. E(3)-isovariant linker design models, including 3DLinker developed by Huang et al. and DiffLinker developed by Igashov et al., whose performance is independent of the initial coordinates, have been developed to generate molecular linkers that preserve the spatial configuration of the two fragments. Compared with traditional computational methods, these deep learning models are able to search a larger chemical space and generate more diverse linkers.
[0004] Although these deep generative models have achieved good results in generating linkers after training on datasets such as ChEMBL and ZINC, many methods are unable to design drug molecules for specific targets or specific target properties. For example, the SMILES generated by models such as SyntaLinker on the ChEMBL dataset have only extremely low legitimacy and cannot obtain correct linkers at all; while models such as DeLinker are trained in a graph-based manner, and their legitimacy is guaranteed, but the generated linkers are far from the real linker structure. At the same time, this type of pre-trained model learns the distribution of molecules in the training set, so the properties of the generated linkers can only be close to the training set, and it is impossible to generate molecules with any desired properties, thus failing to solve some problems such as the drugability and synthetic feasibility of drug molecules. For some models that have introduced reinforcement learning technology, although it can alleviate the problem of generating molecules with target attributes, when the target attributes are too complex, the computational cost of reinforcement learning is high, and it may even fail to reach a convergence state. Summary of the invention
[0005] In order to overcome the defects of the above-mentioned prior art that the legality, authenticity and expected properties cannot be taken into account at the same time, the present invention provides an optimization method for fragment-linked small molecule compounds based on deep reinforcement learning and curriculum learning.
[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:
[0007] In the first aspect, an optimization method for fragment-connected small molecule compounds based on deep reinforcement learning and curriculum learning comprises:
[0008] Establishing a supervised training set; wherein the supervised training set includes a plurality of fragment molecule quadruples and decoding routes in the form of molecular graph representation;
[0009] Establishing an initial graph neural network-based prior model, and using the supervised training set to determine a number of prior models for different drug design tasks;
[0010] Under the framework of curriculum learning, the prior model is used for reinforcement learning to obtain a production model with target attributes;
[0011] The production model is used for large-scale generation of molecules.
[0012] In a second aspect, an electronic device includes:
[0013] A memory for storing computer executable instructions or computer programs;
[0014] The processor is used to implement the method described in the first aspect when executing the computer executable instructions or computer program stored in the memory.
[0015] In a third aspect, a computer program product comprises a computer program or computer executable instructions, wherein when the computer program or computer executable instructions are executed by a processor, the method described in the first aspect is implemented.
[0016] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:
[0017] The present application discloses an optimization method for fragment-connected small molecule compounds based on deep reinforcement learning and curriculum learning. By training a prior model based on a graph neural network on a supervised training set, the problem of previous models generating a large number of invalid molecules and non-real structure compounds is solved, thereby ensuring the basic generation ability of the model. On this basis, reinforcement learning is introduced to flexibly change the model generation distribution, so that the model generates drug molecules with desired properties. In addition, by introducing a curriculum learning framework, the shortcomings of reinforcement learning in generating complex target attribute tasks are compensated, so that the model can generate drug molecules with complex properties at a relatively low computational cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 This is a schematic diagram of the process described in Example 1 of the present application;
[0019] Figure 2 This is another schematic flow chart of the method described in Example 1 of the present application. DETAILED DESCRIPTION
[0020] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged in appropriate cases, which is only to describe the distinction mode adopted by the objects of the same attributes in the embodiments of the present application when describing. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment containing a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment. The term "determine" widely covers various actions, and may include acquisition, calculation, calculation, processing, derivation, investigation, search (for example, search in a table, database or other data structure), ascertainment, and similar actions, and may also include reception (for example, receiving information), access (for example, accessing data in a memory) and similar actions, and may also include generation, creation, establishment and similar actions, as well as parsing, selection, selection and similar actions, etc. The relevant definitions of other terms will be given in the following description.
[0021] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;
[0022] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;
[0023] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0024] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0025] Example 1
[0026] This embodiment provides an optimization method for fragment-linked small molecule compounds based on deep reinforcement learning and curriculum learning, see Figure 1 ,include:
[0027] Establishing a supervised training set; wherein the supervised training set includes a plurality of fragment molecule quadruples and decoding routes in the form of molecular graph representation;
[0028] Establishing an initial graph neural network-based prior model, and using the supervised training set to determine a number of prior models for different drug design tasks;
[0029] Under the framework of Curriculum Learning, the prior model is used for reinforcement learning to obtain a production model with target attributes;
[0030] The production model is used for large-scale generation of molecules.
[0031] This embodiment trains a priori model based on graph neural network on the supervised training set, so that the model can learn the action information of molecular combination based on molecular graph and decoding route, solves the problem that the previous model generates more invalid molecules and non-real structure compounds, and ensures the basic generation ability of the model; on this basis, reinforcement learning is introduced to flexibly change the model generation distribution, so that the model generates drug molecules with desired properties; and by introducing a course learning framework, the shortcomings of reinforcement learning in generating complex target attribute tasks are compensated, so that the model generates drug molecules with complex properties at a lower computational cost. Using the production model, technicians in this field can generate connectors similar to real structures, and secondly, they can generate drug molecules that meet the expected properties according to predefined target attributes.
[0032] In some preferred embodiments, the process of establishing the supervised training set includes:
[0033] Obtain data on several bioactive molecules with drug-like properties and filter them based on the Lipinski RO5 (“Rule of Five”) rule, the pan-assay interference compound substructure filter PAINS, and / or the synthetic accessibility SA score to ensure that the generated data has high drug-like properties and can be synthesized;
[0034] For the bioactive molecular data of drug-like properties after screening, the fragment molecule quadruple (FMQ) is constructed in the form of "first fragment, connector, second fragment, molecule" by using the matching molecular pair fragment algorithm (MMP);
[0035] The fragment molecular quadruplets are filtered using the Lipinski RO3 ("rule of three") rule constraint condition and uniformly converted into a molecular graph representation, which is recorded as the first molecular graph;
[0036] Performing graph traversal on all the first molecular graphs, and gradually segmenting each of the first molecular graphs to obtain subgraphs and corresponding action probability distributions, until the decoded subgraph is an empty graph;
[0037] Based on the subgraph and the action probability distribution, a corresponding decoding route of the first subgraph is constructed, thereby completing the establishment of the supervised training set.
[0038] Those skilled in the art should understand that the action probability distribution consists of a series of action probabilities.
[0039] In some specific implementations, the ChEMBL database is used to build a supervised training set, see Figure 2 :
[0040] The dataset was extracted from the ChEMBL database and filtered using Lipinski's RO5, pan-assay interfering compound substructure filter, and synthetic accessibility score; after screening, the MMP algorithm was applied to construct fragment-molecule quadruplets (FMQs) in the form of "fragment 1, connector, fragment 2, molecule", and a total of 5,873,503 FMQs were obtained; then, the FMQs were filtered using the RO3 constraint, and about 718,652 pairs of terminal fragments and original molecules were retained as fragment-molecule triplets (FMTs), all of which were uniformly converted from the standardized SMILES format to molecular graph representations, such as adjacency tensors and node feature matrices. The filtered dataset was further divided into three groups in a ratio of 8:1:1 for training, validation, and testing of the prior model Prior.
[0041] The preprocessing of training data will be regarded as a graph traversal problem. The molecular graph G with n nodes is n will be segmented step by step to obtain the decoding route r of the molecular graph. n In the graph traversal, it is divided into Gn-1 When , a corresponding action probability APD will be obtained n-1 , until an empty graph G0 is decoded, and APD0 is obtained accordingly. In this way, the decoding route r of the molecular graph can be expressed as:
[0042] r={(G0,APD0),…,(G n-1 ,APD n-1 ),(G n ,APD n )}
[0043] This operation will be repeated for all molecules in the same batch, and finally stored in the hdf file for subsequent data reading and use, thus completing the establishment of the supervised data set.
[0044] In some other specific implementations, the ZINC database is used to establish a supervised training set.
[0045] In some preferred embodiments, the method of using the supervised training set to determine a plurality of prior models for different drug design tasks includes:
[0046] The prior model is trained on the supervised training set using a back propagation algorithm until the value of the loss function converges or reaches a preset training time, so that the prior model learns the correct action information of the molecular combination; wherein the loss function is the KL (Kullback-Leibler) divergence between the action probability of the real molecule and the action probability predicted by the prior model.
[0047] As non-limiting examples, the different drug design tasks may be high synthetic feasibility, high drug-likeness, high 2D similarity molecules, or high biological activity. It should be understood by those skilled in the art that different drug design tasks can be distinguished according to different training sets, for example, first processing the data set clustered into high synthetic feasibility for model training.
[0048] In some specific implementations, the prior models for different drug design tasks will be trained separately under the Pytorch framework, and the loss function is set as:
[0049]
[0050] Where APD init Represents the initial value of the action probability; APD pred Represents the predicted value of the action probability.
[0051] During training, the back-propagation algorithm is used to continuously update the model's parameter values. After iterating a specific Epoch, the verification loss value of the prior model will be output. When the loss value decreases and converges, the model is considered stable.
[0052] In some preferred embodiments, the prior model includes a GGNN block and a global readout block; wherein,
[0053] The GGNN block is composed of a GGNN (Gated Graph Sequence Neural Networks) model, which is used to receive adjacency tensors and node feature matrices about molecular graph representation data, and output corresponding node feature vectors and graph embedding information;
[0054] The global readout block is used to calculate the corresponding action probability (APD) according to the node feature vector and the graph embedding information to guide the model on how to form a graph.
[0055] In some optional embodiments, the global readout block is composed of multiple multi-layer perceptrons MLP; the action probability APD is calculated by the following formula:
[0056] f′ add =MLP add,1 (H L )
[0057] f′ conn =MLP conn,1 (H L )
[0058] f add =MLP add,2 ([f′ add ,g])
[0059] f conn =MLP conn,2 ([f′ conn ,g])
[0060] f term =MLP term,2 (g)
[0061] APD=softmax([f add ,f conn ,f term ])
[0062] Wherein, H represents the node feature vector output by the GGNN block; g represents the graph embedding information output by the GGNN block; f add represents the probability of adding a new node, i.e., adding an atom, to the molecular graph output by MLP; f connrepresents the probability of connecting the last attached node in the molecular graph to another existing node in the graph, i.e., adding a bond, output by the MLP; f term represents the probability of terminating the molecular graph; f′ add and f′ conn Indicates intermediate output.
[0063] It should be noted that the process of model generation graph (i.e., the actions that can be taken) can be basically divided into three steps: adding atoms (add), adding bonds (conn, i.e., bond connection) and termination judgment (term, i.e., sampling termination). These three steps are all based on the global readout module completing the action probability APD calculation in the previous step to determine whether to continue.
[0064] More specifically, f add With f conn Generated using a two-layer MLP network.
[0065] In some preferred embodiments, the reinforcement learning using the prior model under the curriculum learning framework includes:
[0066] The agent model (intelligent agent) is initialized using the prior model Prior, and the preset drug design task C is decomposed into at least one simple learning stage goal O c and production stage target O p ; Wherein, if and only if the final learning stage goal is met, the course learning of the production stage goal is entered to generate molecules;
[0067] For each learning stage goal or production stage goal, the agent model is trained using a reinforcement learning strategy, and a scoring function is introduced to score the generation result of the agent model until the scoring result is not less than the preset course stage threshold or reaches the preset training time, and the agent model enters the training of the next stage goal or ends the training; wherein the progress of the course learning is controlled by the course learning progress P, P = {P1, ..., P n-1 ,P n ), so that C = {O, P},
[0068] It should be emphasized that if and only if the final course progress standard P is met n When the course is completed, the production phase of the course will begin. p To generate molecules, the agent samples compounds in favorable regions of chemical space that meet the production goal.
[0069] In some optional embodiments, the reinforcement learning process includes:
[0070] Using the fragment molecules that need to generate the connection substance as the input of the Agent model, and performing batch generation to obtain a plurality of the generation results;
[0071] Introducing a preset scoring function to score the generation result of the Agent model;
[0072] Introducing a policy gradient method of reinforcement learning to update the Agent model so that the scoring gradient can be conducted, with the goal of maximizing the scoring result for the generated result;
[0073] The updating step is repeated until the scoring results corresponding to the generation results generated in batches are no longer improved or the updating is stopped after reaching a preset training time, and the updated Agent model is used as the production model.
[0074] It should be emphasized that by introducing the scoring function and the policy gradient method, the model generation distribution can be flexibly changed so that the model generates drug molecules with desired properties.
[0075] It should be understood that the trained prior model Prior learns the strategy of action probability sampling distribution (grammar) and compound molecular structure (referred to as "old strategy"). By initializing the Agent model to Prior, the Agent model learns the old strategy, which enables the reinforcement learning model (Agent model) to master the prior knowledge and update the strategy on this basis.
[0076] It should also be understood that in model training, the preset training duration can be expressed as a preset number of training steps.
[0077] It should be noted that the scoring function is formulated by those skilled in the art according to needs. In some specific implementations, if it is necessary to improve the drugability of the generated drug molecules, the scoring function is formulated as follows:
[0078]
[0079] In the formula, is the sequence generated by the prior model, Represents the property value of drug molecule; Target represents the target value; α is a custom parameter.
[0080] Furthermore, the policy gradient method comprises:
[0081] The action probability sampling distribution learned by the Agent model is used as a strategy. For the molecular graph generated by sampling, the product of the sampling probabilities represents the likelihood of the action probability sampling of the model, which is formally:
[0082]
[0083] Where APD C (a i |g i ) indicates that given input g i And the sampled action a obtained by the corresponding model C i The probability of C represents the probability of taking action sequence A calculated using the corresponding model C;
[0084] Update the Agent model based on the learned action probability sampling distribution and the old strategy of the compound molecular structure;
[0085] Constructing enhanced likelihood It is a combination of the prior likelihood and the reward obtained:
[0086]
[0087] In the formula, represents the prior likelihood; represents the scoring result given by the scoring function, i.e., the reward; σ is a definition parameter, the larger its value, the higher the acceptance of the Agent model to deviations from the prior model; the Agent's goal is to learn a strategy to maximize the expected return by minimizing the loss function J(Θ) = -G, where G is:
[0088]
[0089] In the formula, Represents the likelihood of the Agent model, which makes the Agent model likelihood as close as possible to the enhanced likelihood, that is, the target likelihood;
[0090] The parameters of the Agent model are gradient updated according to the loss function, so that the action sampling probability of the updated Agent model is close to the action sampling probability of the prior model until the training is completed.
[0091] It should be noted that, by using the enhanced likelihood, the action sampling probability of the final production model with the desired target properties can be made as close as possible to the action sampling probability of the prior model. During the training of deep reinforcement learning, the parameters of the Agent model are updated by gradient according to the loss value between the likelihood of the Agent model with updated parameters and the enhanced likelihood; after the parameters are updated, the action sampling probability of the Agent model will change accordingly, and the calculated likelihood will also change. The model parameters are repeatedly updated by gradient until the training is completed (such as the training converges or reaches the preset training time), and finally a production model with the desired target properties is obtained.
[0092] Example 2
[0093] This embodiment provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor, so that the processor executes part or all of the steps of the method provided in Example 1 of the present application.
[0094] It is understood that the storage medium may be transient or non-transient. Exemplarily, the storage medium includes, but is not limited to, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0095] Exemplarily, the processor may be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA).
[0096] Exemplarily, the read-only memory includes but is not limited to MASK ROM, PROM, EPROM, EEPROM, Flash, etc.
[0097] Exemplarily, the random access memory includes but is not limited to DRAM, SRAM, SDRAM, DDR SDRAM, etc.
[0098] In some examples, a computer program product is provided, which can be implemented in hardware, software or a combination thereof. As a non-limiting example, the computer program product can be embodied as the storage medium, or as a software product, such as an SDK (Software Development Kit).
[0099] As a non-limiting example, a computer program product is provided, the computer program product includes a computer program or a computer executable instruction, the computer program or the computer executable instruction is stored in a computer-readable storage medium. A processor of an electronic device reads the computer program or the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device performs some or all steps of the method described in the embodiment of the present application.
[0100] In some examples, a computer program is provided, comprising a computer-readable code, and when the computer-readable code is run in a computer device, a processor in the computer device executes a part or all of the steps in the method.
[0101] This embodiment also proposes an electronic device, including a memory and a processor, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and when the processor executes the at least one instruction, at least one program, a code set or an instruction set, it implements part or all of the steps of the method described in Example 1.
[0102] In some examples, a hardware entity of the electronic device is provided, including: a processor, a memory and a communication interface; wherein the processor generally controls the overall operation of the electronic device; the communication interface is used to enable the electronic device to communicate with other terminals or servers through a network; the memory is configured to store instructions and applications executable by the processor, and can also cache data to be processed or processed by the processor and various modules in the electronic device (including but not limited to image data, audio data, voice communication data and video communication data), which can be implemented by flash memory (FLASH), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or random access memory (RAM).
[0103] The processor may include one or more processing elements. Thus, the processor may include one or more integrated circuits (ICs) configured to perform the functions of the processor. In addition, each integrated circuit may include a circuit (e.g., a first circuit, a second circuit, and other circuits, etc.) configured to perform the functions of the processor.
[0104] Furthermore, data may be transmitted between the processor, the communication interface and the memory via a bus, which may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together.
[0105] It can be understood that the options in the above-mentioned embodiment 1 are also applicable to this embodiment, so they will not be described again here.
[0106] The same or similar reference numerals correspond to the same or similar components;
[0107] The terms used to describe the positional relationship in the drawings are only used for illustrative purposes and should not be construed as limiting the present application;
[0108] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.
[0109] In different specific implementations, the method or system described in the present application can be implemented in software, hardware or a combination thereof. In addition, the order of the steps of the method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.
[0110] Obviously, the above-mentioned embodiments of the present application are merely examples for clearly illustrating the present application, and are not intended to limit the implementation methods of the present application, and are not intended to limit the present application. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. Each discrete structure / function module or unit can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part, and the structure and function of the discrete components can be implemented as a combined structure or component. It is not necessary and impossible to enumerate all the implementation methods here. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the claims of the present application.
Claims
1. An optimization method for fragment-linked small molecule compounds based on deep reinforcement learning and curriculum learning, characterized in that: include: Establishing a supervised training set; wherein the supervised training set includes a plurality of fragment molecule quadruples and decoding routes in the form of molecular graph representation; Establishing an initial graph neural network-based prior model, and using the supervised training set to determine a number of prior models for different drug design tasks; Under the framework of curriculum learning, the prior model is used for reinforcement learning to obtain a production model with target attributes; The production model is used for large-scale generation of molecules.
2. The optimization method for fragment-linked small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 1, characterized in that: In the course learning framework, the prior model is used to perform reinforcement learning, including: The agent model is initialized using the prior model, and the preset drug design task C is decomposed into at least one simple learning stage goal O c and production stage target O p ; Wherein, if and only if the final learning stage goal is met, the course learning of the production stage goal is entered to generate molecules; For each learning stage goal or production stage goal, the agent model is trained using a reinforcement learning strategy, and a scoring function is introduced to score the generation result of the agent model until the scoring result is not less than a preset course stage threshold or reaches a preset training time, and the agent model enters the training of the next stage goal or ends the training; wherein the progress of the course learning is controlled by the course learning progress P, P = {P1,…, n-1 ,n), so that C={O,P}, 3. The optimization method of fragment connection small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 2, characterized in that: The reinforcement learning process includes: Using the fragment molecules that need to generate the connection substance as the input of the Agent model, and performing batch generation to obtain a plurality of the generation results; Introducing a preset scoring function to score the generation result of the Agent model; Introducing a policy gradient method to update the Agent model so that the scoring gradient can be conducted, with the goal of maximizing the scoring result for the generated result; The updating step is repeated until the scoring results corresponding to the generation results generated in batches are no longer improved or the updating is stopped after the preset training time is reached, and the updated Agent model is used as the production model.
4. The optimization method of fragment connection small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 3, characterized in that: The policy gradient method comprises: The action probability sampling distribution learned by the Agent model is used as a strategy. For the molecular graph generated by sampling, the product of the sampling probabilities represents the likelihood of the action probability sampling of the model, which is formally: Where APD C (a i |g i ) indicates that given input g i And the sampled action a obtained by the corresponding model C i The probability of c represents the probability of taking action sequence A calculated using the corresponding model C; Update the Agent model based on the learned action probability sampling distribution and the old strategy of the compound molecular structure; Constructing enhanced likelihood It is a combination of the prior likelihood and the reward obtained: In the formula, represents the prior likelihood; represents the scoring result given by the scoring function, i.e., the reward; σ is a definition parameter, the larger its value, the higher the acceptance of the Agent model to deviations from the prior model; the Agent's goal is to learn a strategy to maximize the expected return by minimizing the loss function J(Θ) = -G, where G is: In the formula, Represents the likelihood of the Agent model, which makes the Agent model likelihood as close as possible to the enhanced likelihood, that is, the target likelihood; The parameters of the Agent model are gradient updated according to the loss function, so that the action sampling probability of the updated Agent model is close to the action sampling probability of the prior model until the training is completed.
5. The optimization method of fragment connection small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 1, characterized in that: The process of establishing the supervised training set includes: Obtain data on several bioactive molecules with drug-like properties and filter them based on the Lipinski RO5 rule, the pan-assay interference compound substructure filter PAINS and / or the synthetic accessibility SA score to ensure that the generated data has high drug-like properties and can be synthesized; For the drug-like biologically active molecular data after screening, a matching molecule pair fragment algorithm is applied to construct the fragment molecule quadruple in the form of "first fragment, connector, second fragment, molecule"; The fragment molecular quadruples are filtered using the Lipinski RO3 rule constraint condition and uniformly converted into a molecular graph representation, which is recorded as the first molecular graph; Performing graph traversal on all the first molecular graphs, and gradually segmenting each of the first molecular graphs to obtain subgraphs and corresponding action probability distributions, until the decoded subgraph is an empty graph; Based on the subgraph and the action probability distribution, a corresponding decoding route of the first subgraph is constructed, thereby completing the establishment of the supervised training set.
6. The optimization method of fragment connection small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 1, characterized in that: The method of using the supervised training set to determine a plurality of prior models for different drug design tasks includes: The prior model is trained on the supervised training set using a back propagation algorithm until the value of the loss function converges or reaches a preset training time, so that the prior model learns the correct action information of the molecular combination; wherein the loss function is the KL divergence between the action probability of the real molecule and the action probability predicted by the prior model.
7. A method for optimizing fragment-linked small molecule compounds based on deep reinforcement learning and curriculum learning according to any one of claims 1 to 6, characterized in that: The prior model includes a GGNN block and a global readout block; wherein, The GGNN block consists of a GGNN model, which is used to receive adjacency tensors and node feature matrices about molecular graph representation data, and output corresponding node feature vectors and graph embedding information; The global readout block is used to calculate the corresponding action probability according to the node feature vector and the graph embedding information.
8. The optimization method of fragment connection small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 7, characterized in that: The global readout block is composed of multiple multi-layer perceptrons MLP; the action probability APD is calculated by the following formula: f′ add =MLP add,1 (H L ) f′ conn =MLP conn,1 (H L ) f add =MLP add,2 (f′ add ,g]) f conn =MLP conn,2 ([f′ conn ,g]) f term =MLP term,2 (g) APD=softmax([f add ,f conn ,f term ]) Wherein, H represents the node feature vector output by the GGNN block; g represents the graph embedding information output by the GGNN block; f add represents the probability of adding a new node, i.e., adding an atom, to the molecular graph output by MLP; f conn represents the probability of connecting the last attached node in the molecular graph to another existing node in the graph, i.e., adding a bond, output by the MLP; f term represents the probability of terminating the molecular graph; f′ add and f′ conn Indicates intermediate output.
9. An electronic device, characterized in that: include: A memory for storing computer executable instructions or computer programs; A processor, used to implement the method according to any one of claims 1 to 8 when executing computer executable instructions or computer programs stored in the memory.
10. A computer program product comprising a computer program or computer executable instructions, characterized in that: When the computer program or computer executable instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Drug molecule generator training method based on domain knowledge and deep reinforcement learning
CN113223637A
Compound optimization method based on deep learning connection fragment
CN114187978A
Reagent molecule prediction method and device, reactant molecule model training method and device, equipment and medium
CN115206451A
Cited By
Course learning and reinforcement learning driven dynamic optimal self-consistent clustering method
CN120372315A
