An optimization method for fragment ligation small molecule compounds based on deep reinforcement learning and curriculum learning

Abstract

The application discloses a kind of based on depth reinforcement learning and course learning's optimization method of fragment connection small molecule compound, it is related to artificial intelligence and drug molecule optimization field.The method comprises: establishing supervised training set;Establish initial prior model based on graph neural network, determine several prior models for different drug design tasks using supervised training set;In the course learning framework, reinforcement learning is carried out using prior model, and production model with target attribute is obtained;Large-scale molecular generation is carried out using production model.Compared with prior art, the method disclosed in the application solves the problem of generating more invalid molecules and non-real structured compounds by previous models, and can generate drug molecules with complex desired attributes at a relatively small computational cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and drug molecule optimization, and more particularly to a method for optimizing fragment-ligating small molecule compounds based on deep reinforcement learning and curriculum learning. BACKGROUND

[0002] Fragment-based drug discovery (FBDD) starts from the screening of low molecular weight fragments, and then gradually optimizes molecules, which has attracted widespread attention in the pharmaceutical industry due to its low screening cost and the ability to efficiently identify lead compounds. In FBDD, a useful molecular optimization strategy is fragment ligation, which connects two molecular fragments bound in different but adjacent subpockets together through an appropriate linker. Traditional computational linker design methods are based on fragment database search, combined with physical simulation to determine the priority of a subset of linking compounds. These methods are essentially limited by the small chemical space of the fragment database used and the computational complexity of physical simulation. Recently, the development of deep learning-based generative methods has enabled the exploration of a larger chemical space in a data-driven manner for linker design.

[0003] In the model method based on deep learning, various generative architectures such as recurrent neural networks (RNN), transformer neural networks, and autoencoders (VAE) have been proposed to generate the desired molecules for de novo drug design and lead compound optimization, and these generative strategies have also been effectively used for linker design in FBDD. Yang et al. proposed a SMILES-based model SyntaLinker that treats fragment ligation as a sentence completion task through a conditional transformer neural network and generates complete molecules with linker length constraints; Imrie et al. developed a graph-based model DeLinker that explicitly incorporates 3D structural information through the relative orientation and distance between two fragments during linker design. E(3)-invariant linker design models, including 3DLinker developed by Huang et al. and DiffLinker developed by Igashov et al., whose performance is independent of the initial coordinates, have been developed to generate molecular linkers that preserve the spatial configuration of two fragments. Compared with traditional computational methods, these deep learning models can search a larger chemical space and generate more diverse linkers.

[0004] Although these deep generative models generate linkers with good results on ChEMBL, ZINC and other datasets, many methods cannot design drug molecules for specific target sites or specific target properties. For example, the SMILES generated by SyntaLinker and other models on the ChEMBL dataset has only a very low validity, and cannot obtain correct linkers at all; and the DeLinker and other models are trained based on a graph, and their validity is guaranteed, but the generated linkers are quite different from the real linker structure. At the same time, such pre-training models learn the distribution of the training set molecules, so the generated linkers can only approach the training set, and cannot generate molecules with any desired properties, so as to better solve some drug molecule drugability, synthesis feasibility and other problems. For some models that introduce reinforcement learning technology, although the problem of generating target attribute molecules can be alleviated, when the target attribute is too complex, the computational cost of reinforcement learning is high, and even cannot reach the convergence state. SUMMARY

[0005] The present application is to overcome the defects of the prior art that cannot balance the validity, authenticity and desired attributes, and provides an optimization method for fragment connection of small molecule compounds based on deep reinforcement learning and curriculum learning.

[0006] To solve the above technical problems, the technical solutions of the present application are as follows:

[0007] In a first aspect, an optimization method for fragment connection of small molecule compounds based on deep reinforcement learning and curriculum learning, comprising:

[0008] Establish a supervised training set; wherein the supervised training set comprises a plurality of fragment molecule quadruples in the form of molecular graph representation and decoding routes;

[0009] Establish an initial prior model based on graph neural network, and determine a plurality of prior models for different drug design tasks by using the supervised training set;

[0010] Under the curriculum learning framework, reinforcement learning is performed by using the prior model to obtain a production model with target attributes;

[0011] Large-scale molecule generation is performed by using the production model.

[0012] In a second aspect, an electronic device, comprising:

[0013] A memory for storing computer executable instructions or computer programs;

[0014] A processor for executing the computer executable instructions or computer programs stored in the memory to implement the method of the first aspect.

[0015] In a third aspect, a computer program product includes a computer program or computer executable instructions, which, when executed by a processor, implement the method of the first aspect.

[0016] Compared with the prior art, the beneficial effects of the technical scheme of the present application are as follows:

[0017] The present application discloses an optimization method for fragment connection small molecule compounds based on deep reinforcement learning and curriculum learning. By training a prior model based on a graph neural network on a supervised training set, the problem of generating too many invalid molecules and non-real structured compounds by the previous model is solved, and the basic generation ability of the model is ensured. On this basis, reinforcement learning is introduced to flexibly change the model generation distribution, so that the model generates drug molecules with expected properties. By introducing a curriculum learning framework, the short board of reinforcement learning in generating complex target attribute tasks is made up, so that the model generates drug molecules with complex attributes at a small calculation cost. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A flowchart of the method described in Embodiment 1 of the present application is shown;

[0019] Figure 2 Another flowchart of the method described in Embodiment 1 of the present application is shown. DETAILED DESCRIPTION

[0020] The terms "first", "second", and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the terms thus used can be interchanged, as appropriate, and that the use of such terms in describing the embodiments of the present application is merely intended to distinguish between objects of the same nature that are described in the description. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device that includes a list of elements does not necessarily limit to those elements, but can include other elements not clearly listed or inherent to the process, method, product or device. The term "determine" broadly covers various actions, which can include obtaining, calculating, computing, processing, deriving, investigating, looking up (for example, looking up in a table, a database or another data structure), ascertaining and the like, and can also include receiving (for example, receiving information), accessing (for example, accessing data in a memory) and the like. It can also include generating, creating, establishing and the like, as well as resolving, selecting, choosing and the like. Definitions of other terms will be given in the following description.

[0021] The accompanying drawings are only used for illustrative purposes, and cannot be understood as limiting the patent;

[0022] In order to better illustrate the present embodiment, some components in the drawings may be omitted, enlarged or reduced, and do not represent the actual size of the product;

[0023] It is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0024] The technical solutions of the present application will be further described below in combination with the drawings and embodiments.

[0025] Embodiment 1

[0026] The present embodiment provides an optimization method for fragment connection of small molecule compounds based on deep reinforcement learning and curriculum learning, referring to Figure 1 , comprising:

[0027] Establish a supervised training set; wherein the supervised training set comprises a plurality of fragment molecule quadruples in the form of molecular graph representation and decoding routes;

[0028] Establish an initial prior model based on graph neural network, and determine a plurality of prior models for different drug design tasks using the supervised training set;

[0029] Under the curriculum learning (Curriculum Learning) framework, reinforcement learning is performed using the prior model to obtain a production model with target attributes;

[0030] Large-scale molecule generation is performed using the production model.

[0031] The present embodiment trains a prior model based on graph neural network on the supervised training set, so that the model can learn the action information of molecular combination based on molecular graph and decoding route, solving the problem of generating more invalid molecules and non-real structured compounds by the past model, and ensuring the basic generation ability of the model. On this basis, reinforcement learning is introduced to flexibly change the model generation distribution, so that the model generates drug molecules with expected attributes. By introducing the curriculum learning framework, the short board of reinforcement learning in generating complex target attribute tasks is made up, so that the model generates drug molecules with complex attributes through smaller calculation cost. Those skilled in the art can generate connection substances similar to real structures using the production model, and then generate drug molecules with expected properties according to predefined target attributes.

[0032] In some preferred embodiments, the establishment process of the supervised training set comprises:

[0033] A number of drug-like bioactive molecule data are obtained and filtered based on Lipinski RO5 ("Rule of Five") rule, Pan-Assay Interference Compound (PAINS) filter and / or Synthetic Accessibility (SA) score to ensure that the generated data are highly drug-like and synthetically accessible;

[0034] The filtered drug-like bioactive molecule data are subjected to a Matched Molecular Pair (MMP) algorithm to construct Fragment-Molecule Quadruples (FMQ) in the form of "Fragment 1, Linker, Fragment 2, Molecule".

[0035] The FMQs are filtered using Lipinski RO3 ("Rule of Three") rule constraints and converted into molecular graph representations, denoted as First Molecular Graphs (FMG).

[0036] All the FMGs are subjected to a graph traversal and segmented step by step to obtain subgraphs and corresponding action probability distributions until the decoded subgraph is an empty graph.

[0037] Based on the subgraphs and the action probability distributions, a corresponding decoding path of the FMG is constructed, and the supervised training set is completed.

[0038] Those skilled in the art will understand that the action probability distribution is composed of a series of action probabilities.

[0039] In some implementations, a supervised training set is established using the ChEMBL database, as described in Figure 2 :

[0040] A dataset is extracted from the ChEMBL database and filtered using Lipinski's RO5, Pan-Assay Interference Compound (PAINS) filter and Synthetic Accessibility (SA) score. After filtering, a Matched Molecular Pair (MMP) algorithm is applied to construct Fragment-Molecule Quadruples (FMQ) in the form of "Fragment 1, Linker, Fragment 2, Molecule", and a total of 5,873,503 FMQs are obtained. Then, the FMQs are filtered using RO3 constraints, and about 718,652 pairs of end fragments and original molecules are retained as Fragment-Molecule Triples (FMT). All these compounds are uniformly converted into molecular graph representations, such as adjacency tensors and node feature matrices, etc. The filtered dataset is further divided into three groups in the ratio of 8:1:1 for training, validation and testing of the Prior model.

[0041] The preprocessing of the training data is regarded as a graph traversal problem, and a molecular graph G n with n nodes will be segmented step by step to obtain a decoding path r of the molecular graph. Whenever G n is separated into Gn-1 At each time, a corresponding action probability APD n-1 , until an empty graph G0is decoded and a corresponding APD0is obtained. Thus, the decoding route r of the molecular graph can be expressed as:

[0042] r = {(G0, APD0), …, (G n-1 , APD n-1 ), (G n , APD n )}

[0043] All molecules in the same batch will repeat this operation and finally be stored in an hdf file for subsequent data reading, completing the establishment of the supervision data set.

[0044] In some embodiments, the supervision training set is established using the ZINC database.

[0045] In some preferred embodiments, the supervision training set is used to determine a plurality of prior models for different drug design tasks, including:

[0046] The prior models are trained on the supervision training set using a backpropagation algorithm until the value of a loss function converges or a preset training time is reached, so that the prior models learn the correct action information of the molecular combination; wherein the loss function is the KL (Kullback-Leibler) divergence between the action probability of the real molecule and the action probability predicted by the prior model.

[0047] As a non-limiting example, the different drug design tasks can be high synthetic feasibility, high drug-likeness, high 2D similarity molecules, or high biological activity. Those skilled in the art will understand that different drug design tasks can be distinguished according to different training sets, for example, the data set is first processed to have high synthetic feasibility for model training.

[0048] In some embodiments, the prior models for different drug design tasks will be trained under the Pytorch framework, and the loss function is set as:

[0049]

[0050] wherein APD init represents the initial value of the action probability; and APD pred represents the predicted value of the action probability.

[0051] In the training, the parameter values of the model are constantly updated using the back propagation algorithm, and the validation loss value of the prior model will be output after iterating a certain Epoch. When the loss value decreases and tends to converge, it is considered that the model is stable.

[0052] In some preferred embodiments, the prior model comprises a GGNN block and a global readout block; wherein,

[0053] The GGNN block is composed of a GGNN (Gated Graph Sequence Neural Networks) model, which is used to receive the adjacency tensor and node feature matrix of the molecular graph representation data, and output the corresponding node feature vector and graph embedding information;

[0054] The global readout block is used to calculate the corresponding action probability (APD) according to the node feature vector and the graph embedding information, to guide the model how to form the graph.

[0055] In some optional embodiments, the global readout block is composed of a plurality of multi-layer perceptron MLPs; the action probability APD is calculated by the following formula:

[0056] f′ add =MLP add,1 (H L )

[0057] f′ conn =MLP conn,1 (H L )

[0058] f add =MLP add,2 ([f′ add ,g])

[0059] f conn =MLP conn,2 ([f′ conn ,g])

[0060] f term =MLP term,2 (g)

[0061] APD=softmax([f add ,f conn ,f term ])

[0062] Wherein, H represents the node feature vector output by the GGNN block; g represents the graph embedding information output by the GGNN block; f add represents the probability of adding a new node, i.e. an atom, in the molecular graph output by the MLP; f connf represents the probability of connecting the last added node in the molecular graph to another existing node in the graph, i.e., adding a bond; f term f' represents the probability of terminating the molecular graph; add and f' conn represents the intermediate output.

[0063] It should be noted that the process of generating a graph by the model (i.e., the actions that can be taken) can be basically divided into three steps: adding an atom (add), adding a bond (conn, i.e., bond connection), and termination judgment (term, i.e., sampling termination), all of which are based on the global readout module to calculate the action probability APD in the previous step to determine whether to continue.

[0064] More specifically, f add and f conn is generated by using a two-layer MLP network.

[0065] In some preferred embodiments, under the curriculum learning framework, reinforcement learning is performed using the prior model, which includes:

[0066] The prior model Prior is used to initialize the Agent model (intelligent agent), and a preset drug design task C is decomposed into at least one simple learning stage goal O c and production stage goal O p ; wherein, only when the final learning stage goal is met, the curriculum learning enters the production stage goal to generate a molecule;

[0067] For each learning stage goal or production stage goal, a reinforcement learning strategy is used to train the Agent model, and a scoring function is introduced to score the generated results of the Agent model, until the scoring result is not less than a preset curriculum stage threshold or a preset training time is reached, the Agent model enters the training of the next stage goal or ends the training; wherein, the progress of curriculum learning is controlled by curriculum learning progress P, P = {P1, …, P n-1 , P n}, so that C = {O, P},

[0068] It should be emphasized that, only when the final curriculum progress standard P n is met, the production stage goal O p of curriculum learning is entered to generate a molecule, and the Agent will sample compounds in the favorable area of the chemical space that meets the production goal.

[0069] In some optional embodiments, the process of reinforcement learning includes:

[0070] The fragment molecules that need to generate the linker are taken as the input of the Agent model, and a plurality of generation results are generated in batches;

[0071] A preset scoring function is introduced to score the generation results of the Agent model;

[0072] The policy gradient method of reinforcement learning is introduced to update the Agent model, so that the scoring gradient can be conducted, and the goal is to maximize the scoring result of the generation result;

[0073] The updating step is repeated until the scoring result corresponding to the generation result generated in batches no longer improves or reaches the preset training time, and then the updated Agent model is taken as the production model.

[0074] It should be emphasized that by introducing the scoring function and the policy gradient method, the model generation distribution can be flexibly changed, so that the model generates drug molecules with expected properties.

[0075] It should be understood that the trained Prior model learns the action probability sampling distribution (grammar) and the strategy of compound molecular structure (referred to as "old strategy"). By initializing the Agent model to Prior, the Agent model learns the old strategy, that is, the reinforcement learning model (Agent model) masters the prior knowledge and updates the strategy on this basis.

[0076] It should also be understood that in the model training, the preset training time can be represented by the preset training step.

[0077] It should be noted that the scoring function is formulated by those skilled in the art according to the needs. In some specific implementation processes, if it is necessary to improve the drugability of the generated drug molecules, the scoring function is formulated as follows:

[0078]

[0079] In the formula, is the sequence generated by the Prior model, represents the attribute value of the drug molecule; Target represents the target value; and a is a self-defined parameter.

[0080] Further, the policy gradient method includes:

[0081] The action probability sampling distribution learned by the Agent model is taken as the strategy. For the molecular graph generated by sampling, the product of the sampling probability represents the likelihood of the model action probability sampling, which is formally:

[0082]

[0083] wherein APD C (a i |g i ) represents the probability of the sampling action a i obtained by the given input g i and the corresponding model C, P(A) C represents the probability of taking the action sequence A calculated by the corresponding model C.

[0084] Based on the learned action probability sampling distribution and the old strategy of the molecular structure of the compound, the Agent model is updated.

[0085] The enhanced likelihood is constructed which is a combination of the prior likelihood and the obtained reward:

[0086]

[0087] wherein represents the prior likelihood; represents the scoring result given by the scoring function, i.e., the reward; σ is a definition parameter, and the greater the value, the higher the acceptance of the Agent model to deviate from the prior model; the goal of the Agent is to learn a strategy to maximize the expected return by minimizing the loss function J(Θ) = -G, wherein G is:

[0088]

[0089] wherein represents the likelihood of the Agent model, and the formula makes the Agent model likelihood as close as possible to the enhanced likelihood, i.e., the target likelihood;

[0090] According to the loss function, the parameters of the Agent model are updated by gradient, so that the action sampling probability of the updated Agent model is close to the action sampling probability of the prior model, until the training is completed.

[0091] It should be noted that by using the enhanced likelihood, the action sampling probability of the production model with the required target property obtained finally can be as close as possible to the action sampling probability of the prior model. During the training of deep reinforcement learning, the parameters of the Agent model are updated by gradient according to the loss value between the likelihood of the Agent model with updated parameters and the enhanced likelihood; after the parameters are updated, the action sampling probability of the Agent model changes accordingly, and the calculated likelihood also changes; the model parameters are repeatedly updated by gradient until the training is completed (such as training convergence or reaching a preset training time), and finally the production model with the required target property is obtained.

[0092] Embodiment 2

[0093] The embodiment provides a computer readable storage medium, and at least one instruction, at least one program, a code set or an instruction set are stored on the storage medium. The at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor, so that the processor executes part or all steps of the method provided in the embodiment 1 of the application.

[0094] It can be understood that the storage medium can be transitory or non-transitory. Exemplarily, the storage medium includes but is not limited to a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage program codes.

[0095] Exemplarily, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA).

[0096] Exemplarily, the read-only memory includes but is not limited to a mask ROM, a PROM, an EPROM, an EEPROM, a flash and the like.

[0097] Exemplarily, the random access memory includes but is not limited to a DRAM, an SRAM, an SDRAM, a DDR SDRAM and the like.

[0098] In some examples, a computer program product is provided, which can be implemented by hardware, software or a combination thereof. As a non-limiting example, the computer program product can be embodied as the storage medium, and can also be embodied as a software product, such as an SDK (Software Development Kit) and the like.

[0099] As a non-limiting example, a computer program product is provided, which includes a computer program or computer executable instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer program or computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes part or all steps of the method provided in the embodiment of the application.

[0100] In some examples, a computer program is provided, including computer readable code, which, when run in a computer device, causes a processor in the computer device to perform some or all of the steps in the method.

[0101] The embodiment also provides an electronic device, including a memory and a processor, the memory stores at least one instruction, at least one program, a code set or an instruction set, and the processor implements some or all of the steps of the method as described in embodiment 1 when executing the at least one instruction, at least one program, code set or instruction set.

[0102] In some examples, a hardware entity of the electronic device is provided, including a processor, a memory and a communication interface; wherein the processor generally controls the overall operation of the electronic device; the communication interface is used for the electronic device to communicate with other terminals or servers through a network; the memory is configured to store instructions and applications executable by the processor, and can also cache data to be processed by the processor and data to be processed or having been processed by each module in the electronic device (including but not limited to image data, audio data, voice communication data and video communication data), which can be realized by FLASH, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory) or RAM (Random Access Memory).

[0103] The processor can include one or more processing elements. Thus, the processor can include one or more integrated circuits (ICs) that are configured to perform the functions of the processor. In addition, each integrated circuit can include circuitry (e.g., first circuitry, second circuitry, and other circuitry) that is configured to perform the functions of the processor.

[0104] Further, the processor, the communication interface and the memory can transmit data through a bus, which can include any number of interconnected buses and bridges, connecting the various circuits of the one or more processors and the memory together.

[0105] It can be understood that the optional items in the above embodiment 1 are also applicable to the present embodiment, and thus will not be repeatedly described here.

[0106] The same or similar reference numerals correspond to the same or similar components;

[0107] The terms describing the positional relationship in the drawings are only used for exemplary illustration, and cannot be understood as a limitation on the present application;

[0108] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict.

[0109] In different specific implementations, the methods or systems described in the present application can be implemented in software, hardware or a combination thereof. In addition, the order of the steps of the method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0110] Obviously, the above-described embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the implementation of the present application, and are not used to limit the present application. Based on the above description, other different forms of changes or variations can be made by those of ordinary skill in the art, and each separate structure / function module or unit can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part, and the structure and function of the separate components can be implemented as a combined structure or component. Here, it is not necessary and impossible to enumerate all the embodiments. Any modification, equivalent replacement and improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. An optimization method for fragment connection of small molecule compounds based on deep reinforcement learning and curriculum learning, characterized in that, include: Establish a supervised training set; wherein, the supervised training set includes several fragment molecular quadruples in molecular graph representation and decoding routes; An initial prior model based on a graph neural network is established, and several prior models for different drug design tasks are determined using the supervised training set. Within the course learning framework, reinforcement learning is performed using the prior model to obtain a production model with target attributes; Large-scale molecular generation is carried out using the aforementioned production model; The reinforcement learning using the prior model within the course learning framework includes: The agent model is initialized using the prior model, and the preset drug design task is then implemented. Break it down into at least one simple learning stage goal and production phase goals ; wherein, the course learning proceeds to the production stage objective only after the final learning stage objective is met, in order to generate molecules; For each learning stage objective or production stage objective, a reinforcement learning strategy is used to train the Agent model, and a scoring function is introduced to score the generated results of the Agent model until the score is not less than a preset course stage threshold or a preset training duration is reached. At this point, the Agent model either proceeds to the next stage objective training or terminates training. The progress of the course learning is monitored by the course learning progress... control, , making , ; The process of establishing the supervised training set includes: Data on several drug-like bioactive molecules were acquired and filtered based on the Lipinski RO5 rule, the PAINS general assay interfering compound substructure filter, and / or the SA score for synthetic accessibility to ensure that the generated data are highly drug-like and synthetic. For the screened drug-like bioactive molecule data, the matching molecule pair fragmentation algorithm is applied to construct the fragment molecule quadruple in the form of "first fragment, linker, second fragment, molecule"; The fragment molecular quadruplets were filtered using the Lipinski RO3 rule constraint and uniformly converted into a molecular graph representation, denoted as the first molecular graph. Perform graph traversal on all the first molecular graphs and progressively segment each first molecular graph to obtain subgraphs and corresponding action probability distributions, until the decoded subgraph is an empty graph; Based on the subgraph and the action probability distribution, the corresponding decoding route of the first subgraph is constructed, thereby completing the establishment of the supervised training set.

2. The optimization method for fragment connection of small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 1, characterized in that, The reinforcement learning process includes: The fragment molecules that need to be generated as connectors are used as input to the Agent model to generate several generation results in batches; A preset scoring function is introduced to score the generated results of the Agent model; The Agent model is updated by introducing a policy gradient method, which allows the scoring gradient to be propagated, with the goal of maximizing the scoring result for the generated result; Repeat the update steps until the scores corresponding to the batch-generated results no longer improve or the preset training time is reached, then stop updating and use the updated Agent model as the production model.

3. The optimization method for fragment connection of small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 2, characterized in that, The policy gradient method includes: Using the action probability sampling distribution learned by the Agent model as the policy, for the molecular graph generated by sampling, the product of the sampling probabilities represents the likelihood of the model's action probability sampling, in the form of: In the formula, Indicates the given input and corresponding model The obtained sampling action The probability, Indicates the use of the corresponding model Calculated action sequence The probability of; The Agent model is updated based on the learned action probability sampling distribution and the old strategy of compound molecular structure. Constructing Enhanced Likelihood It is a combination of the prior likelihood and the reward obtained: In the formula, Indicates the prior likelihood; This represents the scoring result given by the scoring function, i.e., the reward; This defines a parameter; the larger its value, the more accepting the agent model is of deviating from the prior model. The agent's goal is to learn a method that minimizes the loss function. The strategy to maximize expected return, among which for: In the formula, Let f(x) represent the likelihood of the Agent model. This formula makes the Agent model likelihood as close as possible to the enhancement likelihood, which is the target likelihood. The parameters of the Agent model are updated using gradients based on the loss function, so that the action sampling probability of the updated Agent model is close to the action sampling probability of the prior model, until training ends.

4. The optimization method for fragment connection of small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 1, characterized in that, The process of determining several prior models for different drug design tasks using the supervised training set includes: The prior model is trained on the supervised training set using the backpropagation algorithm until the value of the loss function tends to converge or reaches the preset training time, so that the prior model learns the action information of the correct molecular combination; wherein, the loss function is the KL divergence between the action probability of the real molecule and the action probability predicted by the prior model.

5. The optimization method for fragment connection of small molecule compounds based on deep reinforcement learning and curriculum learning according to any one of claims 1-4, characterized in that, The prior model includes GGNN blocks and global readout blocks; wherein... The GGNN block is composed of a GGNN model, which is used to receive the adjacency tensor and node feature matrix of molecular graph representation data, and output the corresponding node feature vector and graph embedding information. The global readout block is used to calculate the corresponding action probability based on the node feature vector and the graph embedding information.

6. The optimization method for fragment connection of small molecule compounds based on deep reinforcement learning and curriculum learning according to claim 5, characterized in that, The global readout block is composed of multiple multilayer perceptrons (MLPs); the action probability (APD) is calculated using the following formula: Wherein, H represents the node feature vector output by the GGNN block; g represents the graph embedding information output by the GGNN block; This represents the probability of adding a new node, i.e., adding an atom, to the molecular graph as output by the MLP. This represents the probability output by the MLP used to connect the last added node in the molecular graph to another existing node in the graph, i.e., to add a bond. This represents the probability of terminating the molecular graph; and This indicates intermediate output.

7. An electronic device, characterized in that, include: Memory is used to store executable instructions or computer programs. A processor, configured to execute computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1-6.

8. A computer program product, comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by a processor, the method described in any one of claims 1-6 is implemented.

Images