Drug molecular structure-assisted design method based on neural network

Abstract

A neural network-based drug molecular structure-assisted design method uses a multi-objective reinforcement learning model to batch generate small molecules; drug molecules included in a database are used as positive samples, and molecules with lower quality generated by the reinforcement learning model are used as negative samples to construct a benchmark dataset; the dataset is input into a graph neural network model (GIN) for training to obtain a trained molecular quality assessment (QA) model; a score library is constructed, and the quality assessment score (QAscore) is redefined based on the rank of the QA model's predicted scores for new molecules in the score library; the QAscore is added as one of the optimization targets of the multi-objective reinforcement learning model to regenerate molecules; the molecules generated by the multi-objective reinforcement learning model are fed back to the QA model for retraining; and the method is repeated until indicators such as the drug-like properties of the molecules generated by the multi-objective reinforcement learning model no longer show significant improvement. The present invention achieves multi-objective parallel optimization while organically combining reinforcement learning and quality assessment graph neural networks.

Description

Technical Field

[0001] The present invention relates to a technology in the field of neural network applications, specifically a method for auxiliary design of drug molecular structures based on neural networks. Background Art

[0002] De novo drug design is a key area of ​​research in bioinformatics, encompassing subtasks such as molecule generation and molecular property optimization. Existing molecule generation algorithms focus solely on the chemical validity and novelty of generated molecules, neglecting their physicochemical properties. Existing molecular property optimization algorithms typically optimize only a single objective function and rely heavily on traditional, empirically based optimization metrics. Summary of the Invention

[0003] In response to the shortcomings of existing technologies that are unable to generate and optimize molecular structures from scratch based on the specific physicochemical properties of molecules and the results obtained are overly dependent on the distribution of training samples, the present invention proposes a neural network-based drug molecular structure auxiliary design method, which uses a graph neural network to learn new quality assessment indicators from the drug database as one of the reward functions, and uses a separate neural network for each reward function to achieve multi-objective parallel optimization while organically combining reinforcement learning and quality assessment graph neural networks.

[0004] The present invention is achieved through the following technical solutions:

[0005] The present invention relates to a neural network-based drug molecular structure-assisted design method. The method comprises the following steps: using a multi-objective reinforcement learning model to batch generate small molecules; constructing a benchmark data set using drug molecules included in a database as positive samples and molecules with lower quality generated by the reinforcement learning model as negative samples; inputting the data into a graph neural network model (GIN) for training to obtain a trained molecular quality assessment (QA) model; constructing a score library, and redefining a quality assessment score (QAscore) based on the rank of the QA model's predicted scores for new molecules in the score library; adding QAscore as one of the optimization targets of the multi-objective reinforcement learning model to regenerate molecules; feeding the molecules generated by the multi-objective reinforcement learning model back to the QA model for retraining; and repeatedly iterating until indicators such as the drug-like properties of the molecules generated by the multi-objective reinforcement learning model no longer show significant improvement.

[0006] The present invention relates to a system for implementing the above-mentioned method, comprising: a reinforcement learning model unit, a graph neural network unit and an iterative feedback unit, wherein: the graph neural network unit is trained based on the drug small molecule information included in the drug database to obtain a quality assessment model; the reinforcement learning model unit uses the predicted score of the quality assessment model and the molecular physicochemical property indicators as a reward function to batch generate small molecules; the iterative feedback unit uses the low-quality small molecules generated as negative samples, feeds them back to the graph neural network unit for retraining, obtains a new quality assessment model and updates the reward function in the reinforcement learning model unit, and iterates repeatedly.

[0007] Technical Effects

[0008] By using new molecular quality assessment indicators and introducing a feedback iteration mechanism in molecule generation and optimization, the chemical effectiveness of the designed molecules reaches 100%, and the success rate, novelty and average QED indicators of the optimized molecules are significantly higher than those of the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 Flowchart of the present invention;

[0010] Figure 2 Schematic diagram of the embodiment flow chart;

[0011] Figure 3 A bar chart showing the effects of the embodiments on the evaluation indicators. DETAILED DESCRIPTION

[0012] like Figure 1 As shown, this embodiment relates to a method for de novo drug design and drug quality assessment, comprising the following steps:

[0013] Step 1) Training a multi-objective reinforcement learning method to generate molecules in batches, specifically including:

[0014] Step 1.1) Initialize the memory pool M with size N and set the hyperparameter frequency to FREQUENCY = 100;

[0015] Step 1.2) Initialize the evaluation Q network;

[0016] Step 1.3) Initialize the target Q network, fix the network parameters, and set the initial values ​​to be the same as those of the evaluation Q network;

[0017] Step 1.4) Train the evaluation Q network and the target Q network, specifically including:

[0018] ① Select and execute the action a of the current time step according to the action selection strategy t , receive reward r t and the next state S t+1 Then store it in the memory pool M;

[0019] The action of the current time step Where: A is the action set of the current time step, s t is the molecular state at the current time step, Q eval is the evaluation Q network, ω is the evaluation Q network parameter, and ε is the hyperparameter.

[0020] The storage is in the memory pool, and when the size N is exceeded, it is stored in an overwritten manner.

[0021] ② Randomly sample batches (S i , a i , r i , S i+1 ) and the evaluation Q network calculates q _eval =Q eval (s i , a i ;ω), calculated by the target Q network Where: γ is the discount factor, Q target is the target Q network; then calculate the MSE loss function loss = MSEloss (q_target, q_eval) and perform loss back propagation;

[0022] ③ Execute the above steps ① and ② in a manner of traversing 1 to 60 time steps per round, and loop through 1 to 5000 rounds to achieve training.

[0023] Step 1.5) Set the update frequency to 100. Whenever the update frequency is divisible by the time step, update the target Q network parameters to make them the same as the current evaluation Q network.

[0024] Step 1.6) Each round selects the molecular state S of the last time step T As the generated molecules, the generated molecules output from the last 1000 rounds were selected.

[0025] Step 2) pre-processing the generated molecules, specifically: adding a functional group containing {F, S, Cl, Br, I} elements to each molecule, and correspondingly generating 10 extended molecules containing one random functional group and 2 extended molecules containing two random functional groups for each molecule.

[0026] When the number of functional groups generated for each molecule is less than 12, the maximum number generated shall be used.

[0027] Step 3) Build a benchmark dataset and extract atomic features to create a molecular graph, specifically including:

[0028] Step 3.1) Download bioactive molecules from the CHEMBL database, randomly select 154,000 non-duplicate molecules as positive samples, and use the 108,859 expanded molecules generated in step 2) as negative samples to construct a benchmark dataset for the molecular quality assessment model;

[0029] Step 3.2) Generate a label matrix L from the benchmark dataset; construct a graph with atoms in the molecule as nodes and bonds as edges to generate an adjacency matrix A; extract the atomic species (19 dimensions), whether the atom is on a ring (2 dimensions), atomic hybridization mode (6 dimensions), formal charge (1 dimension), and atomic degree features (1 dimension) to construct a 29-dimensional node feature matrix X; establish a molecular graph G = (A, X), where the nodes of the molecular graph are all atoms except H atoms and the edges of the molecular graph are the chemical bonds connecting atoms.

[0030] Step 4) Build and train a molecular mass assessment model, specifically including:

[0031] Step 4.1) Construct a molecular mass assessment model consisting of 5 GIN layers, 1 pooling layer, and 1 readout layer, where the GIN layer consists of a graph convolution GCN layer, a batch normalization BN layer, and an activation function ReLu. The input of the molecular mass assessment model is the molecular graph G = (A, X), and the output is the embedded features of the entire molecular graph. The classification probability is obtained after passing the Softmax function.

[0032] Step 4.2) Training the molecular mass assessment model obtained in step 4.1, specifically:

[0033] a) The graph convolution layer aggregates the node information of the molecular graph. The feature of each node is obtained by the weighted sum of the feature of the node in the previous layer and the feature of the neighboring nodes through the MLP nonlinear transformation. Specifically, the embedded feature of node i in the new layer l1 is in: is the embedded feature of node i in layer l, N(i) is the set of neighbor nodes of node i, and ε is Weight-related hyperparameters, MLP is a nonlinear transformation. The readout layer will output the embedding features of the entire graph, which is calculated by aggregating the information of all nodes on the graph: Where: G is the molecular graph and n is the number of layers of the neural network.

[0034] b) The loss function was set to Cross Entropy loss, using the Adam optimizer. The learning rate lr was set to 0.01 and scheduled decay was performed, with the decay being 0.5 times every 20 rounds. All molecular graph data were divided into training set, validation set, and test set in a ratio of 7:2:1. 10-fold cross-validation was used, and the batch size was set to 1024. After 200 epochs, the model training was completed.

[0035] Step 5) Construct a score library and define the molecular quality assessment score QAscore; establish a score library based on the predicted scores of the molecules in the benchmark dataset, and define the quality assessment score according to the ranking of the predicted scores of the molecules to be predicted in the score library Where x is the rank of the molecule to be predicted, n and m are the number of positive and negative samples in the score library, respectively. The QAscore value range is [0, 1], and the closer to 1, the higher the quality of the molecule.

[0036] Step 6) Add QAscore as the reward of the multi-objective reinforcement learning model. Do not retain the relevant parameters of the reinforcement learning model in the previous round, and regenerate 5000 EPISODEs.

[0037] Step 7) The generated molecules are fed back to the QA model for further training, and the training is repeated until the QED and SAscore indicators of the generated molecules no longer show significant improvement or the model no longer converges.

[0038] This embodiment tests various current methods on different evaluation indicators and objective functions, and the results are shown in Tables 1 and 2.

[0039] Table 1: Performance of different drug design methods on evaluation indicators

[0040] method Effectiveness Success rate Novelty Diversity molDQN 100％ 0.0170 0.3609 0.5310 MARS 99.7％ 0.7980 0.3332 0.6413 BIMODAL 99.7％ 0.2858 0.3144 0.7201 REINVENT 94.0％ 0.3750 0.3076 0.7552 <![CDATA[QADD-2 nd QA]]> 100％ 0.0760±0.0521 0.3928±0.051 0.7064±0.0651 <![CDATA[QADD-3 rd QA]]> 100％ 0.5166±0.0827 0.4489±0.013 0.6645±0.0347 QADD 100％ 0.8489±0.0192 0.3410±0.024 0.6139±0.0291

[0041] Table 2: Performance of different drug design methods in terms of objective function indicators

[0042]

[0043] The evaluation indicators in the table include:

[0044] QED: Quantitative evaluation of drug-likeness, calculated by weighted regularization of eight molecular descriptors: molecular weight, logP, number of hydrogen bond donors, number of acceptors, molecular polar surface area, number of rotatable bonds, number of aromatic rings, and number of alarm structures. The value ranges from [0 to 1], with values ​​closer to 1 indicating higher drug-likeness.

[0045] SAscore: Synthesizability score, which comprehensively considers the frequency of occurrence of molecular substructures in the PubChem database and the structural complexity penalty. The numerical range is [1,10]. The closer to 10, the more difficult the molecule is to synthesize.

[0046] QAscore: molecular mass assessment score, obtained based on the regularization of the predicted score of the molecular mass assessment model. The value range is [0,1]. The closer to 1, the higher the molecular mass.

[0047] Validity: The proportion of valid molecules that meet the valence pricing rules among the generated molecules.

[0048] Success rate: the proportion of generated molecules that meet QED>0.605 and SAscore<2.797 (the average value of CHEMBL bioactive molecule database).

[0049] Novelty: Evaluates the similarity between the generated molecule and molecules in the CHEMBL database using the formula nov = 1-mean(sim(x, y) | x∈S, y∈S′), where S is the molecule generated by this method, S′ is a molecule in the database, and sim is the Tanimoto similarity between the two molecules. Novelty values ​​range from [0 to 1]; the closer to 1, the more innovative the generated molecule.

[0050] Diversity: Evaluate the similarity between generated molecules Where S is the number of molecules generated by this method, n is the number of molecules in S, and sim is the Tanimoto similarity between two molecules. The diversity value range is [0, 1], and the closer to 1, the more diverse the generated molecules.

[0051] Compared with the existing technology, the effectiveness of the molecules designed by this method reached 100%, proving that the reinforcement learning model has a higher success rate than the existing technology in terms of effectiveness and accuracy, and proving that the molecular quality assessment graph neural network learns the pharmaceutical and chemical properties of high-quality drug-like molecules and successfully guides the generation of high-quality molecules as a reward function for reinforcement learning; As shown in Table 1, the novelty of the designed molecules is higher than that of the previous method, and the diversity is slightly lower than the highest method, proving that the reinforcement learning model used can explore a chemical space that is different from the distribution of drug molecules contained in the existing database; As shown in Table 1, QADD-2 ndQA and QADD-3rdQA are the generative models after the second and third iterations, respectively, and QADD is the final model. After multiple iterations, the generated molecules show significant improvements in the optimization metrics QED and SAscore, demonstrating the effectiveness of feedback iteration. As shown in Table 2, the average QED metric of the designed molecules is higher than that of previous methods, and the average QAscore is higher than that of other reinforcement learning-based methods (molDQN and MARS), demonstrating the advantages of the multi-objective design of reinforcement learning adopted in this technology.

[0052] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principles and purpose of the present invention. The scope of protection of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. All implementation schemes within its scope shall be subject to the constraints of the present invention.

Claims

1. A drug molecular structure auxiliary design method based on neural network, characterized in that: A multi-objective reinforcement learning model is used to batch generate small molecules. A benchmark dataset is constructed using drug molecules included in a database as positive samples and low-quality molecules generated by the reinforcement learning model as negative samples. This is then fed into a graph neural network (GIN) model for training to obtain a trained molecular quality assessment (QA) model. A score library is constructed, and the quality assessment score (QAscore) is redefined based on the rank of the QA model's predicted scores for new molecules in the score library. The QAscore is added as one of the optimization objectives of the multi-objective reinforcement learning model to regenerate molecules. The molecules generated by the multi-objective reinforcement learning model are fed back into the QA model for retraining. Repeated iterations of the multi-objective reinforcement learning model resulted in no significant improvement in molecular drug-like properties, including: Step 1) training a multi-objective reinforcement learning method to generate molecules in batches; Step 2) Pre-process the generated molecules, specifically: add a The functional groups of the elements, each molecule generates 10 extended molecules containing 1 random functional group and 2 extended molecules containing 2 random functional groups; When the number of functional groups generated for each molecule is less than 12, the maximum number generated shall be used; Step 3) Build a benchmark dataset and extract atomic features to create a molecular graph, specifically including: Step 3.1) Download bioactive molecules from the CHEMBL database, randomly select 154,000 non-duplicate molecules as positive samples, and use the 108,859 expanded molecules generated in step 2) as negative samples to construct a benchmark dataset for the molecular quality assessment model; Step 3.2) Generate a label matrix L from the benchmark dataset; construct a graph using atoms in the molecule as nodes and bonds as edges to generate an adjacency matrix A; extract the 19-dimensional atomic species, 2-dimensional whether the atom is on a ring, 6-dimensional atomic hybridization, 1-dimensional formal charge, and 1-dimensional atomic degree features of the atoms to construct a 29-dimensional node feature matrix X; create a molecular graph G = (A, X), where the nodes of the molecular graph are all atoms except H atoms, and the edges of the molecular graph are the chemical bonds connecting atoms; Step 4) Build and train a molecular mass assessment model, specifically including: Step 4.1) Construct a molecular mass assessment model consisting of five GIN layers, one pooling layer, and one readout layer. The GIN layer consists of a graph convolution (GCN) layer, a batch normalization (BN) layer, and a ReLU activation function. The input of this molecular mass assessment model is a molecular graph G = (A, X), and the output is the embedded features of the entire molecular graph. After passing the softmax function, the classification probability is obtained. Step 4.2) training the molecular mass assessment model obtained in step 4.1; Step 5) Construct a score library and define the molecular quality assessment score QAscore; establish a score library based on the predicted scores of the molecules in the benchmark dataset, and define the quality assessment score according to the ranking of the predicted scores of the molecules to be predicted in the score library , where x is the ranking of the molecule to be predicted, n and m are the number of positive and negative samples in the score library respectively; the QAscore value range is [0,1], and the closer to 1, the higher the quality of the molecule; Step 6) Add QAscore as the reward of the multi-objective reinforcement learning model, do not retain the relevant parameters of the reinforcement learning model in the previous round, and regenerate 5000 episodes; Step 7) The generated molecules are fed back to the QA model for further training, and the training is repeated until the QED and SAscore indicators of the generated molecules no longer show significant improvement or the model no longer converges.

2. The drug molecular structure auxiliary design method based on neural network according to claim 1, characterized in that: The batch generation of molecules is achieved by the following methods: Step 1.1) Initialize the memory pool M with size N and set the hyperparameter frequency to 100. Step 1.2) Initialize the evaluation Q network; Step 1.3) Initialize the target Q network, fix the network parameters, and set the initial values ​​to be the same as those of the evaluation Q network; Step 1.4) Train the evaluation Q network and the target Q network; Step 1.5) Set the update frequency to 100. Whenever the update frequency is divisible by the time step, update the target Q network parameters to make them the same as the current evaluation Q network. Step 1.6) Each round selects the molecular state of the last time step As the generated molecules, the generated molecules output from the last 1000 rounds were selected.

3. The drug molecular structure auxiliary design method based on neural network according to claim 2, characterized in that: The step 1.4) specifically includes: ① Select and execute the action of the current time step according to the action selection strategy , receive rewards and the next state Then store it in the memory pool M; The action of the current time step ,in: is the action set of the current time step, is the molecular state at the current time step, To evaluate the Q network, To evaluate the Q network parameters, is a hyperparameter; ② Randomly sample batches from M by sampling from memory pool M ( ) and are calculated by the evaluation Q network , calculated by the target Q network ,in: is the discount factor, is the target Q network; then calculate the MSE loss function Then perform loss back propagation; ③ Execute steps ① and ② above with 1 to 60 time steps per round, and loop through 1 to 5000 rounds to complete training.

4. The drug molecular structure auxiliary design method based on neural network according to claim 1, characterized in that: The molecular mass assessment model is trained in the following manner: a) The graph convolution layer aggregates the node information of the molecular graph. The features of each node are obtained by the weighted sum of the features of the node in the previous layer and the features of the neighboring nodes through MLP nonlinear transformation. Specifically: In the Embedding features of the layer ,in: For nodes In the The embedding features of the layer, For nodes The set of neighbor nodes of is with Weight-related hyperparameters, is a nonlinear transformation; the readout layer will output the embedding features of the entire graph, which is calculated by aggregating the information of all nodes on the graph: ,in: For molecular graphs, is the number of layers of the neural network; b) The loss function was set to CrossEntropy loss, using the Adam optimizer. The learning rate lr was set to 0.01 and a planned decay was performed, decaying by 0.5 times every 20 rounds. All molecular graph data were divided into training set, validation set, and test set in a ratio of 7:2:

1. 10-fold cross-validation was used, and the batch size was set to 1024. After 200 epochs, the model training was completed.

5. A system for implementing the neural network-based drug molecular structure auxiliary design method according to any one of claims 1 to 4, characterized in that: include: Reinforcement learning model unit, graph neural network unit and iterative feedback unit, wherein: the graph neural network unit is trained according to the drug small molecule information included in the drug database to obtain a quality assessment model; the reinforcement learning model unit uses the predicted score of the quality assessment model and the molecular physicochemical property indicators as the reward function to generate small molecules in batches; the iterative feedback unit uses the low-quality small molecules generated as negative samples, and feeds them back to the graph neural network unit for retraining to obtain a new quality assessment model and update the reward function in the reinforcement learning model unit, and iterates repeatedly.

Images