High crystal density molecule generation model and generation method
Through the high crystal density molecular generation model based on deep learning, the problem of difficult to predict and design high crystal density energy-containing materials in the prior art is solved, and efficient, safe and low-cost high crystal density molecular generation and prediction are achieved.
Patent Information
- Application Number
- CN202411296995.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2025-02-14
AI Technical Summary
The prior art is difficult to effectively predict and design energy-containing materials with high crystal density, and experiments on synthesis of these materials are dangerous and costly, making it difficult to obtain a large number of diversified energy-containing materials with high crystal density.
A high crystal density molecule generation model based on artificial intelligence deep learning is adopted to generate the target molecular structure by constructing the initial data set, training the BERT model, diffusion model and CVAE model, and predicting and generating high crystal density molecules.
It realizes the rapid, simple and efficient generation of high-crystalline molecules, and reduces the synthesis cost, avoids the danger of laboratory synthesis, and provides a diverse design of high-crystalline energy-containing materials.
Smart Images
Figure BDA0005046335000000051 
Figure BDA0005046335000000052 
Figure BDA0005046335000000081
Abstract
Description
Technical Field
[0001] The present invention relates to new molecular structure prediction and generation technology, and specifically to a high crystal density molecular generation model and generation method based on artificial intelligence deep learning. Background Art
[0002] Accelerating the discovery of energetic materials with high energy density has always been an important issue in the field of energy storage and conversion. As the most basic application entity unit, the crystal structure significantly affects the energy, safety and other properties of energetic materials. Therefore, the crystal density is an important parameter that determines the detonation performance of all-nitrogen materials. The research on design methods oriented to the crystal scale is of great significance to the creation of new energetic materials.
[0003] Since the famous TNT was first synthesized in 1863, several milestone EMs have been discovered in the course of more than 150 years of development, such as FOX-7, HNIW, TNAZ, etc. Among them, CL-20 is generally considered to be the molecule with the highest energy density (expressed in terms of detonation velocity) at present. Given the increasing difficulty of the synthesizability of compound molecules, it is obvious that finding new molecules with higher crystal density than CL-20 by increasing the strain energy and the number of nitro groups remains a challenge for EMs. However, since the synthesis of energetic materials is still in the laboratory exploration stage, the possible dangers and relatively high costs of synthetic experiments make it difficult to obtain actual samples. How to obtain a large number of diverse and personalized energetic materials with high crystal density remains a huge challenge worldwide.
[0004] Therefore, the energy properties of all-nitrogen materials, especially the prediction of crystal density and its molecular design, can only be predicted through theoretical methods, which cannot meet actual needs. Summary of the invention
[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a high crystal density molecule generation model.
[0006] To this end, the high crystal density molecule generation provided by the present invention is constructed using the following method steps:
[0007] Step 1, constructing an initial data set, wherein the initial data set includes a number of molecules with known crystal density;
[0008] Step 2: Use the initial data set to train the BERT model to obtain a crystal density prediction model; the crystal density prediction model is used to predict the crystal density value of a molecule, and its input is the SMILES data of each molecule in the initial data set, and the output result is the predicted crystal density of the corresponding molecular structure;
[0009] The diffusion model is used to expand the initial data set to obtain expanded molecular data, and the crystal density prediction model is used to predict the crystal density of the expanded molecules to obtain an expanded data set containing the initial data set and the expanded new data; the node feature matrix, edge feature matrix and three-dimensional tensor of each molecule in the initial data set are input to the diffusion model;
[0010] Step 3, using the expanded data set to train the CVAE model to obtain a molecular generation model; the molecular generation model generates a target molecular structure according to the input target crystal density value, the input of the molecular generation model is the crystal density value, and the output result is the SMILES data of the target molecular structure.
[0011] An optional solution is that the molecules in the initial data set are energetic compound molecules containing C, H, O and N atoms. Or the molecules in the initial data set are energetic compound molecules containing C, H, O, F and N atoms.
[0012] A preferred solution is that the crystal density value of the molecules in the initial data set is greater than or equal to 1.6, so as to increase the probability of obtaining a molecular structure with a high crystal density value.
[0013] The present invention also provides a method for generating high crystal density molecules, the method comprising: using the above-mentioned molecular generation model to generate a high crystal density molecular structure. Optionally, the crystal density value of the high crystal density molecular structure is greater than or equal to 1.8.
[0014] The present invention uses a diffusion model, a conditional variational autoencoder model, and a BERT model to generate molecules with a specific value of crystal density. It has been verified that the present invention is simple, efficient, safe, and low-cost. It is not limited to the synthesis of chemical laboratories, is more diverse, and has important significance for exploring molecules with high crystal density. It can guide the design and synthesis of more energetic compounds with high crystal density. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 The crystal density distribution of the selected energetic molecules in the initial data set used in the example.
[0016] Figure 2 The loss changes during the BERT model training process in the embodiment.
[0017] Figure 3 is the change of the diffusion model loss function with step in the embodiment; Figure 3 A is the cross entropy loss of oxygen atoms during training; Figure 3 B is the cross entropy loss curve of hydrogen atoms; Figure 3 C is the cross entropy loss of nitrogen atoms; Figure 3 D is the cross entropy loss of the fluorine atom; Figure 3 E is the cross entropy loss of the double bond;
[0018] Figure 3 F is the cross entropy loss of carbon atoms.
[0019] Figure 4 The loss changes of the training set and the test set during the CVAE model training process in the embodiment. DETAILED DESCRIPTION
[0020] Unless otherwise specified, the scientific and technical terms used herein are understood according to the knowledge of ordinary technicians in the relevant fields.
[0021] The present invention generates target molecules based on three deep learning models. First, an existing molecular data set (i.e., a node feature matrix and an edge feature matrix of related molecules) is imported into a diffusion model (DiGress model) for training and data expansion, so as to generate a large number of molecules similar to the data set. Then, a Bidirectional Encoder Representations from Transformers model (BERT model) trained based on crystal density prediction is used to predict the crystal property density of the previously generated molecules. Finally, a conditional variational autoencoder model (CVAE model) is trained using the amplified data set to obtain a high crystal density molecule generation model that can generate target molecules for specific crystal density values.
[0022] The DiGress, BERT model and CVAE model used in the present invention all adopt the existing related network model structure, and the training data set can be constructed based on the purpose of the present invention and the input and output data disclosed in the scheme of the present invention. The DiGress model adopts a discrete denoising diffusion method to generate a graph structure with category nodes and edge attributes. The DiGress model can process a graph structure with category nodes and edge attributes. When the graph structure is used to represent chemical molecules, the nodes represent atoms and the edges represent chemical bonds. The training data set usually includes a series of molecular graphs, each of which has category labels for nodes (atoms) and edges (chemical bonds). In order to train the DiGress model, it is necessary to extract features from these molecular graphs and encode the attributes of the nodes (such as atom types) and the attributes of the edges (such as chemical bond types). Such as One-hot encoding, the purpose of One-hot encoding is to convert discrete edge attributes (such as chemical bond types) into numerical form so that it can be used for model calculation.
[0023] The node feature matrix of the molecular structure described in this article is a matrix constructed by one-hot encoding each atom and atom type in the molecular structure. The number of rows of the node feature matrix is the number of atoms in the molecular structure, and the number of columns is the number of atom types in the molecular structure; each line of the node feature matrix is the one-hot encoding of each atom; in the preferred embodiment, the encoding order of all atoms in the molecular structure depends on the serial number of each atom in the molecular structure arranged according to the chemical structure formula rules. The edge feature matrix of the molecular structure is a matrix constructed by one-hot encoding each chemical bond and chemical bond type in the molecular structure. The number of rows of the edge feature matrix is the number of chemical bonds in the molecule; the number of columns is the number of chemical bond types in the molecule; each line in the edge feature matrix is the one-hot encoding of each chemical bond; the chemical bond type is determined according to the classification rules of chemical bonds in the chemical field, such as single bonds, double bonds, triple bonds, ionic bonds, covalent bonds, metallic bonds, etc. The three-dimensional tensor of the molecular structure is constructed based on the node feature matrix and edge feature matrix of the corresponding molecular structure, and the dimension of the three-dimensional tensor is N*N*D e , where: N is the number of atoms in the molecular structure, D e is the number of chemical bonds in the molecular structure.
[0024] In order to more clearly understand the construction process of the node feature matrix, edge feature matrix and three-dimensional tensor of the molecular structure described in this article, H2O is taken as an example for explanation:
[0025] It has three atoms, namely three nodes, which are two hydrogen atoms (H) and one oxygen atom (O); the atom types are two categories, namely hydrogen atom (H) and oxygen atom (O); secondly, it has two chemical bonds, namely two edges, which are the chemical bonds between the oxygen atom and the two hydrogen atoms; the chemical bond type is one category, specifically single bond (Single Bond).
[0026] In the node coding, according to the atomic numbering order H1, O, H2; there are two types of atoms; nodes H1 and H2 are encoded as: [1,0] (the first column is 1 for H, and the second column is 0 for non-O); node O is encoded as [0,1] (the first column is 0 for non-H, and the second column is 1 for O). The obtained node feature matrix x is:
[0027]
[0028] In the edge feature coding, the coding order is (H1, O) chemical bond and (O, H2) chemical bond. The value of the chemical bond type is single bond, and the single bond coding is defined as [1]. After one-hot coding each edge, the edge feature matrix E edges for:
[0029]
[0030] In the construction of the three-dimensional tensor EEE, the dimension of the three-dimensional tensor E is 3*3*2; the H1-O edge (from node 1 to node 2) is represented in E as E[1,2,:]=[1], and the O-H2 edge (from node 2 to node 3) is represented in E as E[2,3,:]=[1]; the three-dimensional tensor E can be expressed as:
[0031] E=([[0],[1],[0]],[[1],[0],[1]],[[0],[1],[0]])
[0032] In real molecules, there may be no direct chemical bonds between nodes (i.e. no edges). Suppose there are two atoms in a molecule with no bonds between them, for example, there is no edge between A-BA-BA-B. In the three-dimensional tensor EEE, this situation can be represented as [0] or a specific invalid category code.
[0033] For more details on the construction process of the node feature matrix and edge feature matrix of the molecular structure described above, please refer to the paper: DiGress: Discrete Denoising diffusion models for graph generation.
[0034] The specific embodiments of the present invention are given below. It should be noted that the following embodiments are preferred examples of the present invention. Based on the disclosure of the present invention, skilled technicians can optimize technical factors such as the range of the conditional variational autoencoder latent space, the accuracy of the Bidirectional Encoder Representations from Transformers prediction model, etc. within the scope of the disclosure of the present invention to obtain other implementation plans that fall within the protection scope of the present invention.
[0035] Embodiment 1:
[0036] This embodiment is a training of a specific high crystal density molecule generation model.
[0037] Initial dataset construction:
[0038] The initial data set used in this embodiment contains 26,407 molecules in total. The SMILES data of each molecule are processed using the RDkit program to obtain the corresponding GRAPH data. The GRAPH data are then used to construct the node feature matrix, edge feature matrix and three-dimensional tensor of the molecule. The data set is divided into a training set, a validation set and a test set in a ratio of 8:1:1.
[0039] The structural characteristics of the 26407 molecules selected in this embodiment are: each molecule is an energetic compound composed of carbon, hydrogen, nitrogen, fluorine and oxygen atoms, contains any type of nitrogen-oxygen bond (e.g., single, double, triple), and the crystal structure cannot be solvated or one of the cocrystals (i.e., the crystal structure contains only one type of molecule), and its crystal density value is disclosed by the prior art. The crystal density distribution of the 26407 molecules is shown in Figure 1. In order to meet the needs of the subsequent network training, the crystal density values of all molecules are normalized so that the normalized crystal density value is (0, 1].
[0040] BERT model training:
[0041] The training data are SMILES data of 26,407 molecules in the initial data set and the normalized crystal density values of related molecules disclosed in the prior art.
[0042] The BERT model used in this embodiment adopts the open source PubChem10M_SMILES_BPE_450k model. First, the trained PubChem10M_SMILES_BPE_450k model is directly imported, and then the data set is used for training. Finally, the trained crystal density prediction model is obtained, in which R 2 The average value of the regression line is 0.8624 and the RMSE is 0.04857.
[0043] During the training process of this embodiment, the mean square error loss function is used to calculate the gap between the model output and the actual target value. At the same time, in order to address the problem of unbalanced data sets, sample weights are added on the basis of MSELoss, so that the model can adjust the loss value according to the different importance of samples. That is, samples with crystal density values greater than threshold (1.9) will receive higher weights (weight_high=2), while samples with crystal density values less than or equal to the threshold will receive lower weights (weight_low=1). This makes the model more sensitive to the errors of high-value samples, achieving the purpose of processing target values with high variance.
[0044] The hyperparameters during training are set to: batch_size is 32, epoch is 30, and learning rate is 1 -5 ; The indicator for stopping training is: allow the verification loss not to improve within a certain number of epochs, then stop training; if the verification loss does not improve within 5 consecutive epochs, stop training; the loss changes during the training process of this embodiment are as follows Figure 2 As shown in Table 1, the actual and predicted values of the crystal density predicted by the trained BERT model are shown.
[0045] Table 1 Crystal density predicted based on BERT model
[0046] SMILES Actual crystal density value Predicted crystal density values [O-][N+](=O)C1=CC=CO1 1.571 1.541094 [O-][N+](=O)C1=NNC=C1 1.647 1.599689 [O-][N+](=O)C1=CC=NO1 1.638 1.591328 [O-][N+](=O)C1=COC=C1 1.572 1.53717 [O-][N+](=O)C1=NOC=C1 1.664 1.602928 [O-][N+](=O)C1=CN=CO1 1.659 1.615041 [O-][N+](=O)C1=CNC=N1 1.682 1.623931 [O-][N+](=O)C1=CNN=N1 1.726 1.68186 [O-][N+](=O)C1=CNN=C1 1.627 1.609918
[0047] Data augmentation:
[0048] The node feature matrix, edge feature matrix and three-dimensional tensor of the initial data set are used to construct the DiGress model (the DiGress model used in this embodiment is specifically: GitHub-cvignac / DiGress:code for the paper"DiGress: Discrete Denoising diffusion for graph generation" The DiGress model published in the literature was used for training and data expansion. During the model training process, effectiveness was selected as the training stop criterion. When the effectiveness reached the highest value, the training stopped. Figure 3 It shows how the loss function changes with the step. From the content of the figure, the AF graph shows the changes in the cross-entropy loss (Cross-Entropy Loss) of the model during the training process for different types of atoms or bonds. Figure 3 A shows the cross entropy loss of oxygen atoms during training, indicating that the model has large fluctuations in the prediction of oxygen atom features, and the final loss value is 0.3244; Figure 3 B shows the cross entropy loss curve of hydrogen atoms, which also shows significant fluctuations. The final loss value is 0.3719, indicating that the model has a large prediction error in the characteristics of hydrogen atoms. Figure 3 C shows the cross entropy loss of nitrogen atoms. The final loss value is 0.2711, indicating that the model predicts nitrogen atoms slightly better than oxygen and hydrogen, but there are still large fluctuations; Figure 3 D shows the cross entropy loss of fluorine atoms. The initial loss is high but decreases rapidly, and the final loss is 0.000911, indicating that the model performs very well in predicting the characteristics of fluorine atoms, converges quickly and has very low errors; Figure 3 E shows the cross entropy loss of the double bond. The loss curve is relatively stable, and the final loss is 0.1574, indicating that the model predicts the double bond features relatively accurately. Figure 3 F shows the cross entropy loss of carbon atoms. The curve fluctuates greatly and the final loss value is 0.3586, indicating that the model has a high error in predicting carbon atoms and needs further optimization. This embodiment finally chooses to stop training at 2500 steps.
[0049] Table 2 shows the differences in SA score (Chinese name: synthesizability) and crystal density mean between the molecules (4755) unconditionally generated by DiGress in this embodiment and the molecules in the initial data set (26362).
[0050] The diffusion model finally trained can be used for the crystal density dataset, achieving an unconditional molecular validity of 40.39%, a uniqueness of 99.8%, and a novelty of 98.72%.
[0051] Table 2
[0052]
[0053] The values of the newly obtained 4755 crystal densities after the above operation expansion are calculated by the BERT crystal density prediction model trained in Example 1; an expanded data set containing 26407+4755 molecules is constructed;
[0054] Training of CVAE model:
[0055] The CVAE model was trained using the expanded data set, which was divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The input during the training process was the SMILES data of each molecule in the initial data set in step 1 and the normalized crystal density value of each molecule.
[0056] The data input to the model is the crystal condition vector, and the label is SMILES code; the encoder and decoder in the CVAE model used in this embodiment adopt the long short-term memory network (LSTM), which consists of 3 layers of RNN, each layer has 500 hidden nodes; a softmax layer is used in the output of each decoding unit, and cross entropy is used as the cost function of the reconstruction error. The condition vector is embedded in the encoding and decoding process to control the specific properties of the generated molecule.
[0057] The total loss function of the training process can be expressed as the weighted sum of the reconstruction loss and the KL divergence loss, as shown in formula (1), where β is a hyperparameter used to balance the influence of the reconstruction loss and the KL divergence loss. In this model, after experiments, β = 1 was selected, that is, the KL divergence loss and the reconstruction loss are equally weighted in the total loss.
[0058] Total Loss=Reconstruction Loss+β×KL Divergence Loss (1)
[0059] The specific relevant parameters in the training process are batch_size is 128; latent_size is 200 in CVAE; the dimension of the latent space is 200; unit_size is 512; the hidden layer size of each RNN unit is 512, and the final learning rate is 0.0001; Figure 4 The loss changes of the training set and the test set during the training process are shown. The 33rd epoch when the loss function of the test set is the smallest is selected as the best model, with a loss of 1.962.
[0060] Embodiment 2:
[0061] The CVAE model trained in Example 1 was used for molecule generation and sampling 100 times (the target crystal density value of 1.9 was input for each sampling), and the final conditional generated molecule effectiveness was 26.86%, and 22 molecules with a crystal density of more than 1.8 were obtained, accounting for 1.26%. Table 3 shows the top 10 small molecules with the highest crystal density predicted from the small molecules generated based on the joint model, indicating that the constructed molecular generation model has a strong ability to generate novel, effective, and unique molecules.
[0062] Table 3 Small molecules generated based on the joint model and their predicted crystal density
[0063]
[0064]
[0065]
[0066] The method of the present invention achieves the goal of quickly generating high crystal density molecules through a deep learning model. It has been verified that the invention can generate high crystal density molecules simply and efficiently and predict their crystal density values. The method of the present invention is not limited to the structure of energetic compounds and has good universality. It is of great significance to the discovery of high-energy compounds and can guide the generation of energetic compounds in the future.
Claims
1. A high crystal density molecular generation model, characterized in that: The model building method comprises: Step 1, constructing an initial data set, wherein the initial data set includes a number of molecules with known crystal density; Step 2: Use the initial data set to train the BERT model to obtain a crystal density prediction model; the crystal density prediction model is used to predict the crystal density value of a molecule, and its input is the SMILES data of each molecule in the initial data set, and the output result is the predicted crystal density of the corresponding molecular structure; The diffusion model is used to expand the initial data set to obtain expanded molecular data, and the crystal density prediction model is used to predict the crystal density of the expanded molecules to obtain an expanded data set containing the initial data set and the expanded new data; the node feature matrix, edge feature matrix and three-dimensional tensor of each molecule in the initial data set are input to the diffusion model; Step 3, using the expanded data set to train the CVAE model to obtain a molecular generation model; the molecular generation model generates a target molecular structure according to the input target crystal density value, the input of the molecular generation model is the crystal density value, and the output result is the SMILES data of the target molecular structure.
2. The high crystal density molecular generation model according to claim 1 or 2, characterized in that: The molecules in the initial data set are energetic compound molecules containing C, H, O and N atoms.
3. The high crystal density molecular generation model according to claim 1 or 2, characterized in that: The molecules in the initial data set are energetic compound molecules containing C, H, O, F and N atoms.
4. The high crystal density molecular generation model according to claim 1, characterized in that: The crystal density value of the molecules in the initial data set is greater than or equal to 1.
6.
5. A method for generating high crystal density molecules, characterized in that: The method comprises: using the molecular generation model of claim 1 to generate a high crystal density molecular structure.
6. The high crystal density molecular generation model according to claim 1, characterized in that: The crystal density value of a high crystal density molecular structure is greater than or equal to 1.8.