Molecular property interpretation method based on deep learning

Abstract

The invention discloses a molecular property interpretation method based on deep learning, and belongs to the technical field of molecular property interpretation, and the method comprises the steps: predicting molecular properties based on a pre-trained GNN model, and obtaining molecular node embedding; designing an interpreter, updating molecular node embedding based on a Transform to obtain node features, determining edges in a molecular substructure by using a hierarchical edge selector, and designing a reward function in order to limit and guide the generation process of the substructure; performing iterative training on the interpreter through the designed loss function until the loss function meets a set loss value, completing training learning, and obtaining a trained interpreter; and embedding and inputting molecular nodes into the trained interpreter, and outputting a substructure which plays a key role in molecular properties. The accuracy of molecular property interpretation is improved, and high implementation feasibility is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of molecular property interpretation, and specifically relates to a molecular property interpretation method based on deep learning. Background Art

[0002] The physicochemical properties of molecules (such as solubility, reactivity, biological activity, etc.) are of great significance in drug discovery, materials science, bioinformatics, and fine chemicals. Accurately interpreting molecular properties not only helps to understand the structure-property relationship (SPR) of molecules, but also accelerates the design of new materials, the screening of new drugs, and the prediction of chemical reactions. Traditionally, the study of molecular properties mainly relies on methods such as quantum chemical calculations (such as density functional theory DFT), molecular dynamics simulations, and quantitative structure-activity relationship (QSAR). Although these methods can provide high accuracy, they are usually computationally expensive, especially when dealing with large-scale molecular libraries, which consumes a lot of computing resources. In addition, these methods are highly interpretable, but rely on expert experience and are difficult to apply automatically on a large scale. In recent years, machine learning (ML) and deep learning (DL) have made significant progress in the field of molecular property prediction. Researchers have tried to use methods such as graph neural networks (GNN) to directly learn features from molecular graphs, avoiding the complexity of manually constructed features. Although GNN models perform well in molecular property prediction tasks, they are usually regarded as "black box" models, and it is difficult to directly understand which molecular substructures play a key role in the prediction results; and existing GNN models are difficult to effectively identify functional groups, conjugated systems or active centers that affect molecular properties, resulting in limited interpretability and generalization capabilities of GNN models.

[0003] At present, there are many methods applied to the interpretability of molecular property prediction models at home and abroad, and have achieved certain results in practical applications. Among them, the Grad-CAM method proposed by Selvaraju et al. is a technology commonly used for molecular property interpretation. This method identifies the key molecular substructures that determine the molecular properties by calculating gradient weights. The GNNExplainer designed by Ying et al. uses an edge mask mechanism to determine the key substructures that affect molecular properties and their corresponding feature subsets. The PGExplainer proposed by Luo et al. introduces a parameterized explanation generator and uses an independent multi-layer perceptron (MLP) to learn the substructures in the molecular structure that contribute most significantly to the properties. The FlowX method proposed by Gui et al. introduces the concept of message flow, and determines the substructure that contributes most to the prediction of molecular properties by tracing the message passing process.

[0004] In the task of molecular property interpretation, reinforcement learning (RL) and contrastive learning (CL) are widely used to improve the interpretability and robustness of molecular property prediction, and play an important role in key substructure identification, feature learning and optimization. Contrastive learning trains the representation of molecular substructures by constructing positive and negative sample pairs, making the substructure features more stable and more discriminative, and enhancing the generalization ability of the model. In the process of molecular property interpretation, contrastive learning can help the model more effectively identify the key structures that affect molecular properties and reduce the impact of noise. Reinforcement learning selects the optimal substructure through exploration and utilization mechanisms, and uses reward mechanisms to optimize the extraction strategy of key substructures, thereby further improving the interpretability, consistency and stability of molecular property interpretation. Compared with traditional attention-based or gradient-based methods, reinforcement learning can more effectively mine core substructures with chemical or physical significance and improve the reliability of the model. Wang et al. proposed RCExplainer to combine reinforcement learning with contrastive learning to build an end-to-end interpretation framework. In this method, contrastive learning is used to learn more robust molecular substructure representations, while reinforcement learning is used to optimize the substructure selection process, thereby significantly improving the stability, interpretability and generalization ability of molecular property interpretation, providing an efficient and interpretable analysis tool for fields such as chemistry, drug discovery and materials science. Summary of the invention

[0005] In view of the above-mentioned problems existing in the prior art, the present invention proposes a molecular property interpretation method based on deep learning, which is reasonably designed, solves the shortcomings of the prior art, and has good effects.

[0006] A molecular property interpretation method based on deep learning, comprising the following steps: S1. Predict molecular properties based on the pre-trained GNN model to obtain molecular node embedding; S2. Design an interpreter to update the molecular node embedding based on the graph transformer to obtain node features, use the hierarchical edge selector to determine the edges in the molecular substructure, and design a reward function to restrict and guide the generation process of the substructure; S3. Iteratively train the interpreter through the designed loss function until the loss function meets the set loss value, completes the training and learning, and obtains the trained interpreter; S4. Embed the molecular nodes into the trained interpreter and output the substructures that play a key role in the molecular properties.

[0007] Furthermore, in S1, the molecule is formally represented as an undirected graph ,in is a collection of nodes, corresponding to atoms in the molecule, is the set of edges, corresponding to the chemical bonds in the molecule; The initial feature representation of the node is ,in, Indicates The feature vector of each node, is the number of nodes, is the dimension of each node feature; after the initial features are encoded by the GNN model, a node embedding representation is generated , is the dimension of node embedding.

[0008] Furthermore, in S2, the query matrix is ​​calculated in the self-attention mechanism of the graph Transformer , key matrix Sum Matrix , the expression is: ; (1) in, , , is a learnable weight matrix, are the dimensions of the key matrix, query matrix, and value matrix, , is the number of attention heads; For each attention head, use the query and key Calculating attention weights , the expression is: ; (2) Using attention weights Pair Matrix A weighted sum is performed to generate the output of the attention head, expressed as: ; (3) In the multi-head attention mechanism, the outputs of all attention heads are concatenated together, expressed as: ; (4) in, is the concatenated multi-head attention output, Concatenate operation: It means concatenating the outputs of all attention heads together. For the The output of an attention head is is the number of attention heads; right Perform linear transformation and add residual connection and activation function to obtain updated node features , the expression is: ; (5) in, is a nonlinear activation function, is the trainable parameter matrix of the linear transformation.

[0009] Furthermore, in S2, the substructures used to explain the molecular properties predicted by the model are determined by maximizing the mutual information metric , the expression is: ; (6) in, is the mutual information, which is used to measure the substructure and The degree of information sharing between is to have Edge graph The collection of subgraphs of It is a picture The prediction results in the predictive property model GNN, =0 means the molecule is inactive, =1 means the molecule is active; According to the knowledge of mutual information, the above formula becomes: ; (7) in, for The Strip edge, Indicates that given the predicted molecular properties In the case of Probability distribution of occurrence; The generation of substructure is regarded as a Markov chain process, and the substructure is obtained using the following formula Edge: ; (8) in, is the first The best choice of edge, For the Any edge in the candidate edge set, is the conditional entropy, which means that given and In the case of uncertainty, Indicates The substructure after the step, Indicates adding to the current interpretation subgraph The set of candidate edges; Design a hierarchical edge selector to find the first The edge, in The probability of each candidate edge is calculated at each step, and the edge with the highest score is selected. , the first MLP layer edge represents the generator Perform the following processing: ; (9) in, , Respectively represent Nodes and Nodes in The characteristics of and , Represents edge The edge representation of The second MLP layer edge possibility generator The possibility of generating an edge, that is, the probability score of the edge, is expressed as: ; (10) in, is a connection operation, is the representation of the molecular graph obtained by the prediction model GNN, It is the edge The edge with the highest score will be added to the substructure; the edge that has been selected into the substructure will not enter the candidate edge set again; repeat until the termination condition is met until.

[0010] Furthermore, in S2, the reward function for: ; (16) in is a hyperparameter used to balance the two rewards; for and substructure Molecular Diagram The combined contribution evaluation of the GNN model prediction is expressed as: ; (15) in, represents a model for predicting molecular properties; for The contribution to the explanation of the molecular activity is evaluated as: ; (14) in, For the nodes, For the nodes, is the first substructure, Is with A collection of substructures with the same predicted label, Is with A collection of substructures with different tags.

[0011] Furthermore, in S3, the designed loss function for: ; (17) in, is a hyperparameter used to ensure that the input to the logarithmic function is always greater than 0; During the training process, the loss function is first optimized using stochastic gradient descent or Adam optimizer so that the interpreter can learn the optimal substructure selection strategy; the training data consists of molecular graphs and their corresponding property labels. At each iteration, the model selects the most representative substructure based on the current molecular structure and calculates the loss function to update the parameters; to improve the generalization and interpretability of the model, the hyperparameters are fine-tuned, and grid search or Bayesian optimization is used to find the optimal parameters; to prevent the model from overfitting, Dropout, weight decay or data augmentation technology are introduced to improve the stability and explanatory power of the model.

[0012] Beneficial technical effects brought by the present invention: The present invention introduces Graph Transformer into the interpretation of molecular properties, and enhances the global interaction modeling of nodes and edges of molecular graphs through the self-attention mechanism, thereby improving the accuracy of molecular property interpretation. The hierarchical edge selector optimizes the selection of key edges in combination with the multi-level structure of deep learning, and has technical uniqueness; it can improve the interpretation ability of molecular graphs and is widely used in chemistry, drug development and other fields. Through the substructure reward mechanism, it encourages the priority selection of high-confidence edges, which improves the stability and interpretability of the model; combined with the self-attention mechanism and reinforcement learning, the molecular graph interpretation process is optimized, manual intervention is reduced, the level of automation and intelligence is improved, and it has a high degree of implementation feasibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flow chart of feature extraction based on graph Transformer in the present invention.

[0014] Figure 2This is a workflow diagram of the hierarchical edge selector in the present invention. DETAILED DESCRIPTION

[0015] The specific implementation of the present invention is further described below in conjunction with specific embodiments: A molecular property interpretation method based on deep learning, comprising the following steps: S1. Predict molecular properties based on the pre-trained GNN model to obtain molecular node embedding; In cheminformatics and computer science, molecules are usually represented as graph structures, with atoms as nodes and chemical bonds as edges. This representation can intuitively depict the topological relationships and chemical properties of molecules. Compared with traditional vector representations, graph structures can not only flexibly adapt to molecules of different sizes and complexities, but also effectively capture key information such as ring structure, aromaticity, and branching in molecules, providing a basis for the prediction and interpretation of molecular properties.

[0016] Formally represent molecules as undirected graphs ,in is a collection of nodes, corresponding to atoms in the molecule, is the set of edges, corresponding to the chemical bonds in the molecule; The initial feature representation of the node is ,in, Indicates The feature vector of each node, is the number of nodes, is the dimension of each node feature; after the initial features are encoded by the GNN model, a node embedding representation is generated , is the dimension of node embedding. In neural networks, local structure and feature information can be effectively captured, but the reliance on local neighborhoods during the update process may lead to the loss of global information. At the same time, as the number of model network layers increases, node embedding may face the problem of over-smoothing of information, making the features of different nodes too similar, reducing the model's discriminative ability.

[0017] S2. Design an interpreter to update the molecular node embedding based on the graph transformer to obtain node features, use the hierarchical edge selector to determine the edges in the molecular substructure, and design a reward function to restrict and guide the generation process of the substructure; In order to overcome the above limitations, graph transformers are introduced when processing molecular node features, such as Figure 1As shown in Figure 1, the self-attention mechanism of the graph Transformer allows each node to pay attention to all nodes in the graph when updating features. At the same time, the structure of the Transformer also allows the use of deeper networks without worrying about the problem of over-smoothing of information. The embedding representation of a given node , calculate the query matrix in the self-attention mechanism , key matrix Sum Matrix , the expression is: ; (1) in, , , is a learnable weight matrix, are the dimensions of the key matrix, query matrix, and value matrix, , is the number of attention heads; For each attention head, use the query and key Calculating attention weights , the expression is: ; (2) Using attention weights Pair Matrix A weighted sum is performed to generate the output of the attention head, expressed as: ; (3) In the multi-head attention mechanism, the outputs of all attention heads are concatenated together, expressed as: ; (4) in, is the concatenated multi-head attention output, Concatenate operation: It means concatenating the outputs of all attention heads together. For the The output of an attention head is is the number of attention heads; right Perform linear transformation and add residual connection and activation function to obtain updated node features , the expression is: ; (5) in, is a nonlinear activation function, is the trainable parameter matrix of the linear transformation.

[0018] The node characteristics after such update That is, the GNN network to be interpreted is used to capture the local structure in the graph and aggregate the information of neighboring nodes, so that the interpreter can better understand the relationship and pattern between local nodes. At the same time, the graph Transformer is used to pay attention to the global node information, and the self-attention mechanism is used to effectively capture long-distance dependencies and important global context information. This combination not only enhances the expressive power of node features, but also facilitates the interpretation of the molecular property prediction model, making the identification of key substructures more accurate and efficient. At the same time, this method helps to reasonably determine the order of candidate edges in the substructure that affects the molecular properties, so as to more accurately extract structural information that has an important impact on the molecular properties. By comprehensively considering global and local features, It can provide a richer and more comprehensive molecular representation, thereby improving the accuracy, stability and interpretability of molecular property interpretation.

[0019] Typically, the substructures that explain the molecular properties predicted by the model are determined by maximizing the mutual information metric , the expression is: ; (6) in, is the mutual information, which is used to measure the substructure and The degree of information sharing between is to have Edge graph The collection of subgraphs of It is a picture The prediction results in the predictive property model GNN, =0 means the molecule is inactive, =1 means the molecule is active; According to the knowledge of mutual information, the above formula becomes: ; (7) in, for The Strip edge, Indicates that given the predicted molecular properties In the case of Probability distribution of occurrence; As can be expected, as the explanation subgraph The number of edges included The increase in possible candidate subgraphs The number will increase super-exponentially, which means that it is difficult to optimize it directly.

[0020] To solve this problem, the generation of substructures is regarded as a Markov chain process and the following formula is used to obtain the substructure Edge: ; (8) in, is the first The best choice of edge, is any edge in the candidate edge set of step s, is the conditional entropy, which means that given and In the case of uncertainty, Indicates The substructure after the step, Indicates adding to the current interpretation subgraph The set of candidate edges of ; Design a hierarchical edge selector to find the first The edge, in The probability of each candidate edge is calculated at each step, and the edge with the highest score is selected. , the first MLP layer edge represents the generator Perform the following processing: ; (9) in, , Respectively represent Nodes and Nodes in The characteristics of and , Represents edge The edge representation of The second MLP layer edge possibility generator The possibility of generating an edge, that is, the probability score of the edge, is expressed as: ; (10) in, is a connection operation, is the representation of the molecular graph obtained by the prediction model GNN, It is the edge score; For having 5 nodes and For a molecular graph with 6 edges, the hierarchical edge selector process is as follows Figure 2 shown. By fusing the edge representation, the current substructure graph representation, and the entire molecular graph graph representation, the probability score of each candidate edge is calculated. The edge with the highest score will be added to the substructure; the edge that has been selected into the substructure will not enter the candidate edge set again; repeat until the termination condition is met. until.

[0021] In order to restrict and guide the generation process of substructures, the following reward mechanism is designed.

[0022] side Validity: For the edges selected by the substructure reward mechanism , it should be a model for predicting molecular properties About Molecular Graphs The prediction results of molecular properties are considered as a classification task. The classification categories are ,in is the number of classification categories.

[0023] Clustering methods can be used to Partition into subgraphs: ; (11) ; (12) ; (13) in, Is with The set of substructures with the same predicted labels, on the other hand, Is with A collection of substructures with different labels. Obviously, for a given molecular graph , the expected selected edge belong Therefore, the following rewards were designed: for The contribution to the explanation of the molecular activity is evaluated as: ; (14) in, For the nodes, For the nodes, is the first substructure; Subgraph With edge The effectiveness of the cooperation between the two: and subgraph Molecular Diagram Evaluation of the combined contribution of GNN model predictions as follows: ; (15) in, represents a model for predicting molecular properties; Ensure the edge Makes a unique contribution to the explanation, make sure and the previously selected explanation subgraph The combination of also has a positive impact on explanation. The final reward function is: (16).

[0024] S3. Iteratively train the interpreter through the designed loss function until the loss function meets the set loss value, completes the training and learning, and obtains the trained interpreter; In the task of molecular property interpretation, in order to improve the interpretability and stability of the model, it is usually necessary to design a special loss function to optimize the interpreter so that it can effectively identify substructures that play a key role in the molecular properties. for: ; (17) in, is a hyperparameter used to ensure that the input to the logarithmic function is always greater than 0; During the training process, the loss function is first optimized using stochastic gradient descent or Adam optimizer, so that the interpreter can learn the optimal substructure selection strategy; the training data consists of molecular graphs and their corresponding property labels. The model selects the most representative substructure based on the current molecular structure at each iteration, and calculates the loss function to update the parameters; to improve the generalization and interpretability of the model, the hyperparameters are fine-tuned, and grid search or Bayesian optimization is used to find the optimal parameters; to prevent the model from overfitting, Dropout, weight decay or data enhancement technology are introduced to improve the stability and explanatory power of the model. Ultimately, the optimized explanatory model can automatically identify key substructures that affect molecular properties, and provide stable and reliable molecular property explanations, providing strong support for molecular design, drug discovery, and materials science.

[0025] At the same time, we split the molecular data set, using 80% for training and 20% for testing. The model is trained using the above method to achieve the interpretation of molecular properties.

[0026] S4. Embed the molecular nodes into the trained interpreter and output the substructures that play a key role in the molecular properties.

[0027] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A molecular property interpretation method based on deep learning, characterized in that: The following steps are involved: S1. Predict molecular properties based on the pre-trained GNN model to obtain molecular node embedding; S2. Design an interpreter to update the molecular node embedding based on the graph transformer to obtain node features, use the hierarchical edge selector to determine the edges in the molecular substructure, and design a reward function to restrict and guide the generation process of the substructure; S3. Iteratively train the interpreter through the designed loss function until the loss function meets the set loss value, completes the training and learning, and obtains the trained interpreter; S4. Embed the molecular nodes into the trained interpreter and output the substructures that play a key role in the molecular properties.

2. A molecular property interpretation method based on deep learning according to claim 1, characterized in that: In S1, the molecule is formally represented as an undirected graph ,in is a collection of nodes, corresponding to atoms in the molecule, is the set of edges, corresponding to the chemical bonds in the molecule; The initial feature representation of the node is ,in, Indicates The feature vector of a node, is the number of nodes, is the dimension of each node feature; after the initial features are encoded by the GNN model, a node embedding representation is generated , is the dimension of node embedding.

3. The molecular property interpretation method based on deep learning according to claim 1, characterized in that: In S2, the query matrix is ​​calculated in the self-attention mechanism of the graph Transformer , key matrix Sum Matrix , the expression is: ；(1) in, , , is a learnable weight matrix, are the dimensions of the key matrix, query matrix, and value matrix, , is the number of attention heads; For each attention head, use the query and key Calculating attention weights , the expression is: ；(2) Using attention weights Pair Matrix A weighted sum is performed to generate the output of the attention head, expressed as: ；(3) In the multi-head attention mechanism, the outputs of all attention heads are concatenated together, expressed as: ；(4) in, is the concatenated multi-head attention output, Concatenate operation: It means concatenating the outputs of all attention heads together. For the The output of an attention head is is the number of attention heads; right Perform linear transformation and add residual connection and activation function to obtain updated node features , the expression is: ；(5) in, is a nonlinear activation function, is the trainable parameter matrix of the linear transformation.

4. The molecular property interpretation method based on deep learning according to claim 3, characterized in that: In S2, the substructures used to explain the molecular properties predicted by the model are determined by maximizing the mutual information metric , the expression is: ；(6) in, is the mutual information, which is used to measure the substructure and The degree of information sharing between is to have Edge graph The collection of subgraphs of It is a picture The prediction results in the predictive property model GNN, =0 means the molecule is inactive, =1 means the molecule is active; According to the knowledge of mutual information, the above formula becomes: ；(7) in, for The Strip edge, Indicates that given the predicted molecular properties In the case of Probability distribution of occurrence; The generation of substructure is regarded as a Markov chain process, and the substructure is obtained using the following formula Edge: ；(8) in, is the first The best choice of edge, For the Any edge in the candidate edge set, is the conditional entropy, which means that given and In the case of uncertainty, Indicates The substructure after the step, Indicates adding to the current interpretation subgraph The set of candidate edges; Design a hierarchical edge selector to find the first The edge, in The probability of each candidate edge is calculated at each step, and the edge with the highest score is selected. , the first MLP layer edge represents the generator Perform the following processing: ；(9) in, , Respectively represent Nodes and Nodes in The characteristics of and , Represents edge The edge representation of The second MLP layer edge possibility generator The possibility of generating an edge, that is, the probability score of the edge, is expressed as: ；(10) in, is a connection operation, is the representation of the molecular graph obtained by the prediction model GNN, It is the edge The edge with the highest score will be added to the substructure; the edge that has been selected into the substructure will not enter the candidate edge set again; repeat until the termination condition is met until.

5. The molecular property interpretation method based on deep learning according to claim 4, characterized in that: In S2, the reward function for: ；(16) in is a hyperparameter used to balance the two rewards; for and substructure Molecular Diagram The combined contribution evaluation of the GNN model prediction is expressed as: ；(15) in, represents a model for predicting molecular properties; for The contribution to the explanation of the molecular activity is evaluated as: ；(14) in, For the nodes, For the nodes, is the first substructure, Is with A collection of substructures with the same predicted label, Is with A collection of substructures with different tags.

6. The molecular property interpretation method based on deep learning according to claim 5, characterized in that: In S3, the loss function designed for: ；(17) in, is a hyperparameter used to ensure that the input to the logarithmic function is always greater than 0; During the training process, the loss function is first optimized using stochastic gradient descent or Adam optimizer so that the interpreter can learn the optimal substructure selection strategy; the training data consists of molecular graphs and their corresponding property labels. At each iteration, the model selects the most representative substructure based on the current molecular structure and calculates the loss function to update the parameters; to improve the generalization and interpretability of the model, the hyperparameters are fine-tuned, and grid search or Bayesian optimization is used to find the optimal parameters; to prevent the model from overfitting, Dropout, weight decay or data augmentation technology are introduced to improve the stability and explanatory power of the model.

Images