A molecular graph comparative learning method based on chemical element knowledge graph

Abstract

The present invention discloses a molecular graph contrast learning method based on a chemical element knowledge graph, comprising: constructing a chemical element knowledge graph according to all chemical properties of each chemical element in a periodic table of chemical elements; enhancing the molecular graph by using the chemical element knowledge graph to obtain a molecular enhanced graph; obtaining graph representations of the molecular graph and the molecular enhanced graph by using a pluggable representation model; selecting other molecular graphs similar to the molecular graph in a molecular fingerprint space as negative samples by using a hard negative sample mining technique; mapping the graph representations of positive sample pairs and negative sample pairs to the same space, constructing a contrast loss function by maximizing the consistency between positive sample pairs and minimizing the consistency between negative sample pairs, and performing optimization learning by using the contrast loss function; forming a prediction model by using a pluggable representation model determined by parameters and a nonlinear classifier, and predicting the molecular properties of the molecular graph by using the prediction model with fine-tuned parameters to improve the prediction accuracy of the molecular properties.

Description

Technical Field

[0001] The present invention relates to the field of graph contrast learning, and in particular to a molecular graph contrast learning method based on a chemical element knowledge graph. Background Art

[0002] In the past few years, graph representation learning (GRL) has become a hot research field for analyzing graph structured data. Graph representation learning aims to learn an encoding function that transforms graph data with complex structures into a dense representation in a low-dimensional space that preserves diverse graph attributes and structural features.

[0003] Traditional graph representation learning methods use random walks to convert graphs into node sequences to model the co-occurrence relationship between central nodes and neighboring nodes. However, this type of learning framework has two obvious disadvantages: first, there is a lack of parameter sharing between encoders, which will take up too many computing resources; second, the model lacks generalization ability and is difficult to apply to new graphs. In recent years, the use of graph neural networks for graph representation learning has received widespread attention. Graph neural networks usually update the hidden state of nodes by the weighted sum of domain states. Through information transfer between nodes, graph neural networks can capture information from their domain.

[0004] Molecular graphs are a type of graph data that naturally has rich structural information. Currently, many studies use deep learning methods to learn embedded representations of molecular graphs to accelerate drug development and molecular identification. Among them, supervised learning methods learn the representation of molecules in low-dimensional space by improving the message passing process and network architecture, and apply them to downstream property prediction tasks. However, the experimental process of obtaining molecular property labels is time-consuming and resource-consuming, and the task faces problems such as insufficient labeled data. At the same time, due to the extremely large molecular space, the model's generalization ability is poor, which limits its application in practical scenarios. Some work has designed complex pre-training tasks to learn the graph representation of molecules.

[0005] As a kind of pre-training model, graph contrastive learning uses graph augmentation methods to randomly expand the input graph to obtain two views and maximize the representation consistency of the two views, so as to learn the graph embedding representation in a self-supervised manner. This self-supervised learning method can mine supervisory information for the learning algorithm from the data itself. Although graph contrastive learning methods have made great progress, the design of graph augmentation methods is still rarely explored. Traditional graph augmentation methods include node deletion, edge perturbation, subgraph extraction, and attribute masking. However, this contrastive framework has two obvious disadvantages when applied to molecular graphs: first, removing or adding chemical bonds or groups will change the characteristics and chemical meaning of the molecular graph; second, the atoms in the molecular graph are modeled as individuals that can only be connected by chemical bonds, without considering the correlation of atoms at the microscopic level (the commonality between atoms with the same properties).

[0006] Therefore, it is urgent to design a new molecular graph comparative learning method to solve the above problems existing in the existing technology. Summary of the invention

[0007] As mentioned above, the purpose of the present invention is to provide a molecular graph comparative learning method based on chemical element knowledge graph, which improves the accuracy of molecular property prediction by guiding the graph enhancement method through the chemical element knowledge graph.

[0008] In order to achieve the above invention objectives, the present invention provides the following technical solutions:

[0009] A molecular graph comparative learning method based on chemical element knowledge graph includes the following steps:

[0010] Based on all the chemical properties of each chemical element obtained from the periodic table of chemical elements, a knowledge graph of chemical elements is constructed to establish the microscopic chemical connections between chemical elements;

[0011] Use the chemical element knowledge graph to enhance the molecular graph to obtain a molecular enhanced graph that contains both topological structure and chemical domain knowledge;

[0012] Obtain graph representations of molecular graphs and molecular enhanced graphs using pluggable representation models;

[0013] The hard negative sample mining technology is used to select other molecular graphs similar to the molecular graph in the molecular fingerprint space as negative samples. The molecular graph and its corresponding molecular enhancement graph form a positive sample pair, and the molecular graph and its corresponding negative sample and the molecular enhancement graph corresponding to the negative sample form a negative sample pair. The molecular enhancement graph corresponding to the molecular graph and the negative sample corresponding to the molecular graph and the molecular enhancement graph corresponding to the negative sample form a negative sample pair.

[0014] After mapping the graph representations of positive sample pairs and negative sample pairs to the same space, a contrast loss function is constructed by maximizing the consistency between positive sample pairs and minimizing the consistency between negative sample pairs. The contrast loss function is used to optimize the pluggable representation model parameters and the hyperparameters of the contrast loss function. After the optimization is completed, a pluggable representation model with determined parameters is obtained.

[0015] When applied, the pluggable representation model with determined parameters and the nonlinear classifier form a prediction model and the parameters of the prediction model are fine-tuned, and the prediction model with fine-tuned parameters is used to predict the molecular properties of the molecular graph.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] (1) Based on the periodic table of chemical elements, a chemical element knowledge graph was constructed for the first time. The constructed chemical element knowledge graph represents the microscopic chemical connections between chemical elements.

[0018] (2) Different from the existing graph enhancement methods that may destroy the meaning of molecular chemistry, the present invention uses the chemical element knowledge graph that characterizes the microscopic chemical connections between chemical elements to enhance the molecular graph. The molecular enhanced graph obtained by graph enhancement can not only retain the basic structural information of the molecular graph, but also capture the microscopic chemical connections between each atom.

[0019] (3) By utilizing a pluggable representation model to obtain graph representations of molecular graphs and molecular enhanced graphs and mapping the graph representations of positive sample pairs and negative sample pairs into the same space, different types of message passing are provided for different types of neighbor nodes to better learn the structural knowledge and chemical domain knowledge contained in the graph;

[0020] (4) Different from the existing random negative sampling method, the present invention adopts hard negative sample mining technology to find similar molecular graphs as negative samples in the molecular fingerprint space to learn more discriminative molecular graph representation.

[0021] (5) On the basis of constructing positive sample pairs and negative sample pairs, a contrast loss function is constructed by maximizing the consistency between positive sample pairs and minimizing the consistency between negative sample pairs, and then optimizing the pluggable representation model. Such an optimized pluggable representation model can well learn the structural information of molecular graphs and the microscopic chemical connections between chemical elements to obtain accurate graph representation. Based on the accurate graph representation, the prediction accuracy of the molecular properties of molecular graphs can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 It is a flow chart of a molecular graph comparative learning method based on a chemical element knowledge graph provided by an embodiment of the present invention;

[0024] Figure 2 It is a schematic diagram of the principle of enhancing a molecular graph using a chemical element knowledge graph provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0026] Figure 1 : is a flow chart of a molecular graph comparative learning method based on a chemical element knowledge graph provided by an embodiment of the present invention. Figure 1 As shown, the molecular graph comparison learning method based on the chemical element knowledge graph provided in the embodiment includes the following steps:

[0027] Step 1: Based on all the chemical properties of each chemical element obtained from the periodic table of chemical elements, a chemical element knowledge graph is constructed to establish the microscopic chemical connections between chemical elements.

[0028] The periodic table of chemical elements records all the chemical properties of each chemical element, among which the chemical properties include metallicity, periodicity, family, state, atomic weight, electronegativity, melting point, boiling point, electron affinity, ionization energy, radius, hardness, modulus, density, conductivity, thermal energy, and abundance. Based on these chemical properties, a chemical element knowledge graph can be constructed. In the constructed chemical element knowledge graph, each triple is (attribute, relationship, element), that is, the head entity is the attribute, the tail entity is the chemical element, and the relationship is the relationship between the chemical element and the attribute. For example, the triple (Halogens, isFamilyOf, Cl) and the triple (Solid, isFamilyOf, Na) respectively indicate that the chemical element Cl belongs to the halogen family and the chemical element Na belongs to solid. Since the constructed chemical element knowledge graph contains chemical properties, it characterizes the microscopic chemical connection between chemical elements. Table 1 is the statistical result of the chemical element knowledge graph:

[0029] Table 1

[0030]

[0031] Step 2: Use the chemical element knowledge graph to enhance the molecular graph to obtain a molecular enhanced graph that contains both topological structure and chemical domain knowledge.

[0032] Figure 2 FIG. 1 is a schematic diagram of the principle of enhancing a molecular graph using a chemical element knowledge graph according to an embodiment of the present invention. Figure 2 As shown in the figure, the process of enhancing the molecular graph using the chemical element knowledge graph includes:

[0033] For each node in the molecular graph, the node represents an atom or element. In the chemical element knowledge graph, a triple with the node as the tail entity is found, and the head entity of the triple is used as a new node. The relationship of the triple is used as the edge between the node and the new node. The molecular graph is topologically enhanced and enhanced to obtain the molecular enhanced graph corresponding to the molecular graph. The obtained molecular enhanced graph contains not only the topological structure knowledge in the molecular graph, but also the basic chemical field knowledge, which can capture the microscopic connection between atoms that are not connected by chemical bonds.

[0034] Step 3: Obtain graph representations of molecular graphs and molecular enhanced graphs using a pluggable representation model.

[0035] In an embodiment, the pluggable representation model is a graph encoding model built based on a graph neural network, including a graph convolutional network (GCN), a graph attention network (GAT), a message passing neural network (MPNN), and a communicative message passing neural network (CMPNN). Preferably, MPNN can be selected to obtain a graph representation of a molecular graph and its corresponding molecular enhanced graph.

[0036] Step 4: Use hard negative sample mining technology to select other molecular graphs that are similar to the molecular graph in the molecular fingerprint space as negative samples, and construct positive sample pairs and negative sample pairs.

[0037] In the embodiment, the process of using the hard negative sample mining technology to select other molecular graphs similar to the molecular graph in the molecular fingerprint space as negative samples includes:

[0038] The molecular graph is converted into a molecular fingerprint, and the Tanimoto coefficient is used to calculate the similarity between two molecular fingerprints. When the similarity is greater than the set threshold, the two molecular graphs corresponding to the two molecular fingerprints are considered to be negative samples of each other, that is, when one molecular graph is a positive sample, the other molecular graph is a negative sample.

[0039] In one possible implementation, the chemical informatics tool Rdkit is used to convert molecules into molecular fingerprints; the molecular fingerprint is a Morgan fingerprint, wherein the Morgan fingerprint is formed by setting a radius starting from a specific atom and counting the number of partial molecular structures within the radius. The molecular fingerprint is represented by a vector composed of 0 and 1. For the same type of molecular fingerprint, each molecular graph corresponds to one molecular fingerprint, and the vector dimension is the same. Based on this, the formula for calculating the similarity between two molecular fingerprints using the Tanimoto coefficient is:

[0040]

[0041] Among them, e1 and e2 represent two molecular fingerprints, s(e1, e2) represents the similarity, N1 and N2 represent the number of 1s in the molecular fingerprint e1 and the molecular fingerprint e2 respectively, and N 12 It indicates the number of 1s that appear simultaneously in the same position in the molecular fingerprint e1 and the molecular fingerprint e2.

[0042] In the embodiment, the similarity threshold is set according to actual needs, and the similarity threshold can be set to 0.7. When the obtained similarity s(e1, e2) is greater than the similarity threshold, the two molecular graphs corresponding to the two molecular fingerprints are negative samples of each other. Negative samples are a relative concept relative to positive samples. If the molecular graph corresponding to the molecular fingerprint e1 is a positive sample, the molecular graph corresponding to the molecular fingerprint e2 is a negative sample.

[0043] After the negative samples are determined, the positive sample pairs and negative sample pairs can be constructed. First, a molecular graph is selected as the positive sample. The molecular graph and its corresponding molecular enhancement graph form a positive sample pair. The molecular graph and its corresponding negative sample and the molecular enhancement graph corresponding to the negative sample form a negative sample pair. The molecular enhancement graph corresponding to the molecular graph and the negative sample corresponding to the molecular graph and the molecular enhancement graph corresponding to the negative sample form a negative sample pair.

[0044] Step 5: After mapping the graph representations of the positive sample pairs and the negative sample pairs to the same space, a contrast loss function is constructed by maximizing the consistency between the positive sample pairs and minimizing the consistency between the negative sample pairs, and the contrast loss function is used for optimization learning.

[0045] In the embodiment, a two-layer multilayer perceptron (MLP) is used as a mapping head, and the mapping head is used to map the graph representations of the positive sample pairs and the negative sample pairs to the same space.

[0046] For a training batch consisting of N molecular graphs with similar molecular fingerprints, the molecular graphs in the training batch and their corresponding molecular enhanced graphs are positive sample pairs, and the other molecular graphs in the training batch and the molecular enhanced graphs corresponding to these molecular graphs are regarded as negative samples of the molecular graphs, forming negative sample pairs. The constructed contrast loss function is:

[0047]

[0048] Among them, z i and z′ i denote the graph representation of the i-th molecular graph and the graph representation of the molecular enhancement graph that constitute the i-th positive sample pair, respectively, z′ j represents the graph representation of the jth negative sample that constitutes a negative sample pair with the i-th molecular graph or the graph representation of the molecular enhancement graph corresponding to the negative sample, τ represents the temperature parameter, that is, the parameter to be optimized for learning, sim(·) is the cosine similarity, and the total contrast loss of the same training batch is the contrast loss function l for all positive sample pairs i sum.

[0049] The total contrast loss is used to optimize the pluggable representation model parameters and the hyperparameter τ of the contrast loss function. After the optimization, a pluggable representation model with determined parameters is obtained. This pluggable representation model can well learn the structural information of molecular graphs and the microscopic chemical connections between chemical elements to obtain accurate graph representation.

[0050] Step 6, when applied, the pluggable representation model with determined parameters and the nonlinear classifier are combined to form a prediction model and the parameters of the prediction model are fine-tuned, and the prediction model with fine-tuned parameters is used to predict the molecular properties of the molecular graph.

[0051] In the embodiment, the nonlinear classifier adopts MLP. After the MLP and the pluggable representation model with determined parameters form a prediction model, the prediction model is fine-tuned using a small sample molecular graph of a downstream prediction task, and the molecular properties of the molecular graph are predicted using the prediction model with fine-tuned parameters.

[0052] In the embodiment, the prediction task is mainly carried out on six classification data sets, including judging whether a molecule is permeable in the blood-brain barrier permeability data set (BBBP), judging whether a molecule is toxic in the compound toxicity data set (Tox21), the large compound toxicology data set based on in vitro high-throughput screening (ToxCast) and the clinical trial data set (ClinTox), judging whether a drug has side effects in the database of marketed drugs and adverse drug reactions (SIDER), and judging whether a molecule is inhibitory in the human secretase BACE-1 inhibitor data set (BACE). The corresponding molecular properties are permeability, toxicity, side effects and inhibition.

[0053] In the molecular graph comparative learning method based on the chemical element knowledge graph provided in the above embodiment, through steps 1 to 5, a chemical element knowledge graph is constructed for the first time based on the periodic table of chemical elements. The constructed chemical element knowledge graph represents the microscopic chemical connections between chemical elements. On this basis, the molecular graph is enhanced using the chemical element knowledge graph. The molecular enhanced graph obtained in this way can not only retain the basic structural information of the molecular graph, but also capture the microscopic chemical connections between each atom. Furthermore, by enhancing the molecular graph using the chemical element knowledge graph that represents the microscopic chemical connections between chemical elements, the molecular enhanced graph obtained by graph enhancement can not only retain the basic structural information of the molecular graph, but also capture the microscopic chemical connections between each atom. Furthermore, hard negative sample mining technology is used to find similar molecular graphs as negative samples in the molecular fingerprint space to learn more discriminative molecular graph representations. On the basis of constructing positive and negative sample pairs using molecular graphs with large sample sizes, a contrast loss function is constructed by maximizing the consistency between positive sample pairs and minimizing the consistency between negative sample pairs, and then optimizing the pluggable representation model. Such an optimized pluggable representation model can well learn the structural information of the molecular graph and the microscopic chemical connection between chemical elements to obtain an accurate graph representation. Using step 6, when applying specific downstream tasks, a prediction model is composed of a trained pluggable representation model and a nonlinear classifier, and the prediction model is re-tuned using a molecular graph with a small sample size, so that the prediction accuracy of the molecular properties of the molecular graph can be improved based on an accurate graph representation.

[0054] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A molecular graph comparative learning method based on chemical element knowledge graph, characterized in that: The following steps are involved: Based on all the chemical properties of each chemical element obtained from the periodic table of chemical elements, a knowledge graph of chemical elements is constructed to establish the microscopic chemical connections between chemical elements; The molecular graph is enhanced by using the chemical element knowledge graph to obtain a molecular enhanced graph that contains both topological structure and chemical domain knowledge. The process of enhancing the molecular graph by using the chemical element knowledge graph includes: for each node in the molecular graph, the node represents an atom or an element, a triple with the node as the tail entity is found in the chemical element knowledge graph, and the head entity of the triple is used as a new node, and the relationship of the triple is used as the edge between the node and the new node, and the molecular graph is topologically enhanced to obtain a molecular enhanced graph corresponding to the molecular graph; Obtain graph representations of molecular graphs and molecular enhanced graphs using pluggable representation models; The hard negative sample mining technology is used to select other molecular graphs similar to the molecular graph in the molecular fingerprint space as negative samples, which specifically includes: converting the molecular graph into a molecular fingerprint, using the Tanimoto coefficient to calculate the similarity between two molecular fingerprints, and when the similarity is greater than the set threshold, the two molecular graphs corresponding to the two molecular fingerprints are considered to be negative samples of each other, that is, when one molecular graph is a positive sample, the other molecular graph is a negative sample; The molecular graph and its corresponding molecular enhancement graph constitute a positive sample pair, the molecular graph and its corresponding negative sample and the molecular enhancement graph corresponding to the negative sample constitute a negative sample pair, and the molecular enhancement graph corresponding to the molecular graph and the negative sample corresponding to the molecular graph and the molecular enhancement graph corresponding to the negative sample constitute a negative sample pair; After mapping the graph representations of positive sample pairs and negative sample pairs to the same space, a contrast loss function is constructed by maximizing the consistency between positive sample pairs and minimizing the consistency between negative sample pairs. The contrast loss function is used to optimize the pluggable representation model parameters and the hyperparameters of the contrast loss function. After the optimization is completed, a pluggable representation model with determined parameters is obtained. When applied, the pluggable representation model with determined parameters and the nonlinear classifier form a prediction model and the parameters of the prediction model are fine-tuned, and the prediction model with fine-tuned parameters is used to predict the molecular properties of the molecular graph.

2. The molecular graph comparative learning method based on chemical element knowledge graph according to claim 1, characterized in that: The chemical properties include metallicity, periodicity, family, state, atomic weight, electronegativity, melting point, boiling point, electron affinity, ionization energy, radius, hardness, modulus, density, conductivity, thermal energy, abundance; In the constructed chemical element knowledge graph, each triple is (attribute, relationship, element), that is, the head entity is the attribute, the tail entity is the chemical element, and the relationship is the relationship between the chemical element and the attribute.

3. The molecular graph comparative learning method based on chemical element knowledge graph according to claim 1, characterized in that: The pluggable representation model is a graph coding model built based on graph neural network, including GCN, GAT, MPNN, and CMPNN.

4. The molecular graph comparative learning method based on chemical element knowledge graph according to claim 1, characterized in that: The cheminformatics tool Rdkit was used to convert the molecules into molecular fingerprints; the molecular fingerprints were Morgan fingerprints.

5. The molecular graph comparative learning method based on chemical element knowledge graph according to claim 1, characterized in that: A two-layer MLP is used as the mapping head, which is used to map the graph representations of positive sample pairs and negative sample pairs into the same space.

6. The molecular graph comparative learning method based on chemical element knowledge graph according to claim 1, characterized in that: The constructed contrast loss function is: Among them, z i and z i ′ denote the graph representation of the i-th molecular graph and the graph representation of the molecular enhancement graph that constitute the i-th positive sample pair, respectively, j ′ represents the graph representation of the jth negative sample that constitutes a negative sample pair with the i-th molecular graph or the graph representation of the molecular enhanced graph corresponding to the negative sample, τ represents the temperature parameter, and sim(·) is the cosine similarity.

7. The molecular graph comparative learning method based on chemical element knowledge graph according to claim 1, characterized in that: The nonlinear classifier adopts MLP. After the MLP and the pluggable representation model with determined parameters form a prediction model, the prediction model is fine-tuned using a small sample molecular graph of the prediction task, and the molecular properties of the molecular graph are predicted using the prediction model with fine-tuned parameters.

8. The molecular graph comparative learning method based on chemical element knowledge graph according to claim 7 is characterized in that: The prediction tasks include judging whether a molecule is permeable in a blood-brain barrier permeability dataset, judging whether a molecule is toxic in a compound toxicity dataset, a large compound toxicology dataset based on in vitro high-throughput screening, and a clinical trial dataset, judging whether a drug has side effects in a database of marketed drugs and adverse drug reactions, and judging whether a molecule is inhibitory in a human secretase BACE-1 inhibitor dataset. The corresponding molecular properties are permeability, toxicity, side effects, and inhibition.

Images