Small sample self-interpretable graph learning method based on meta learning

Abstract

The invention relates to the technical field of interpretable artificial intelligence, and discloses a small sample self-interpretable graph learning method based on meta learning, which comprises the steps of sampling a molecular graph data set of a plurality of small sample chemical molecule attribute prediction tasks as a training data set in a graph data set of a basic graph classification task; constructing a self-interpretable graph learning model; fast parameters and slow parameters of the self-interpretable graph learning model are updated on the chemical molecule attribute prediction task of each small sample; and for a new small sample classification task, fast parameters of the self-interpretable graph learning model are updated on a support set of the new small sample classification task, and graph data without labels are subjected to classification prediction and interpretation is generated. According to the method, a graph self-interpretable model composed of an interpreter and a predictor is designed, task information is introduced, so that the human decision process is imitated, and the quality of interpretation generated by the self-interpretable graph learning model and the accuracy of a prediction result in a small sample scene are improved by utilizing a meta-learning method.

Description

Technical Field

[0001] The present invention relates to the field of explainable artificial intelligence technology, and in particular to a small sample self-interpretable graph learning method based on meta-learning. Background Art

[0002] Interpretable graph learning methods aim to provide the ability to explain and understand the prediction and decision-making processes in graph-structured data, thereby providing users or decision makers with an explainable decision process.

[0003] Currently, graph neural networks are a mainstream method in graph learning. In traditional graph neural networks, models perform prediction tasks by learning the representation of nodes and edges, but these models are usually black-box and it is difficult to explain the basis of their predictions, which limits their application in specific fields, such as molecular graph attribute prediction tasks in the field of biochemistry.

[0004] The usual form of interpretable graph learning is to select the subgraph part of the input graph that can best determine the model's prediction results as the explanation of the input graph. Specifically, the explanation is in the form of the weight of each node or edge in the graph. The larger the weight, the more important the node or edge is, that is, the more it can determine the model's prediction results.

[0005] At present, interpretable graph learning is mainly divided into two types: post-explanation and self-explanatory methods:

[0006] Among them, post-hoc explanation aims to explain the predictions made by the trained graph neural network model for the input data. This method usually requires another model to select a part of the input graph as the explanation of the prediction. However, due to the non-convexity and high complexity of graph neural networks, post-hoc explanation methods are usually difficult to explain the true internal reasoning process of the model. In addition, post-hoc explanation methods require multiple optimizations for each input sample to obtain the generated explanation, which has a high time complexity.

[0007] A self-interpretable graph neural network is usually a specially designed graph neural network that can simultaneously output prediction results and corresponding explanations for the input graph data. The advantage of this self-interpretable graph learning is that it can generate more accurate explanations and can output predictions and explanations simultaneously during the inference process without the need for additional optimization. However, it is difficult to achieve satisfactory results in small sample scenarios with only a small amount of labeled data. Summary of the invention

[0008] In order to solve the above technical problems, the present invention provides a small sample self-interpretable graph learning method based on meta-learning. A self-interpretable graph learning model for small samples is obtained through a meta-learning training method, so as to improve the accuracy of model prediction in small sample scenarios and generate high-quality explanations at the same time.

[0009] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0010] A small sample self-interpretable graph learning method based on meta-learning is provided. A self-interpretable molecular graph prediction model for a small sample scenario is obtained through a meta-learning training method. The molecular properties of the molecular graph data in the chemical molecular property prediction task under the small sample scenario are predicted and the corresponding explanation is generated through the trained self-interpretable molecular graph prediction model, that is, the degree of dependence of the self-interpretable molecular graph prediction model on each atom in the molecular graph when making a prediction is generated. Specifically, the following steps are included:

[0011] Step A, in the graph dataset of the basic graph classification task, a plurality of small sample molecular graph datasets of the chemical molecule property prediction task are sampled as training datasets, and each molecular graph dataset of the chemical molecule property prediction task is divided into a support set and a query set;

[0012] Step B, constructing a self-interpretable molecular graph prediction model: the self-interpretable molecular graph prediction model includes an interpreter and a predictor; vector representations of different attribute categories are obtained through a graph neural network encoder and input into the interpreter and the predictor as task information, thereby helping the self-interpretable molecular graph prediction model to predict molecular attributes and generate explanations;

[0013] Step C, updating the initial value and slow parameter of the fast parameter of the self-interpretable molecular graph prediction model on each small sample chemical molecular property prediction task, specifically includes the following steps:

[0014] Step C1: Calculate the total loss function including the contrast loss of data enhancement on the support set of each small sample chemical molecule property prediction task, and update the fast parameters of the self-interpretable molecular graph prediction model;

[0015] Step C2: Evaluate on the query set in each small sample chemical molecule property prediction task, and update the initial values ​​of the slow parameters and fast parameters of the self-interpretable molecular graph prediction model according to the loss function on the query set;

[0016] Step D: For new small sample chemical molecule property prediction tasks, the fast parameters of the self-interpretable molecular graph prediction model are updated on the corresponding support set, and the unlabeled molecular graph data are classified and predicted and explanations are generated.

[0017] Furthermore, in step B, the vector representations of different attribute categories are obtained through the graph neural network encoder and input into the interpreter and predictor as task information, thereby helping the self-interpretable molecular graph prediction model to predict molecular attributes and generate explanations, specifically including:

[0018] Each molecular graph data is passed through an encoder f based on a graph neural network, and for each molecular graph, a matrix consisting of a high-dimensional representation of each atom containing neighboring atom information is obtained:

[0019]

[0020] H i The node representation matrix representing the i-th molecule, h (v) The vector representation of the vth atom in the molecular graph, V i Represents the node set of the i-th molecule data, G i represents the i-th molecular graph, and d is the dimension represented by the vector;

[0021] Borrowing the concept of prototype vector, use the aggregation function to obtain the overall vector representation containing each attribute category information:

[0022]

[0023] TI c represents the vector representation of the molecular attribute category c, f readout represents the aggregation function, y of the i-th molecule i Attribute tags;

[0024] Finally, for each chemical molecule property prediction task, the vector representations of each property category are concatenated together, and the final vector representation obtained is used as the task information;

[0025] Then, the node representation matrix H of each molecule is i The task information is simultaneously input into the interpreter composed of a graph neural network encoder and a multi-layer perceptron to obtain the weight of each atom, which indicates the degree of dependence on each atom when making predictions from the self-interpretable molecular graph prediction model:

[0026] m i =σ(MLP([H i ,TI]));

[0027] Where m i ∈[0,1] represents the node weight matrix of the i-th molecular graph, σ represents the normalization function, MLP represents the multi-layer perceptron, GNN represents the encoder based on the graph neural network, and TI represents the vector obtained by concatenating the vector representation of each attribute category in the chemical molecule attribute prediction task;

[0028] The molecular graph data is divided into explained and non-explained parts according to the weight of each node:

[0029]

[0030] in, It is the explanation part, that is, the self-interpretable molecular graph prediction model predicts the molecular properties of the input molecular graph data based on this part. The non-explained part, that is, the noise part, the intermediate variable

[0031] Through the aggregation function, we can get the vector representation of the explained part and the vector representation of the non-explained part respectively:

[0032]

[0033] in, is the vector representation of the explanation part, is the vector representation of the non-interpreted part;

[0034] The vector representation of the explanation part is input into the predictor part to obtain the final molecular property prediction results and generate explanations.

[0035] Furthermore, the step C1 specifically includes:

[0036] First, the molecular graph data in the support set of a chemical molecule property prediction task is forward propagated, and for the i-th molecular graph data, the predicted value based on the explanation part vector representation is obtained. Then, according to the category label y of the i-th molecular graph data i , and get the loss function of the attribute prediction part That is:

[0037]

[0038] Next, we add the vector representations of the interpreted part of each molecule data and the non-interpreted part of different molecules to obtain a total of 4K 2 Artificially enhanced vector representation, calculate the contrast loss function

[0039]

[0040] Among them, τ is the temperature hyperparameter, k represents the number of data of each category in the chemical molecule property prediction task, 1 i≠j It means that it is 1 only when i≠j. It means that only when y i =y j 1; In addition, in order to solve the problem of deviation in the size of generated explanations, a penalty loss function L is designed reg To control the size of generated explanations:

[0041]

[0042] Among them, m iRefers to the generated node weights; then each loss function is weighted summed to obtain the total loss function;

[0043] Based on the back propagation of the total loss function, the fast parameters of the self-interpretable molecular graph prediction model, that is, the parameters of the predictor, are updated according to the gradient.

[0044] Furthermore, the step C2 specifically includes:

[0045] Predict the molecular graph data on the query set in each chemical molecule property prediction task and calculate the loss value. Update the slow parameters of the self-interpretable molecular graph prediction model, i.e., the encoder and interpreter parameters, and the fast parameters, i.e., the initial value of the predictor, according to the loss value on the query set.

[0046] The forward propagation method for each molecular graph data in the query set is the same as step C1. To ensure that the label is not leaked, the task information used by the self-interpretable molecular graph prediction model on the query set is the attribute category vector on the corresponding support set. After obtaining the same loss value as step C1 and performing back propagation, the slow parameters are updated according to the gradient, and the initial values ​​of the fast parameters before being updated on the corresponding chemical molecule property prediction task.

[0047] Compared with the prior art, the beneficial technical effects of the present invention are:

[0048] The present invention designs a graph self-interpretable model consisting of two parts, an explainer and a predictor, and introduces task information to imitate the human decision-making process. It also uses the meta-learning method to add a data-enhanced contrast loss function to train the self-interpretable model, thereby greatly improving the quality of explanations generated by the self-interpretable graph learning model and the accuracy of prediction results in small sample scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0050] Figure 1 A flowchart of a self-interpretable graph learning method provided by an embodiment of the present invention;

[0051] Figure 2 A framework diagram of a self-interpretable graph learning model provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The small sample scenario in the present invention generally refers to a scenario in which the number of samples in each category is less than 10 in a classification task.

[0054] like Figure 1 As shown, the small sample self-interpretable graph learning method based on meta-learning in the present invention includes the following steps:

[0055] Step A: Generate a small sample classification dataset:

[0056] In the embodiment of the present invention, for the data set of the basic graph classification task, N categories are sampled each time, K molecular graph data samples are sampled for each category as the support set, and M molecular graph data samples are sampled as the query set to form a task. Then, through multiple samplings, a training set and a test set are formed respectively. There is no overlap in the categories of the training set and the test set.

[0057] Step B: Build a self-interpretable graph learning model:

[0058] like Figure 2 As shown, the self-interpretable graph learning model structure designed in the present invention for small sample scenarios is divided into two parts: an interpreter and a predictor. In addition, vector representations of different attribute categories are obtained through a graph neural network encoder to generate task information, which is input into the interpreter and predictor to help explain the generation and classification prediction tasks.

[0059] Specifically, each molecular graph data passes through the encoder f to obtain a matrix consisting of each node representation:

[0060]

[0061] Next, we borrow the concept of prototype vector and use the aggregation function to obtain the overall vector representation containing the category information of each attribute:

[0062]

[0063] Finally, for each chemical molecule property prediction task, the vector representations of each property category are concatenated together, and the final vector representation obtained is used as the task information.

[0064] Next, the representation matrix and task information are simultaneously input into the interpreter part consisting of the graph neural network encoder and the multi-layer perceptron to obtain the weight of each node:

[0065] m i =σ(MLP([GNN(G i ),TI])).

[0066] Where m i ∈[0,1], then the molecular graph data can be divided into explained part and non-explained part according to the weight of each node:

[0067]

[0068] in, This is the explanation part. Non-explanatory part, intermediate variable

[0069] Furthermore, through the aggregation function, we can get the vector representation of the explained part and the non-explained part respectively:

[0070]

[0071] in, is the vector representation of the explanation part, is the vector representation of the non-interpreted part.

[0072] The vector representation of the explanation part is input into the predictor part to obtain the final molecular property prediction results and generate explanations.

[0073] Step C: updating the initial value of the model's fast parameters and slow parameters for each small sample chemical molecule property prediction task.

[0074] Step C1: Update the fast parameters of the model on the support set in each small sample chemical molecule property prediction task:

[0075] The specific parameter update method is to first propagate the support set data of a task forward, and for the i-th molecular graph data, obtain the predicted value p(h i ), according to its category label y i The loss function of the classification prediction part is:

[0076]

[0077] Next, we add the vector representations of the interpreted part of each molecular graph data and the non-interpreted part of different molecular graph data to obtain a total of 4K 2 Artificially enhanced vector representation, calculate the contrast loss function

[0078]

[0079] Among them, τ is a temperature hyperparameter. In addition, in order to solve the problem of deviation in the size of generated explanations, we designed a penalty term to control the size of generated explanations:

[0080]

[0081] Among them, m i Refers to the generated node weight matrix. Finally, by designing the weight of each loss function according to the specific task, the final total loss function can be obtained.

[0082] Next, back propagate the total loss function and update the model fast parameters, i.e., the predictor parameters, according to the gradient.

[0083] Step C2: Evaluate on the query set in each training task, and update the slow parameters of the model, namely the encoder and interpreter parameters, and the initial values ​​of the fast parameters according to the loss function on the query set.

[0084] The forward propagation method for each data in the query set is the same as C1. To ensure that the label is not leaked, the task information used by the model on the query set is the task information on the corresponding support set. After obtaining the loss value and performing backpropagation, the slow parameters are updated according to the gradient and the initial value of the fast parameters before updating on the corresponding task.

[0085] Step D: For new small sample chemical molecule property prediction tasks, the fast parameters of the self-interpretable molecular graph prediction model are updated on the corresponding support set, and the unlabeled molecular graph data are classified and predicted and explanations are generated.

[0086] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0087] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A small sample self-interpretable graph learning method based on meta-learning, characterized in that: A self-interpretable molecular graph prediction model for small sample scenarios is obtained through a meta-learning training method. The trained self-interpretable molecular graph prediction model is used to predict the molecular properties of the molecular graph data in the chemical molecular property prediction task under small sample scenarios and generate corresponding explanations, that is, to generate the degree of dependence of the self-interpretable molecular graph prediction model on each atom in the molecular graph when making predictions. The specific steps include: Step A, in the graph dataset of the basic graph classification task, a plurality of small sample molecular graph datasets of the chemical molecule property prediction task are sampled as training datasets, and each molecular graph dataset of the chemical molecule property prediction task is divided into a support set and a query set; Step B, constructing a self-interpretable molecular graph prediction model: the self-interpretable molecular graph prediction model includes an interpreter and a predictor; vector representations of different attribute categories are obtained through a graph neural network encoder and input into the interpreter and the predictor as task information, thereby helping the self-interpretable molecular graph prediction model to predict molecular attributes and generate explanations; Step C, updating the initial value and slow parameter of the fast parameter of the self-interpretable molecular graph prediction model on each small sample chemical molecular property prediction task, specifically includes the following steps: Step C1: Calculate the total loss function including the contrast loss of data enhancement on the support set of each small sample chemical molecule property prediction task, and update the fast parameters of the self-interpretable molecular graph prediction model; Step C2: Evaluate on the query set in each small sample chemical molecule property prediction task, and update the initial values ​​of the slow parameters and fast parameters of the self-interpretable molecular graph prediction model according to the loss function on the query set; Step D: For new small sample chemical molecule property prediction tasks, the fast parameters of the self-interpretable molecular graph prediction model are updated on the corresponding support set, and the unlabeled molecular graph data are classified and predicted and explanations are generated.

2. The small sample self-interpretable graph learning method based on meta-learning according to claim 1, characterized in that: In step B, the vector representations of different attribute categories are obtained through the graph neural network encoder and input into the interpreter and predictor as task information, thereby helping the self-interpretable molecular graph prediction model to predict molecular attributes and generate explanations, specifically including: Each molecular graph data is passed through an encoder f based on a graph neural network, and for each molecular graph, a matrix consisting of a high-dimensional representation of each atom containing neighboring atom information is obtained: H i The node representation matrix representing the i-th molecule, h (v) The vector representation of the vth atom in the molecular graph, V i Represents the node set of the i-th molecule data, G i represents the i-th molecular graph, and d is the dimension represented by the vector; Borrowing the concept of prototype vector, use the aggregation function to obtain the overall vector representation containing each attribute category information: TI c represents the vector representation of the molecular attribute category c, f readout represents the aggregation function, y of the i-th molecule i Attribute tags; Finally, for each chemical molecule property prediction task, the vector representations of each property category are concatenated together, and the final vector representation obtained is used as the task information; Then, the node representation matrix H of each molecule is i The task information is simultaneously input into the interpreter composed of a graph neural network encoder and a multi-layer perceptron to obtain the weight of each atom, which indicates the degree of dependence on each atom when making predictions from the self-interpretable molecular graph prediction model: m i σ(MLP([H i ,THE])); Where m i ∈[0,1] represents the node weight matrix of the i-th molecular graph, σ represents the normalization function, MLP represents the multi-layer perceptron, GNN represents the encoder based on the graph neural network, and TI represents the vector obtained by concatenating the vector representation of each attribute category in the chemical molecule attribute prediction task; The molecular graph data is divided into explained and non-explained parts according to the weight of each node: in, It is the explanation part, that is, the self-interpretable molecular graph prediction model predicts the molecular properties of the input molecular graph data based on this part. The non-explained part, that is, the noise part, the intermediate variable Through the aggregation function, we can get the vector representation of the explained part and the vector representation of the non-explained part respectively: in, is the vector representation of the explanation part, is the vector representation of the non-interpreted part; The vector representation of the explanation part is input into the predictor part to obtain the final molecular property prediction results and generate explanations.

3. The small sample self-interpretable graph learning method based on meta-learning according to claim 1, characterized in that: The step C1 specifically comprises: First, the molecular graph data in the support set of a chemical molecule property prediction task is forward propagated, and for the i-th molecular graph data, the predicted value based on the explanation part vector representation is obtained Then, according to the category label y of the i-th molecular graph data i , and get the loss function of the attribute prediction part That is: Next, by adding the vector representations of the interpreted part of each molecule data and the non-interpreted part of different molecules, a total of 4K 2 Artificially enhanced vector representation, calculate the contrast loss function Among them, τ is the temperature hyperparameter, k represents the number of data of each category in the chemical molecule property prediction task, 1 i≠j It means that it is 1 only when i≠j. It means that only when y i =y j 1; In addition, in order to solve the problem of deviation in the size of generated explanations, a penalty loss function L is designed teg To control the size of generated explanations: Among them, m i Refers to the generated node weights; then each loss function is weighted summed to obtain the total loss function; Based on the back propagation of the total loss function, the fast parameters of the self-interpretable molecular graph prediction model, that is, the parameters of the predictor, are updated according to the gradient.

4. The small sample self-interpretable graph learning method based on meta-learning according to claim 1, characterized in that: The step C2 specifically includes: Predict the molecular graph data on the query set in each chemical molecule property prediction task and calculate the loss value. Update the slow parameters of the self-interpretable molecular graph prediction model, i.e., the encoder and interpreter parameters, and the fast parameters, i.e., the initial value of the predictor, according to the loss value on the query set. The forward propagation method for each molecular graph data in the query set is the same as step C1. To ensure that the label is not leaked, the task information used by the self-interpretable molecular graph prediction model on the query set is the attribute category vector on the corresponding support set. After obtaining the same loss value as step C1 and performing back propagation, the slow parameters are updated according to the gradient, and the initial values ​​of the fast parameters before being updated on the corresponding chemical molecule property prediction task.

Images