A meta-learning molecular property prediction method based on sample-task relationship graph

Abstract

The present invention discloses a meta-learning molecular property prediction method based on a sample-task relationship graph. This constructed sample-task relationship graph explicitly establishes the relationship between sample and task properties. This sample-task relationship graph is the first method to utilize the graph structure between samples and tasks to guide meta-learning molecular property prediction. A designed auxiliary task selector can select auxiliary tasks for different known target tasks, thereby better utilizing the auxiliary tasks to enhance the prediction model's ability to predict the target task. The designed target task selector can select sample target tasks within the same batch during prediction model training as a negative sample mining technique using a contrastive loss function. This contrastive loss function for sample target tasks allows the prediction model to better distinguish between different sample target tasks while maximizing consistency within the same sample target task.

Description

Technical Field

[0001] The present invention belongs to the field of molecular property prediction, and specifically relates to a meta-learning molecular property prediction method based on a sample-task relationship graph. Background Art

[0002] Deep learning methods require large amounts of labeled data for training. However, labeling data is expensive, especially for molecular property prediction tasks, where molecular properties must be experimentally determined in the laboratory. This process is both time-consuming and costly. The scarcity of labeled samples and the difficulty of labeling data limit the practical application of deep learning. Therefore, achieving good performance for deep learning models with a small number of labeled samples is a crucial issue. Meta-learning methods were proposed to address this issue of small, scarce labeled data.

[0003] The problem of predicting molecular properties from small samples differs from the widely studied problem of small sample image classification. Typically, in image classification, an image has only one label. However, for molecules, the same molecule can simultaneously possess different properties, meaning multiple labels, such as the degree of binding to different proteins, absorbability, toxicity, solubility, and so on. Some molecular properties are easy to measure in the laboratory, while others are more expensive to measure in the laboratory. Therefore, these easily measurable molecular properties can be used to infer more difficult-to-measure molecular properties. Traditional small sample learning methods have achieved good results for image classification tasks, but they have significant drawbacks when applied to small sample molecular property prediction: they cannot consider other properties of the molecule when predicting the target property.

[0004] Patent application publication number CN112164428A discloses a method for predicting the properties of small drug molecules based on deep learning. This method uses supervised learning based on constructed training samples to construct a model for the properties of small drug molecules. This process requires more labeled sample data, but in actual applications, the labels of sample data are difficult to measure, so this method is not applicable.

[0005] Patent application publication number CN113990408 A discloses a molecular graph contrastive learning method based on a chemical element knowledge graph. This method constructs a molecular representation through contrastive learning of constructed positive and negative sample pairs. However, molecular property labels are still required to optimize a nonlinear classifier. Molecular property prediction can be achieved based on this molecular representation and the optimized nonlinear classifier. However, in practical applications, sample data labels are difficult to measure, making this method unsuitable. Summary of the Invention

[0006] In view of the above, the present invention provides a meta-learning molecular property prediction method based on a sample-task relationship graph. By constructing a sample-task relationship graph, other known properties of the molecule are taken into consideration when predicting the target properties of the molecule, thereby improving the accuracy of meta-learning molecular property prediction.

[0007] To achieve the above object of the invention, the present invention provides a meta-learning molecular property prediction method based on a sample-task relationship graph, comprising the following steps:

[0008] Step 1: Create a sample-task relationship graph based on the molecular samples and the corresponding tasks, where both the molecular samples and the tasks are used as nodes in the graph, and the task labels corresponding to the molecular samples and tasks are used as attributes of the edges between the nodes;

[0009] Step 2: Build a prediction model that includes a graph representation model and a classifier. Use the graph representation model to extract the node representation of the molecular sample and the node representation of the task in the input sample-task relationship graph. Use the classifier to predict the task label based on the concatenation of the node representation of the molecular sample and the node representation of the task to output the label prediction probability.

[0010] Step 3: Build an auxiliary task selector based on LSTM, and use the auxiliary task selector to sample auxiliary tasks for the known target task.

[0011] Step 4: Build a target task selector based on LSTM, use auxiliary tasks and adopt the target task selector to select the sampled target tasks in the same batch;

[0012] Step 5: Based on the same sampling target task, additional molecular samples are sampled simultaneously to construct two sets of sample-task relationship graphs to form positive sample pairs. Based on different sampling target tasks in the same batch, additional molecular samples are sampled to construct two different sets of sample-task relationship graphs to form negative sample pairs. The contrast loss is constructed by maximizing the consistency between the positive sample pairs and the consistency between the negative sample pairs.

[0013] Step 6: Optimize the parameters of the auxiliary task selector, target task selector, and prediction model based on the contrast loss and the classification loss constructed according to the label prediction probability;

[0014] In step 7, the molecular properties are predicted using the parameter-optimized auxiliary task selector and prediction model.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] The relationship between sample and task attributes is explicitly established based on a constructed sample-task relationship graph. This sample-task relationship graph is the first method to utilize the graph structure between samples and tasks to guide meta-learning of molecular property prediction. The designed auxiliary task selector can select auxiliary tasks for different known target tasks, thereby better utilizing the auxiliary tasks to enhance the prediction model's ability to predict the target task. The designed target task selector selects sampled target tasks within the same batch during prediction model training as a negative sample mining technique using a contrastive loss function. This contrastive loss function for sampled target tasks allows the prediction model to better distinguish between different sampled target tasks while maximizing consistency within the same sampled target task. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 Schematic diagram of the process of a meta-learning molecular property prediction method based on a sample-task relationship graph provided in an embodiment of the present invention.

[0019] Figure 2 A schematic diagram of constructing a sample-task relationship diagram provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0021] Figure 1 Schematic diagram of the process of the meta-learning molecular property prediction method based on the sample-task relationship graph provided by the embodiment of the present invention. Figure 1 As shown, the embodiment provides a meta-learning molecular property prediction method based on a sample-task relationship graph, comprising the following steps:

[0022] Step 1: Establish a sample-task relationship diagram based on molecular samples and corresponding tasks.

[0023] In an embodiment, when constructing a sample-task relationship graph, both molecular samples and tasks are used as nodes in the graph, and the task labels corresponding to the molecular samples and tasks are used as attributes of the edges between the nodes, so that the molecular samples and different tasks are connected using the topological structure of the graph.

[0024] In the embodiment, molecular samples refer to various biological molecules or chemical molecules. These molecular samples correspond to various properties, including absorbability, toxicity, solubility, and degree of binding with proteins, etc. Each type of property corresponds to a type of task.

[0025] Step 2: Build a prediction model that includes a graph representation model and a classifier, and use the prediction model to predict the label probability of the molecular sample relative to the task in the input sample-task relationship graph.

[0026] In an embodiment, before the sample-task relationship graph is input into the graph representation model, it needs to undergo node initialization, wherein the node representation of the molecular sample is initialized to the primary vector of the molecular sample, and the initial vector is obtained by learning the representation of the molecular sample through the molecular representation model. The node representation of the task is initialized to a random vector, and then the graph representation model is used to extract the node representation of the molecular sample and the node representation of the task in the input sample-task relationship graph. Finally, the classifier is used to predict the task label on the splicing result of the node representation of the molecular sample and the node representation of the task to output the label prediction probability.

[0027] In the embodiments, both the graph representation model and the molecular representation model are pluggable graph encoding models built on graph neural networks, including GCN, GIN, GraphSAGE, MPNN, etc. The molecular representation model is preferably a graph encoding model built on GIN, and the graph representation model is preferably a graph encoding model built on GraphSAGE. In the embodiments, the classifier is preferably an MLP, that is, an MLP is used to predict the task label of the molecular sample relative to the task.

[0028] Step 3: Build an auxiliary task selector based on LSTM, and use the auxiliary task selector to sample auxiliary tasks for the known target task.

[0029] In an embodiment, an auxiliary task selector is constructed based on LSTM, and is used to adopt auxiliary tasks from a candidate auxiliary task set for a known target task, wherein the known target task is randomly sampled from a sample-task relationship graph, and the candidate auxiliary task set is composed of other tasks in the sample-task relationship graph except the known target task.

[0030] In an embodiment, an auxiliary task selector is used to sample auxiliary tasks of a known target task, including: for the auxiliary task selector, the input data includes two parts: (a1) a first representation similarity between the vector representation of the known target task and the vector representation of the candidate auxiliary task, preferably, the first representation similarity is obtained by Calculated, where are the vector representations of the known target task and the candidate auxiliary task, respectively. The vector representations are randomly initialized. sim1(·) represents the similarity calculation method, preferably the cosine similarity calculation method. (a2) The label similarity of the molecular sample on the known target task and the candidate auxiliary task. Preferably, the label similarity is calculated by dist(y t ,y a ) is calculated, where y t ,y a are the task label vectors of the molecular sample relative to the known target task and the candidate auxiliary task, respectively. dist(·) represents the calculation method of the distance between the task label vectors, preferably the Euclidean distance calculation method.

[0031] The above two parts of input data are input to the auxiliary task selector, and the sampling probability of the candidate auxiliary tasks is output after calculation. Auxiliary tasks are sampled from the candidate auxiliary tasks based on the sampling probability.

[0032] Step 4: Build a target task selector based on LSTM, use auxiliary tasks and adopt the target task selector to adopt the sampled target tasks within the same batch.

[0033] In an embodiment, the target task selector is constructed based on LSTM and is used to sample sampling target tasks within the same batch in the training phase, wherein the sampling target tasks come from auxiliary tasks, that is, the auxiliary tasks sampled by the auxiliary task selector serve as the candidate set of the next round of sampling target tasks.

[0034] In the embodiment, the auxiliary task is used and the target task selector is used to select the sampling target task in the same batch, including: for the target task selector, the input data includes three parts: (b1) the prediction loss value of the prediction model for the molecular sample on the known target task Preferably, the predicted loss value is the cross entropy between the predicted probability of the label of the molecular sample for the known target task and the true value of the task label. (b2) is the prediction loss value of the prediction model for the molecular sample on the auxiliary task Preferably, the predicted loss value is the cross entropy between the predicted probability of the label of the molecular sample for the auxiliary task and the true value of the task label. (b3) The second representation similarity between the vector representation of the known target task and the vector representation of the auxiliary task is preferably obtained by Calculated, where are the vector representations of the known target task and the auxiliary task respectively. The vector representation is the node representation output by the graph representation model. L represents the number of layers of the graph representation model. sim2(·) represents the similarity calculation method, preferably the cosine similarity calculation method.

[0035] The three input data parts are fed into the target task selector, which calculates and outputs the sampling probability of the sampled target task. Based on the sampling probability, the next auxiliary task is sampled from the auxiliary tasks as the sampled target task. This process is repeated multiple times to obtain multiple sampled target tasks within the same batch.

[0036] In step 5, positive and negative sample pairs are constructed based on the sampling target task, and the contrast loss is constructed by maximizing the consistency between positive sample pairs and the consistency between negative sample pairs.

[0037] In the embodiment, for all sampling target tasks in the same batch, two additional groups of molecular samples are sampled simultaneously based on the same sampling target task to construct two groups of sample-task relationship graphs to form positive sample pairs. Additional molecular samples are sampled based on different sampling target tasks in the same batch to construct two different groups of sample-task relationship graphs to form negative sample pairs. It should be noted that when constructing negative sample pairs, it is not limited whether the sampled additional molecular samples are the same. After obtaining the positive sample pairs and the negative sample pairs, the contrast loss is constructed by maximizing the consistency between the positive sample pairs and the consistency between the negative sample pairs. The table compares the loss Shown as:

[0038]

[0039] Where B is the batch size, They represent the node representations of the same sampling target task t on the two sets of sample-task relationship graphs in the positive sample pair, It represents the node representation of a set of sample-task relationship graphs for different sampling target tasks t′ in negative sample pairs, τ is the temperature parameter, sim() is the similarity calculation method, preferably cosine similarity.

[0040] In step 6, the parameters of the auxiliary task selector, target task selector, and prediction model are optimized based on the contrast loss and the classification loss constructed according to the label prediction probability.

[0041] In the embodiment, the classification loss constructed is the cross entropy loss of the label prediction probability and the task label true value, and the sum of the classification loss and the contrast loss is used to optimize the parameters of the prediction model using the gradient descent algorithm. Expressed as:

[0042]

[0043] Among them, y i,t represents the true value of the task label of the ith molecular sample for the tth task, represents the label prediction probability of the i-th molecular sample for the t-th task, Expressed as:

[0044]

[0045] Where σ is the Sigmoid activation function, MLP(·) represents the MLP classifier, They respectively represent the node representations of the output of the graph representation model for the i-th molecular sample and the t-th task.

[0046] In the embodiment, the auxiliary task selector and the target task selector adopt a reinforcement learning strategy, including:

[0047] For the auxiliary task selector, negative classification loss is used as a reward to optimize the model parameters of the auxiliary task selector, which is expressed as:

[0048]

[0049] Among them, φ1 represents the model parameters of the auxiliary task selector, γ1 represents the learning rate, P(ξ) represents the sampling probability of the auxiliary task, represents the classification loss;

[0050] For the target task selector, contrast loss is used as the reward to optimize the model parameters of the target task selector, which is expressed as:

[0051]

[0052] Among them, φ2 represents the model parameters of the target task selector, γ2 represents the learning rate, and P(η) represents the sampling probability of sampling the target task. represents contrast loss, and the arrow ← represents update.

[0053] In step 7, the molecular properties are predicted using the parameter-optimized auxiliary task selector and prediction model.

[0054] In the embodiment, the prediction of molecular properties using the parameter-optimized auxiliary task selector and prediction model includes:

[0055] (a) The auxiliary task selector based on parameter optimization samples auxiliary tasks of the known target task. The specific selection and sampling process is shown in step 3 and will not be repeated here.

[0056] (b) Construct a new sample-task relationship diagram based on known target tasks, auxiliary tasks, and molecular samples.

[0057] like Figure 2As shown in Figure 1, a new sample-task relationship graph is constructed based on the molecular sample, known target task, auxiliary task, and the task labels of the molecular sample on the known target task and auxiliary task. The known target task, auxiliary task, and molecular sample are used as nodes in the new sample-task relationship graph, and the task labels of the molecular sample on the task are used as the attributes of the edges between the nodes.

[0058] (c) The parameter-optimized prediction model is used to predict the type sample-task relationship graph to obtain the label prediction probability of the molecular sample on the known target task and auxiliary task, thereby realizing the prediction of molecular properties.

[0059] When using the parameter-optimized prediction model to predict the new sample-task relationship graph, the graph representation model is used to extract the node representation of the molecular sample, the node representation of the known target task, and the node representation of the auxiliary task in the input new sample-task relationship graph, and then the classifier is used to predict the label prediction probability of the molecular sample on the known target task and the auxiliary task based on the two types of input node representations.

[0060] In the embodiment, when performing molecular property prediction, the method further includes: after constructing a sample-task relationship graph based on the known target task, auxiliary tasks and molecular samples, re-tuning the parameter-optimized prediction model using the constructed new sample-task relationship graph, and the loss function used for re-tuning is the classification loss. The re-tuned prediction model is then used to predict the new sample-task relationship graph to obtain the label prediction probability of the molecular sample on the known target task and auxiliary task, thereby realizing the prediction of molecular properties.

[0061] The meta-learning molecular property prediction method based on the sample-task relationship graph provided in the embodiment, on the basis of constructing the sample-task relationship graph, reduces the deviation during training by introducing contrastive learning loss, thereby improving the accuracy of meta-learning molecular property prediction.

[0062] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A meta-learning molecular property prediction method based on sample-task relationship graph, characterized in that: The following steps are involved: Step 1: Create a sample-task relationship graph based on the molecular samples and the corresponding tasks, where both the molecular samples and the tasks are used as nodes in the graph, and the task labels corresponding to the molecular samples and tasks are used as attributes of the edges between the nodes; Step 2: Build a prediction model that includes a graph representation model and a classifier. Use the graph representation model to extract the node representation of the molecular sample and the node representation of the task in the input sample-task relationship graph. Use the classifier to predict the task label based on the concatenation of the node representation of the molecular sample and the node representation of the task to output the label prediction probability. Step 3: Build an auxiliary task selector based on LSTM, and use the auxiliary task selector to sample auxiliary tasks for the known target task. Step 4: Build a target task selector based on LSTM, use auxiliary tasks and adopt the target task selector to select the sampled target tasks in the same batch; Step 5: Based on the same sampling target task, additional molecular samples are sampled simultaneously to construct two sets of sample-task relationship graphs to form positive sample pairs. Based on different sampling target tasks in the same batch, additional molecular samples are sampled to construct two different sets of sample-task relationship graphs to form negative sample pairs. The contrast loss is constructed by maximizing the consistency between the positive sample pairs and the consistency between the negative sample pairs. Step 6: Optimize the parameters of the auxiliary task selector, target task selector, and prediction model based on the contrast loss and the classification loss constructed according to the label prediction probability; In step 7, the molecular properties are predicted using the parameter-optimized auxiliary task selector and prediction model.

2. The meta-learning molecular property prediction method based on the sample-task relationship graph according to claim 1 is characterized in that: Before the sample-task relationship graph is input into the graph representation model, it needs to undergo node initialization. The node representation of the molecular sample is initialized to the primary vector of the molecular sample, which is obtained by learning the representation of the molecular sample through the molecular representation model. The node representation of the task is initialized to a random vector. The graph representation model and the molecular representation model are both pluggable graph encoding models built based on graph neural networks, where graph neural networks include GCN, GIN, GraphSAGE, and MPNN.

3. The meta-learning molecular property prediction method based on the sample-task relationship graph according to claim 1 is characterized in that: Use the auxiliary task selector to sample auxiliary tasks for known target tasks, including: The first representation similarity between the vector representation of the known target task and the vector representation of the candidate auxiliary task, and the label similarity of the molecular sample on the known target task and the candidate auxiliary task are used as the input of the auxiliary task selector. The sampling probability of the candidate auxiliary task is output after calculation, and auxiliary tasks are sampled from the candidate auxiliary tasks according to the sampling probability.

4. The method for predicting molecular properties by meta-learning based on a sample-task relationship graph according to claim 3, characterized in that: The first representation of the similarity is given by Calculated, where are the vector representations of the known target task and the candidate auxiliary task, respectively. The vector representations are randomly initialized. Indicates the similarity calculation method, using cosine similarity calculation method; The tag similarity is determined by Calculated, where are the task label vectors of the molecular samples relative to the known target task and the candidate auxiliary task, respectively. Indicates the calculation method of the distance between task label vectors, which uses the Euclidean distance calculation method.

5. The meta-learning molecular property prediction method based on sample-task relationship graph according to claim 1, characterized in that: Leverage auxiliary tasks and adopt a target task selector to sample target tasks within the same batch, including: The prediction loss value of the prediction model for molecular samples on the known target task, the prediction loss value of the prediction model for molecular samples on the auxiliary task, and the second representation similarity between the vector representation of the known target task and the vector representation of the auxiliary task are used as inputs of the target task selector. The sampling probability of the sampling target task is output after calculation, and the auxiliary task is sampled from the auxiliary tasks as the sampling target task according to the sampling probability.

6. The meta-learning molecular property prediction method based on the sample-task relationship graph according to claim 5 is characterized in that: The prediction loss value of the prediction model for the molecular sample on the known target task is the cross entropy value between the label prediction probability of the molecular sample for the known target task and the true value of the task label; The prediction loss value of the prediction model for the molecular sample on the auxiliary task is the cross entropy value between the prediction probability of the label of the molecular sample for the auxiliary task and the true value of the task label; The second representation of similarity is achieved by Calculated, where are the vector representations of the known target task and the auxiliary task respectively, and the vector representation is the node representation output by the graph representation model, Indicates the similarity calculation method, which uses cosine similarity.

7. The meta-learning molecular property prediction method based on sample-task relationship graph according to claim 1, characterized in that: Constructed contrast loss Expressed as: ； in, is the batch size, Represent the same sampling target task The nodes on the two sets of sample-task relationship graphs in the positive sample pairs are represented as follows: Represents different sampling target tasks A set of node representations on the sample-task relationship graph in the negative sample pair, is the temperature parameter, As the similarity calculation method, cosine similarity is used.

8. The meta-learning molecular property prediction method based on sample-task relationship graph according to claim 1, characterized in that: The classification loss is the cross entropy loss between the label prediction probability and the task label true value. The sum of the classification loss and the contrast loss is used to optimize the parameters of the prediction model. The auxiliary task selector and target task selector adopt the learning strategy of reinforcement learning, including: For the auxiliary task selector, negative classification loss is used as a reward to optimize the model parameters of the auxiliary task selector, which is expressed as: ； in, represents the model parameters of the auxiliary task selector, represents the learning rate, represents the sampling probability of the auxiliary task, represents the classification loss; For the target task selector, contrast loss is used as the reward to optimize the model parameters of the target task selector, which is expressed as: ； in, represents the model parameters of the target task selector, represents the learning rate, represents the sampling probability of the sampling target task, Indicates contrast loss, arrow Indicates an update.

9. The meta-learning molecular property prediction method based on sample-task relationship graph according to claim 1, characterized in that: Predict molecular properties using parameter-optimized auxiliary task selectors and prediction models, including: (a) Auxiliary task selector based on parameter optimization to sample auxiliary tasks for known target tasks; (b) Construct a new sample-task relationship graph based on known target tasks, auxiliary tasks, and molecular samples; (c) Use the parameter-optimized prediction model to predict the new sample-task relationship graph to obtain the label prediction probability of the molecular sample on the known target task and auxiliary task, and realize the prediction of molecular properties.

10. The meta-learning molecular property prediction method based on sample-task relationship graph according to claim 9, characterized in that: It also includes: using the constructed new sample-task relationship graph to re-tune the parameter-optimized prediction model, the loss function used for re-tuning is classification loss, and then using the re-tuned prediction model to predict the new sample-task relationship graph to obtain the label prediction probability of the molecular sample on the known target task and auxiliary task, thereby realizing the prediction of molecular properties.

Images