Diffusion pre-training based graph prompt fine-tuning method and system

Abstract

The application provides a graph prompt fine-tuning method and system based on diffusion pre-training, which comprises the following steps: obtaining to-be-classified graph data, using a graph prompt fine-tuning model to classify the to-be-classified graph data, and obtaining a classification result of the to-be-classified graph data, wherein the to-be-classified graph data is graph data of a protein structure, and the classification result comprises a protein structure classification result; wherein the graph prompt fine-tuning model optimizes the graph prompt by minimizing the information bottleneck feedback, determines the graph prompt fine-tuning model according to the graph prompt, and is used for classification of the protein structure. The application has the effect that information learned from a pre-training model can be effectively transferred to a downstream classification task in a few-sample scene such as a protein structure, and the classification accuracy of the protein is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer artificial intelligence, and particularly relates to a graph prompt fine-tuning method and system based on diffusion pre-training. BACKGROUND

[0002] Graph neural networks (GNNs) can effectively model non-Euclidean data consisting of interconnected nodes and edges. On this basis, GNNs are good at classifying the labels of nodes and graphs, such as classifying user roles in social networks and retrieving researchers' research fields in citation networks. Current GNN-based classification methods largely rely on a large amount of labeled data to identify the features of a certain class, and adopt the learning paradigm of "pre-training, prompting and prediction". However, the existing acquisition of a large amount of well-labeled data set is both time-consuming and laborious, and the learning paradigm based on meta-learning still needs a large amount of well-labeled data set.

[0003] Generally, in the prompt learning paradigm, pre-training tasks such as link prediction for protein molecules are often used for node and graph classification. However, the use of such pre-training tasks may bring irrelevant and noisy information to downstream tasks. The difference may be due to the performance of the pre-training model in the link prediction task, which pulls topologically similar nodes closer in the embedding space rather than feature-similar nodes. As shown in Figure 1 , in the link prediction task, the model may choose to connect nodes of different classes as positive samples. However, as shown in Figure 2 , in the downstream classification task, the positive samples should belong to the same class. Therefore, there may be a mismatch between the goals of the pre-training task and the downstream task. In addition, the construction of the prompt is another key factor in prompt learning, because it determines how much knowledge is extracted from the pre-training model to facilitate the downstream task. Traditionally, weight vectors are used as prompts and combined with node or graph embeddings through Hadamard product to produce task-specific representations. However, such a scheme may destroy the structural relationship and fail to address the unique characteristics of graph data, resulting in difficulty in effectively transferring the learned information from the pre-training model to the downstream classification task in the few-shot scenario, which leads to errors in the classification of graph structure data and low classification efficiency. SUMMARY

[0004] The application provides a graph prompt fine-tuning method and system based on diffusion pre-training, which improves the classification accuracy of protein structures.

[0005] In a first aspect, the application provides a graph prompt fine-tuning method based on diffusion pre-training, which comprises the following steps:

[0006] obtaining to-be-classified graph data, performing classification processing on the to-be-classified graph data using a graph prompt fine-tuning model to obtain a classification result of the to-be-classified graph data, wherein the to-be-classified graph data is graph data of a protein structure, and wherein the classification result includes a protein structure classification result;

[0007] The training process of the graph prompt fine-tuning model includes:

[0008] obtaining a plurality of unlabeled graph networks of protein structures;

[0009] determining a query node in the unlabeled graph network and a pre-training sample pair corresponding to the query node by using a preset graph diffusion convolution function, wherein the query node is a sequence and structural characteristics of a protein structure;

[0010] inputting all the unlabeled graph networks into a preset graph neural network model, and combining the query node and the pre-training sample pair to complete pre-training of the graph neural network model;

[0011] obtaining a task target graph network of a downstream classification task and a task-specific label of the downstream classification task, wherein the downstream classification task is a protein structure classification task, and wherein the task-specific label is used to represent a specific protein structure that is designated and predicted;

[0012] inputting the task target graph network and the task-specific label into the pre-trained graph neural network model to obtain a first target graph network containing a general embedding;

[0013] constructing a graph prompt for the downstream classification task based on a graph convolution layer, wherein the graph prompt is used for a node degree, a clustering coefficient, a path length, and a centrality index of graph data of a protein;

[0014] deleting a preset number of edges in the first target graph network, and performing a prompt step on the first target graph network after the edges are deleted by using the graph prompt to obtain a second target graph network containing a task-specific embedding;

[0015] performing few-shot classification of the downstream classification task based on the second target graph network, and calculating a task loss of the downstream classification task in the few-shot classification process;

[0016] calculating a cosine similarity between the general embedding and the task-specific embedding;

[0017] calculating an information bottleneck of the downstream classification task in combination with the task loss and the cosine similarity;

[0018] The graph prompt is optimized by minimizing the information bottleneck feedback, and a graph prompt fine-tuning model is determined according to the graph prompt. Optionally, the step of determining the query node in the unlabeled graph network and the pre-training sample pair corresponding to the query node by using the preset graph diffusion convolution function comprises the following steps:

[0019] inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network;

[0020] determining a query node in the diffusion matrix;

[0021] selecting a maximum value of a row in which the query node is located in the diffusion matrix as a positive node index to obtain a pre-training positive sample corresponding to the query node;

[0022] obtaining a pre-training negative sample corresponding to the query node based on a structure-based sampling method;

[0023] combining the pre-training positive sample and the pre-training negative sample into a pre-training sample pair.

[0024] Optionally, the step of inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network comprises the following steps:

[0025] inputting the unlabeled graph network into a preset graph diffusion convolution function, and obtaining a diffusion matrix of the unlabeled graph network by using a diffusion algorithm in the graph diffusion convolution function, wherein the diffusion algorithm is as follows:

[0026]

[0027] wherein S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents a diffusion step length, a∈(0, 1) represents a probability that a node in the unlabeled graph network transmits a message through a connection thereof, and T represents a symmetric transition matrix;

[0028] the symmetric transition matrix is as follows:

[0029] T=I N +D -1 / 2 AD 1 / 2

[0030] wherein I N represents a unit matrix of a self-loop in the unlabeled graph network, A represents an adjacency matrix of the unlabeled graph network, and D represents a first degree matrix of a node.

[0031] Optionally, the expression formula of the graph prompt is as follows:

[0032] h φ (G′)=σ(D′ -1 / 2 A′D′ 1 / 2 X′W)

[0033] Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, W represents the weight matrix of the graph prompt, and X′ is the matrix of the transformed features.

[0034] Optionally, the pre-training loss function in the pre-training process of the graph neural network model is expressed as follows:

[0035]

[0036] Where: L pre represents the pre-training loss function, v q represents the query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, score(v1, v2) represents a score function used to measure the feature similarity between node v1 and node v2, and exp(·) represents an exponential function.

[0037] The present invention also provides a graph prompt fine-tuning system based on diffusion pre-training, the system comprising:

[0038] a graph data recognition module, which obtains graph data to be classified, and performs classification processing on the graph data to be classified using a graph hint fine-tuning model to obtain a classification result of the graph data to be classified, wherein the graph data to be classified is graph data of a protein structure, and wherein the classification result includes a protein structure classification result;

[0039] Sample graph acquisition module, used to obtain unlabeled graph networks of multiple protein structures;

[0040] a sample pair selection module, configured to determine a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node using a preset graph diffusion convolution function, wherein the query node is a sequence and structural characteristics of a protein structure;

[0041] A pre-training module, configured to input all of the unlabeled graph networks into a preset graph neural network model, and complete the pre-training of the graph neural network model by combining the query nodes and the pre-training sample pairs;

[0042] a downstream task acquisition module, configured to acquire a task target graph network for a downstream classification task and a task-specific label for the downstream classification task, wherein the downstream classification task is a protein structure classification task, and wherein the task-specific label is used to characterize a specified and predicted specific protein structure;

[0043] A model application module, configured to input the task target graph network and the task feature labels into the pre-trained graph neural network model to obtain a first target graph network containing a universal embedding;

[0044] A graph hint construction module, configured to construct graph hints for the downstream classification task based on a graph convolutional layer, wherein the graph hints are used for node degree, clustering coefficient, path length, and centrality index of protein graph data;

[0045] a graph hint application module, configured to delete a preset number of edges from the first target graph network, and use the graph hint to complete a hint step on the first target graph network after the edges are deleted, to obtain a second target graph network containing task-specific embeddings;

[0046] a loss calculation module, configured to complete the few-shot classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task during the few-shot classification process;

[0047] a similarity calculation module for calculating the cosine similarity between the universal embedding and the task-specific embedding;

[0048] An information bottleneck calculation module, configured to calculate the information bottleneck of the downstream classification task by combining the task loss and the cosine similarity;

[0049] A graph hint fine-tuning module is configured to optimize the graph hint by minimizing the information bottleneck feedback and determine a graph hint fine-tuning model based on the graph hint. Optionally, the sample pair selection module includes:

[0050] A matrix acquisition unit, configured to input the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network;

[0051] a node selection unit, configured to determine a query node in the diffusion matrix;

[0052] a positive sample selection unit, configured to select the maximum value of the row where the query node is located in the diffusion matrix as a positive node index, and obtain a pre-trained positive sample corresponding to the query node;

[0053] A negative sample selection unit, configured to select the node with the lowest similarity in the diffusion matrix as a pre-trained negative sample corresponding to the query node based on a structure-based sampling method;

[0054] The sample combination unit is used to combine the pre-training positive sample and the pre-training negative sample into a pre-training sample pair.

[0055] Optionally, the matrix acquisition unit is specifically configured to input the unlabeled graph network into a preset graph diffusion convolution function, and obtain the diffusion matrix of the unlabeled graph network using a diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows:

[0056]

[0057] Where: S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents the diffusion step size, α∈(0,1) represents the probability that a node in the unlabeled graph network transmits a message through its connection, and T represents the symmetric transfer matrix;

[0058] The formula of the symmetric transfer matrix is ​​as follows:

[0059] T=I N +D -1 / 2 AD 1 / 2

[0060] Where: I N represents the identity matrix of self-loops in the unlabeled graph network, A represents the adjacency matrix of the unlabeled graph network, and D represents the first degree matrix of the nodes.

[0061] Optionally, the expression formula of the graph prompt is as follows:

[0062] h φ (G′)=σ(D′ -1 / 2 A′D′ 1 / 2 X′W)

[0063] Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, W represents the weight matrix of the graph prompt, and X′ is the matrix of the transformed features.

[0064] Optionally, the pre-training loss function in the pre-training process of the graph neural network model is expressed as follows:

[0065]

[0066] Where: L pre represents the pre-training loss function, v q represents the query node, represents the pre-training positive sample in the pre-training sample pair, denotes a pre-training negative sample in the pair of pre-training samples, score(v1, v2) denotes a score function for measuring feature similarity between node v1 and node v2, and exp(·) denotes an exponential function.

[0067] The beneficial effects of the present application are: the protein structure graph data is sampled and queried for similar positive instance nodes based on the diffusion strategy, which generates a robust node representation for the graph data in this way, thereby reducing the difference between pre-training and downstream tasks; and the graph convolution layer is used as a graph prompt to improve the effectiveness of the model in processing specific downstream tasks during training and effectively capture the structure and node features in the graph data; the loss function based on the information bottleneck optimizes the prompt layer, which retains the minimum sufficient statistics of the input data and discards redundant information, thereby reducing the influence of irrelevant and noisy features in the previous process on the generalization to downstream tasks; and the obtained graph prompt fine-tuning model improves the protein structure classification accuracy and classification efficiency when classifying the graph data. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 Node diagram for the pre-training phase link prediction task.

[0069] Figure 2 Node diagram for the downstream classification task.

[0070] Figure 3 Flowchart of the graph prompt fine-tuning method based on diffusion pre-training in the embodiment of the present application.

[0071] Figure 4 Flowchart of the graph prompt fine-tuning method based on diffusion pre-training in the embodiment of the present application. DETAILED DESCRIPTION

[0072] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0073] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0074] Figure 3 FIG1 is a flow chart of a method and system for fine-tuning graph hints based on diffusion pre-training in one embodiment. It should be understood that although Figure 3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 3 At least part of the steps in the above process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps. Figure 3 As shown, the present invention discloses a method and system for fine-tuning image prompts based on diffusion pre-training, which specifically includes the following steps:

[0075] In some embodiments, the graph data to be classified is classified using a graph hint fine-tuning model to obtain a classification result of the graph data to be classified, wherein the graph data to be classified is graph data of protein structure, and the classification result includes a protein structure classification result.

[0076] S101. Obtaining unlabeled graph networks of multiple protein structures.

[0077] Among them, the graph network or graph data (hereinafter referred to as graph) can be defined as a tuple G = (V, A), where V = {v1, v2, ..., v N} represents the vertex set, N = |V| represents the number of vertices, and A = {0, 1} N×N Is the adjacency matrix, which represents the connection between vertices, that is, the structure of the graph. If the node v i and v j Connected by A i，j =1 means, otherwise A i，j= 0. In addition, a property graph G = (V, A, X) has a node feature matrix Where F represents the number of dimensions. Note that each node v i By vector Description, this is the i-th row in the matrix X. By preparing a large amount of unlabeled graph structure data, it can be used to help the model learn general features or domain-specific features of the graph.

[0078] In one embodiment, taking the ENZYMES dataset as an example, the dataset includes 600 molecules in 6 categories obtained from the BRENDA enzyme database: each graph network represents a protein molecule, and each node in the graph network represents a component unit (such as an amino acid) in the protein molecule. The component unit is a secondary structure element of the protein, and each node feature includes AA length, 3d length, van der Waals volume, polarity and polarizability, etc. Specifically, the dataset determines a total normalized van der Waals value for each node separately, and marks the total number of residues with low, medium and high normalized van der Waals volumes, which is called a 3-bin distribution. The length of each secondary structure element in the atomic energy spectrum is called AA length, C α The distance between the first and last residues of an atom constitutes the 3D length. S102. Determine a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node using a preset graph diffusion convolution function.

[0079] In some embodiments, where the query node is a sequence and structural property of a protein structure, the choice of query node may affect the accuracy of the prediction for a protein structure. If the query node is an amino acid residue, then the chemical properties of the residue (such as charge, polarity, and size) and its position in the protein may affect its role in the three-dimensional structure. Similarly, if the query node is a larger protein fragment or the entire protein, then its sequence and structural properties (such as the distribution and sequence pattern of secondary structure elements) may affect the prediction results. Therefore, it is necessary to determine the protein structure to improve the prediction and classification effect.

[0080] Among them, in the pre-training stage, the goal is to narrow the gap between nodes in the same category and increase the distance between nodes in different categories. However, when the graph structure and the feature matrix When nodes are available but their labels are unknown, nodes with high similarity have the same label, while nodes with low similarity have different labels.

[0081] In one embodiment, step S102 specifically includes the following steps:

[0082] Input the unlabeled graph network into the preset graph diffusion convolution function to obtain the diffusion matrix of the unlabeled graph network;

[0083] Determine the query node in the diffusion matrix;

[0084] Select the maximum value of the row where the query node is located in the diffusion matrix as the positive node index, and obtain the pre-trained positive sample corresponding to the query node;

[0085] The structure-based sampling method uses the node with the lowest similarity in the diffusion matrix as the pre-training negative sample corresponding to the query node;

[0086] The pre-training positive samples and pre-training negative samples are combined into pre-training sample pairs, where the query nodes are the sequence and structural features of the protein structure.

[0087] In this embodiment, the step of inputting the unlabeled graph network into the preset graph diffusion convolution function to obtain the diffusion matrix of the unlabeled graph network is as follows:

[0088] The unlabeled graph network is input into the preset graph diffusion convolution function, and the diffusion matrix of the unlabeled graph network is obtained using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows:

[0089]

[0090] where S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents the diffusion step size, α∈(0,1) represents the probability that a node in the unlabeled graph network transmits a message through its connection, and T represents the symmetric transfer matrix.

[0091] In order to strike a balance between sampling quality and time and space overhead, a diffusion-based strategy is used to sample positive node instances in the unlabeled case, which can capture long-range information in the graph to enrich the samples of the pre-training task. Formally, given a graph G, it is first input into a graph diffusion convolution function GDC(·) to obtain a diffusion matrix S. Each element s in the diffusion matrix S is ij Representative node v i and v j degree of proximity.

[0092] The formula of the symmetric transfer matrix is ​​as follows:

[0093] T=I N +D -1 / 2 AD 1 / 2

[0094] Where: I N represents the identity matrix of self-loops in the unlabeled graph network, A represents the adjacency matrix of the unlabeled graph network, and D represents the first degree matrix of the node, that is, The diffusion operation achieves linear running time O(N).

[0095] To obtain the query node v q Pre-training positive samples The index of the maximum value in the qth row of the diffusion matrix S is selected as the pre-training positive sample index. The formula is as follows:

[0096]

[0097] Where Sq,m represents the value of the mth element in the qth row of the diffusion matrix S.

[0098] For node v q Pre-training negative samples A structure-based sampling method can be used to add a self-loop to each node and prioritize the nodes that are related to v q Sampling in unconnected nodes, if v q is connected to all other nodes in the graph G, i.e. The nodes with the lowest similarity can be directly regarded as pre-training negative samples:

[0099]

[0100] Wherein, score(v1, v2) represents a score function used to measure the feature similarity between node v1 and node v2.

[0101] S103. Input all unlabeled graph networks into the preset graph neural network model, and complete the pre-training of the graph neural network model by combining the query nodes and pre-training samples.

[0102] The unlabeled graph network is input into the preset graph neural network model to obtain the transformed graph of the lth layer and the matrix of the transformed features. The elements in the matrix X′ are expressed as follows:

[0103]

[0104] UPDATE(·) represents a node feature updater, and AGGREGATE(·) serves as an aggregation function for neighbor information. In this embodiment, a sum function may be selected for aggregation.

[0105] Finally, by comparing the similarity between the query node and its positive and negative samples, the pre-training loss function is defined based on the matrix X′. The pre-training loss function in the pre-training process of the graph neural network model is expressed as follows:

[0106]

[0107] Where: L pre represents the pre-training loss function, v q Represents a query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, score(v1, v2) represents the score function used to measure the feature similarity between node v1 and node v2, and exp(·) represents the exponential function.

[0108] S104. Obtain the task target graph network of the downstream classification task and the task specific label of the downstream classification task.

[0109] In some embodiments, the downstream classification task is a protein structure classification task, wherein task-specific labels are used to characterize specific protein structures that are assigned and predicted.

[0110] Among them, the downstream classification task is usually a graph classification task or a node classification task. For the graph classification task, there is a general graph set G, which can be divided into two subsets. The first subset Gl is a set containing all labeled graph samples and all categories in G, and the other subset G u Contains the remaining unlabeled graph samples, that is, G = G l ∪G u and Use Y G To represent the label set of graph G. The goal of graph classification is to train a model that can accurately predict the set G u The labels of the remaining unlabeled samples in . In addition, for few-shot image classification, there are only K labeled samples for each category, which is called K-shot classification. It is formally expressed as where |Y G | represents the graph label set Y G size.

[0111] Similarly, the node classification task is performed on a separate graph G = (V, A, X), where the set of labeled nodes V l and the set of unlabeled nodes V u All are in the same graph G, that is, V = V l ∪V u and Node label set is Y v The goal of node classification is to train a model that can effectively predict the set V u Labels for unlabeled nodes.

[0112] S105. Input the task target graph network and task feature labels into the pre-trained graph neural network model to obtain a first target graph network containing universal embedding.

[0113] S106. Constructing graph hints for downstream classification tasks based on graph convolutional layers.

[0114] In some embodiments, the graph hints are used for node degree, clustering coefficient, path length, and centrality metrics of graph data for proteins.

[0115] In some embodiments, the node degree is represented in a protein structure graph, and the node degree may correspond to the coordination number of an amino acid residue (ie, the number of interactions it has with other residues). Highly coordinated residues may play a key role in the structure of the protein.

[0116] In some embodiments, the clustering coefficient is represented in a protein structure graph, and the clustering coefficient may reflect the degree of interaction between the neighbors of an amino acid residue. Residues with a high clustering coefficient may be present in tightly packed regions of the protein.

[0117] In some embodiments, path length is represented in a protein structure diagram. Path length may correspond to the spatial distance or sequence distance between two amino acid residues. Path length information can help predict the three-dimensional structure of a protein.

[0118] In some embodiments, centrality indices are represented in a protein structure graph. Centrality indices (such as degree centrality, closeness centrality, and betweenness centrality) can help identify amino acid residues that play a key role in the protein structure.

[0119] Among them, due to the structural differences between graph data and text data, the original language prompt format cannot be directly adapted to graph prompts. Specifically, an obvious difference is that natural language text mainly consists of a one-dimensional, sequential structure, while graph data embodies a multi-dimensional, nonlinear network structure. From this perspective, trainable text prompts guide the language model to generate task-specific word vectors. Accordingly, a trainable graph convolution layer h can be used φ (·) is used to prompt the graphical model to generate task-specific graph vectors, which are expressed as follows:

[0120] h φ (G′)=σ(D′ -1 / 2 A′D′ 1 / 2 X′W)

[0121] Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, W represents the weight matrix of the graph hint. Through this message passing mechanism, h φ (G′) can use the structure of the graph to obtain the feature vector of the aggregated graph topology information.

[0122] S107. Delete a preset number of edges in the first target graph network, and use graph prompts to complete the prompt step for the first target graph network after the edges are deleted, to obtain a second target graph network containing task-specific embeddings.

[0123] S108. Complete the few-shot classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task during the few-shot classification process.

[0124] Among them, since the downstream classification task is usually a graph classification task or a node classification task. For the node classification task, the node set V of the graph G′ can be divided into a k-shot labeled set V l and an unlabeled set V u , that is, V={V l , V u},in Pay attention to Y V is the label set of the node, whose size is |Y V | indicates that. For node classification tasks, use The vectors in are used as node features, which are obtained by node-level specific hints In order to adapt to the small sample scenario, metric-based learning can be introduced to calculate the query node v q With class y v Prototype The distance between v ∈Y V Among them, prototype is a virtual node used as a representation of the class level. Query node v q The label of is predicted to be the class closest to its prototype:

[0125]

[0126] Where dist(·,·) describes the distance between two nodes. For convenience, we directly define dist(·,·) = 1-sCore(·,·). To obtain the class y v For the prototype embedding, we use the class y v The average value of these label nodes is used as its prototype embedding

[0127]

[0128] in is Produced A row vector in . Thus, the loss of training node classification task can be defined as

[0129]

[0130] in is the query node v q The real prototype.

[0131] For graph classification tasks, a graph-level specific hint can be applied Its structure is the same as defined above The same. Therefore, the corresponding feature matrix can be obtained Then by using the entire matrix To obtain the query graph G q Embedding:

[0132]

[0133] Where V q It is a query graph The node set of express The corresponding node features in . Readout(·) is an integration operation. Specifically, in the implementation, by It is realized by summing the representations of all nodes in Similar to the node classification task, the graph label y g The graph prototype embedding of can be obtained as follows:

[0134]

[0135] where Y G is the graph label set, and is marked as y g In addition, the loss of the graph classification task is It can be calculated as follows:

[0136]

[0137] in It is a query graph The real prototype.

[0138] S109. Compute the cosine similarity between the universal embedding and the task-specific embedding.

[0139] S110. Combining task loss and cosine similarity to calculate the information bottleneck of downstream classification tasks.

[0140] S111. Optimize graph hints by minimizing information bottleneck feedback, and determine graph hint fine-tuning models based on the graph hints.

[0141] In one embodiment, for steps S109 to S111, since the prompting process may introduce a large amount of noise features, the image prompt cannot directly convey the information of the pre-trained model. Therefore, an attempt can be made to reduce the influence of irrelevant features by minimizing the information bottleneck (IB). The information bottleneck expression formula is as follows:

[0142]

[0143] in is the original input, Y represents the query label set, Z represents the embedding to be optimized, and β is a positive real number used to balance the two terms. In the embodiment, In the image classification task, it refers to the atlas In node-level tasks, it refers to a single graph G. I(·;·) represents the mutual information between two variables. The information bottleneck aims to learn a minimal sufficient representation for a given task by maximizing the mutual information between the input graph embedding and the target label. Furthermore, it also constrains the mutual information between the input graph embedding and the original features of the input graph.

[0144] Specifically, the first term of the information bottleneck expression formula represents the mutual information between the task-specific label and the graph embedding. According to the relationship between mutual information and entropy, I(Y; Z) can be expressed as:

[0145] I(Y; Z) = H(Y) - H(Y| Z),

[0146] Where H(Y) represents the entropy of Y. When Y is a single dataset, its entropy can be regarded as a fixed value, so we can focus on the entropy between the task-specific labels and the graph embedding, that is, H(Y|Z). Formally, according to the definition of entropy, H(Y|Z) can be expressed as:

[0147]

[0148] For a classification task, p(z j |y i ) can be calculated by an indicator function, namely p(z j |y i )=1(z j ,y i ), where if y i It is z j The true label of p(z j |y i )=1, otherwise p(z j |y i )=0. The number of available instances for each class is K, so p(y i ) is a uniform distribution. Therefore, the loss function That is, the loss of the training node classification task and the loss of the graph classification task, which refers to the first term I(Y;Z) of the information bottleneck expression formula.

[0149] For the second term of the information bottleneck expression formula, namely I(D;Z), it can be expressed in the desired form as follows:

[0150]

[0151] However, since p(Z) cannot be obtained, it is impractical to use the above formula directly for calculation. Therefore, an auxiliary distribution r(Z) can be introduced to separate p(Z) in order to construct a variational upper bound for the above formula:

[0152]

[0153] in is the constructed variational upper bound, and D KL (·||·) represents the KL divergence. Therefore, minimizing The process can be transformed into minimizing its variational upper bound. In detail, the pre-trained GNN is used to model the conditional distribution And r(Z) can be instantiated as any GNN model. In the implementation, in order to reduce the number of parameters, the graph hint layer h is used φ (·) Instantiate r(Z). In addition, to eliminate the dependence of r(Z) on the original input, a certain proportion of edges in the graph are randomly dropped before feeding the graph into the hint layer.

[0154] Note that to simplify the computation of the variational upper bound, it can be converted to computing the cosine similarity between the embeddings obtained by the pre-trained GNN and the graph hint layer:

[0155]

[0156] where x′ represents the output of the pre-trained model, and Indicates the diagram prompt h φ Therefore, the final downstream task loss is the information bottleneck, that is, the information bottleneck expression formula can be expressed as

[0157]

[0158] In one embodiment, an optimized second target graph network is determined by graph prompts, and a graph prompt fine-tuning model is determined using the optimized second target graph network and a pre-trained GNN.

[0159] S201, obtaining graph data to be classified, and performing classification processing on the graph data to be classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified.

[0160] In one embodiment, the graph data to be classified may also be a BRENDA enzyme graph network, a protein-like graph network, a product recommendation network, etc.

[0161] In one embodiment, graph data or a graph network is a nonlinear data structure consisting of nodes (or vertices) and edges. Graph data structures can be used to represent network structures, such as social networks, the internet, and transportation networks. In a graph, nodes typically represent entities, while edges represent relationships between entities.

[0162] In one embodiment, a total of 600 molecules in 6 categories obtained from the BRENDA enzyme database are used as an example: each graph network represents a protein molecule, and each node in the graph network represents a component unit in the protein molecule, which is a secondary structure element of the protein. Each node feature includes AA length, 3d length, van der Waals volume, polarity, and polarizability. Specifically, the dataset determines a total normalized van der Waals value for each node and marks the total number of residues with low, medium, and high normalized van der Waals volumes, which is called a 3-bin distribution. The length of each secondary structure element in the atomic energy spectrum is called AA length, and C α The distance between the first and last atoms in a residue constitutes the 3d length.

[0163] In one embodiment, the classification processing of the graph prompt fine-tuning model can obtain the node classification results of the graph data, the classification results of the relationship between nodes and the classification results of the graph data (graph network). For example, the node classification results are such as the AA length, 3D length, van der Waals volume, polarity and polarizability of the above-mentioned protein structure; the classification results of the relationship between nodes are such as the length of each secondary structure element in the atomic energy spectrum is called AA length, C α The distance between the first and last atoms in a residue constitutes the 3d length.

[0164] Reference Figure 4 , Figure 4 This is a process framework diagram of the graph hint fine-tuning method based on diffusion pre-training, combined with Figure 4 The flowchart in FIG. 1 can more clearly describe the operation process of the above steps S101 to S111.

[0165] The present invention also discloses a graph prompt fine-tuning system based on diffusion pre-training, which specifically includes:

[0166] Sample graph acquisition module, used to obtain multiple unlabeled graph networks;

[0167] A sample pair selection module is used to determine the query node in the unlabeled graph network and the pre-trained sample pair corresponding to the query node using a preset graph diffusion convolution function;

[0168] The pre-training module is used to input all unlabeled graph networks into the preset graph neural network model, and complete the pre-training of the graph neural network model by combining query nodes and pre-training sample pairs;

[0169] Downstream task acquisition module, used to obtain the task target graph network of the downstream classification task and the task specific label of the downstream classification task;

[0170] A model application module is used to input the task target graph network and task feature labels into the pre-trained graph neural network model to obtain a first target graph network containing a universal embedding;

[0171] A graph hint building module, which is used to build graph hints for downstream classification tasks based on graph convolutional layers;

[0172] a graph hint application module, configured to delete a preset number of edges from the first target graph network, and use the graph hint to complete the hint step on the first target graph network after the edges are deleted, to obtain a second target graph network containing the task-specific embedding;

[0173] A loss calculation module is used to complete the few-shot classification of the downstream classification task based on the second target graph network and calculate the task loss of the downstream classification task during the few-shot classification process;

[0174] Similarity calculation module, which is used to calculate the cosine similarity between the universal embedding and the task-specific embedding;

[0175] The information bottleneck calculation module is used to combine task loss and cosine similarity to calculate the information bottleneck of downstream classification tasks;

[0176] A graph hint fine-tuning module is used to optimize graph hints by minimizing information bottleneck feedback.

[0177] In one embodiment, the sample pair selection module includes:

[0178] A matrix acquisition unit, used to input the unlabeled graph network into a preset graph diffusion convolution function to obtain the diffusion matrix of the unlabeled graph network;

[0179] a node selection unit for determining a query node in the diffusion matrix;

[0180] A positive sample selection unit is used to select the maximum value of the row where the query node is located in the diffusion matrix as the positive node index, and obtain the pre-trained positive sample corresponding to the query node;

[0181] A negative sample selection unit is used to select the node with the lowest similarity in the diffusion matrix as the pre-trained negative sample corresponding to the query node based on the structure-based sampling method;

[0182] The sample combination unit is used to combine pre-training positive samples and pre-training negative samples into pre-training sample pairs.

[0183] In this embodiment, the matrix acquisition unit is specifically used to input the unlabeled graph network into a preset graph diffusion convolution function, and obtain the diffusion matrix of the unlabeled graph network using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows:

[0184]

[0185] Where: S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents the diffusion step size, α∈(0,1) represents the probability that a node in the unlabeled graph network transmits a message through its connection, and T represents the symmetric transfer matrix;

[0186] The formula of the symmetric transfer matrix is ​​as follows:

[0187] T=I N +D -1 / 2 AD 1 / 2

[0188] Where: I N represents the identity matrix of self-loops in the unlabeled graph network, A represents the adjacency matrix of the unlabeled graph network, and D represents the first degree matrix of the nodes.

[0189] In one embodiment, the expression formula of the graph prompt constructed by the graph prompt construction module is as follows:

[0190] h φ (G′)=σ(D′ -1 / 2 A′D′ 1 / 2 X′W)

[0191] Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, and W represents the weight matrix of the graph hint.

[0192] In one embodiment, the pre-training loss function in the pre-training process of the graph neural network model by the pre-training module is expressed as follows:

[0193]

[0194] Where: L pre represents the pre-training loss function, vq Represents a query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, score(v1, v2) represents the score function used to measure the feature similarity between node v1 and node v2, and exp(·) represents the exponential function.

[0195] An embodiment of the present invention also includes a graph data recognition module, wherein the graph data recognition module is used to classify the graph data to be classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified. It can be understood that the graph prompt fine-tuning model is obtained through the training method as described above.

[0196] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as above, which are not provided in detail for the sake of simplicity.

[0197] The one or more embodiments of this application are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this application should be included in the scope of protection of this application.

Claims

1. A method for fine-tuning image hints based on diffusion pre-training, characterized in that: The steps include: Acquiring graph data to be classified, and performing classification processing on the graph data to be classified using a graph hint fine-tuning model to obtain a classification result of the graph data to be classified, wherein the graph data to be classified is graph data of a protein structure, and wherein the classification result includes a protein structure classification result; The figure shows that the training process of the fine-tuning model includes: Obtain unlabeled graph networks of multiple protein structures; Determining a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node using a preset graph diffusion convolution function, wherein the query node is a sequence and structural characteristics of a protein structure; Inputting all of the unlabeled graph networks into a preset graph neural network model, and combining the query nodes and the pre-trained sample pairs to complete the pre-training of the graph neural network model; Obtaining a task target graph network for a downstream classification task and a task-specific label for the downstream classification task, wherein the downstream classification task is a protein structure classification task, and wherein the task-specific label is used to characterize a specified and predicted specific protein structure; Inputting the task target graph network and the task specific label into the pre-trained graph neural network model to obtain a first target graph network containing a universal embedding; Constructing a graph hint for the downstream classification task based on the graph convolutional layer, wherein the graph hint is used to characterize the node degree, clustering coefficient, path length and centrality index of the protein graph data; Deleting a preset number of edges from the first target graph network, and completing the prompting step on the first target graph network after the edges are deleted using the graph prompt to obtain a second target graph network containing the task-specific embedding; Complete the few-shot classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task during the few-shot classification process; computing the cosine similarity between the universal embedding and the task-specific embedding; Calculating the information bottleneck of the downstream classification task by combining the task loss and the cosine similarity; The graph hint is optimized by minimizing the information bottleneck feedback, and a graph hint fine-tuning model is determined according to the graph hint.

2. The image hint fine-tuning method based on diffusion pre-training according to claim 1 is characterized in that: The method of using a preset graph diffusion convolution function to determine a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node comprises the following steps: Inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network; determining a query node in the diffusion matrix; Selecting the maximum value of the row where the query node is located in the diffusion matrix as a positive node index, and obtaining a pre-trained positive sample corresponding to the query node; The structure-based sampling method uses the node with the lowest similarity in the diffusion matrix as the pre-training negative sample corresponding to the query node; The pre-training positive samples and the pre-training negative samples are combined into a pre-training sample pair.

3. The image hint fine-tuning method based on diffusion pre-training according to claim 2 is characterized in that: Inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network comprises the following steps: The unlabeled graph network is input into a preset graph diffusion convolution function, and the diffusion matrix of the unlabeled graph network is obtained using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows: Where: represents the diffusion matrix, represents the graph diffusion convolution function, represents the unlabeled graph network, represents the diffusion step length, represents the probability that a node in the unlabeled graph network passes a message through its connection, represents the symmetric transfer matrix; The formula of the symmetric transfer matrix is ​​as follows: Where: represents the identity matrix of self-loops in the unlabeled graph network, represents the adjacency matrix of the unlabeled graph network, Represents the first degree matrix of the node.

4. The image hint fine-tuning method based on diffusion pre-training according to claim 3 is characterized in that: The expression formula of the graph is as follows: Where: Indicates the diagram prompt, represents the first target graph network, represents the activation function, , represents the second degree matrix of the node, represents the weight matrix of the graph prompt, is the matrix of the transformed features.

5. The image hint fine-tuning method based on diffusion pre-training according to claim 1, characterized in that: The pre-training loss function in the pre-training process of the graph neural network model is expressed as follows: Where: represents the pre-training loss function, represents the query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, Indicates the node used to measure and nodes The scoring function for the feature similarity between Represents the exponential function.

6. A graph hint fine-tuning system based on diffusion pre-training, characterized in that: The system comprises: A graph data recognition module is configured to obtain graph data to be classified, and classify the graph data to be classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified, wherein the graph data to be classified is graph data of a protein structure, and the classification result includes a protein structure classification result; a sample graph acquisition module is configured to obtain an unlabeled graph network of multiple protein structures; a sample pair selection module is configured to determine a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node using a preset graph diffusion convolution function, wherein the query node is a sequence and structural characteristic of a protein structure; a pre-training module is configured to input all the unlabeled graph networks into a preset graph neural network model, and complete the pre-training of the graph neural network model by combining the query node and the pre-trained sample pair; a downstream task acquisition module, configured to acquire a task target graph network for a downstream classification task and a task-specific label for the downstream classification task, wherein the downstream classification task is a protein structure classification task, and wherein the task-specific label is used to characterize a specified and predicted specific protein structure; A model application module, configured to input the task target graph network and the task specific labels into the pre-trained graph neural network model to obtain a first target graph network containing a universal embedding; A graph hint construction module, configured to construct graph hints for the downstream classification task based on the graph convolution layer, wherein the graph hints are used to characterize the node degree, clustering coefficient, path length, and centrality index of the graph data of the protein; a graph hint application module, configured to delete a preset number of edges from the first target graph network, and use the graph hint to complete a hint step on the first target graph network after the edges are deleted, to obtain a second target graph network containing task-specific embeddings; a loss calculation module, configured to complete the few-shot classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task during the few-shot classification process; a similarity calculation module for calculating the cosine similarity between the universal embedding and the task-specific embedding; An information bottleneck calculation module, configured to calculate the information bottleneck of the downstream classification task by combining the task loss and the cosine similarity; A graph hint fine-tuning module is configured to optimize the graph hint by minimizing the information bottleneck feedback, and determine a graph hint fine-tuning model according to the graph hint.

7. The image prompt fine-tuning system based on diffusion pre-training according to claim 6 is characterized in that: The sample pair selection module includes: A matrix acquisition unit, configured to input the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network; a node selection unit, configured to determine a query node in the diffusion matrix; a positive sample selection unit, configured to select the maximum value of the row where the query node is located in the diffusion matrix as a positive node index, and obtain a pre-trained positive sample corresponding to the query node; A negative sample selection unit, configured to select the node with the lowest similarity in the diffusion matrix as a pre-trained negative sample corresponding to the query node based on a structure-based sampling method; The sample combination unit is used to combine the pre-training positive sample and the pre-training negative sample into a pre-training sample pair.

8. The image prompt fine-tuning system based on diffusion pre-training according to claim 7 is characterized in that: The matrix acquisition unit is specifically used to input the unlabeled graph network into a preset graph diffusion convolution function, and obtain the diffusion matrix of the unlabeled graph network using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows: Where: represents the diffusion matrix, represents the graph diffusion convolution function, represents the unlabeled graph network, represents the diffusion step length, represents the probability that a node in the unlabeled graph network passes a message through its connection, represents the symmetric transfer matrix; The formula of the symmetric transfer matrix is ​​as follows: Where: represents the identity matrix of self-loops in the unlabeled graph network, represents the adjacency matrix of the unlabeled graph network, Represents the first degree matrix of the node.

9. The image hint fine-tuning system based on diffusion pre-training according to claim 8, characterized in that: The expression formula of the graph is as follows: Where: Indicates the diagram prompt, represents the first target graph network, represents the activation function, , represents the second degree matrix of the node, represents the weight matrix of the graph prompt, is the matrix of the transformed features.

10. The image hint fine-tuning system based on diffusion pre-training according to claim 6, characterized in that: The pre-training loss function in the pre-training process of the graph neural network model is expressed as follows: Where: represents the pre-training loss function, represents the query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, Indicates the node used to measure and nodes The scoring function for feature similarity between Represents the exponential function.

Images