Graph prompt fine tuning method and system based on diffusion pre-training

Through the graph prompt fine-tuning method based on diffusion pre-training, the problem of relying on a large amount of labeled data and noise information in the prior art is solved, the classification accuracy and efficiency of protein structure are improved, and the robust node representation and structural characteristics of graph data are realized. Capture.

CN119028448AActive Publication Date: 2024-11-26NAT UNIV OF DEFENSE TECH
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411057885.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-26
Estimated Expiration
2044-08-02

AI Technical Summary

Technical Problem

The existing protein structure classification method based on graph neural networks relies on a large amount of labeled data, and pre-training tasks may introduce uncorrelated and noisy information, resulting in low classification efficiency and poor accuracy of downstream tasks, especially in scenarios with few samples, which is difficult to effectively transfer pre-transfer. Information about training the model.

Method used

The graph prompt fine-tuning method based on diffusion pre-training is adopted to determine the query node and pre-trained sample pairs through the graph diffusion convolution function, and build graph prompts based on the graph convolution layer to optimize the information bottleneck loss function to reduce the impact of noise characteristics and improve the graph data. node representation and classification efficiency.

Benefits of technology

The classification accuracy and efficiency of protein structure are improved, the differences between pre-training and downstream tasks are reduced, the structural and node characteristics of graph data are effectively captured, and the effectiveness of the model in downstream tasks is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028448A_ABST
    Figure CN119028448A_ABST
Patent Text Reader

Abstract

The invention provides a graph prompt fine tuning method and system based on diffusion pre-training, and the method comprises the following steps: obtaining to-be-classified graph data, carrying out the classification processing of the to-be-classified graph data through a graph prompt fine tuning model, and obtaining a classification result of the to-be-classified graph data, wherein the classification result comprises a protein structure classification result; wherein the graph prompt fine-tuning model feeds back and optimizes the graph prompt by minimizing the information bottleneck, the graph prompt fine-tuning model is determined according to the graph prompt, and the graph prompt fine-tuning model is used for classifying protein structures. The method has the effect of effectively transferring the information learned from the pre-training model to a downstream classification task in a few-sample scene such as a protein structure and the like, and the classification accuracy of the protein is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer artificial intelligence technology, and in particular relates to a method and system for fine-tuning image prompts based on diffusion pre-training. Background Art

[0002] Graph neural networks (GNNs) can effectively simulate non-Euclidean spatial data consisting of interconnected nodes and edges. On this basis, GNNs are good at classifying node and graph labels, such as classifying user roles in social networks and retrieving scholars' research fields in citation networks. Current GNN-based classification methods rely heavily on a large amount of labeled data to identify the characteristics of a certain class, and adopt a learning paradigm of "pre-training, prompting, and prediction". However, it is time-consuming and laborious to obtain a large number of well-annotated datasets. At the same time, the learning paradigm based on meta-learning still requires a large number of well-annotated datasets.

[0003] Typically, in the context of the hinted learning paradigm, pre-training tasks such as link prediction for protein molecules are often used for node and graph classification. However, using such pre-training tasks may bring irrelevant and noisy information to downstream tasks. The difference may stem from the performance of pre-trained models in link prediction tasks, which pull topologically similar nodes closer in the embedding space rather than feature-similar nodes. Figure 1 As shown in , in the link prediction task, the model may select nodes that are connected but not of the same type as positive samples. Figure 2 As shown, in the downstream classification task, the positive samples should belong to the same class. Therefore, there may be a mismatch between the objectives of the pre-training task and the downstream task. In addition, the construction of hints is another key factor in hint learning because it determines how much knowledge is elicited from the pre-trained model to facilitate the downstream task. Traditionally, weight vectors are used as hints and combined with node or graph embeddings through the Hadamard product to produce task-specific representations. However, such schemes may destroy structural relationships and fail to address the unique characteristics of graph data, making it difficult to effectively transfer the information learned from the pre-training model to downstream classification tasks in few-sample scenarios, which can cause errors in the classification of graph-structured data and low classification efficiency. Summary of the invention

[0004] The present invention provides a method and system for fine-tuning graph prompts based on diffusion pre-training, which improves the classification accuracy of protein structures.

[0005] In a first aspect, the present invention provides a method for fine-tuning a graph prompt based on diffusion pre-training, the method comprising the following steps:

[0006] Acquire graph data to be classified, and classify the graph data to be classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified, wherein the graph data to be classified is graph data of a protein structure, and wherein the classification result includes a protein structure classification result;

[0007] The figure shows that the training process of the fine-tuning model includes:

[0008] Obtain unlabeled graph networks of multiple protein structures;

[0009] Determine a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node using a preset graph diffusion convolution function, wherein the query node is a sequence and a structural characteristic of a protein structure;

[0010] Inputting all of the unlabeled graph networks into a preset graph neural network model, and combining the query nodes and the pre-trained sample pairs to complete the pre-training of the graph neural network model;

[0011] Obtaining a task target graph network for a downstream classification task and a task specific label for the downstream classification task, wherein the downstream classification task is a protein structure classification task, wherein the task specific label is used to characterize a specified and predicted specific protein structure;

[0012] Inputting the task target graph network and the task feature labels into the pre-trained graph neural network model to obtain a first target graph network containing a universal embedding;

[0013] Constructing a graph prompt for the downstream classification task based on the graph convolution layer, wherein the graph prompt is used for node degree, clustering coefficient, path length and centrality index of graph data of proteins;

[0014] Deleting a preset number of edges in the first target graph network, and using the graph prompt to complete the prompt step on the first target graph network after the edges are deleted, to obtain a second target graph network containing task-specific embeddings;

[0015] Complete the few-sample classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task in the few-sample classification process;

[0016] computing a cosine similarity between the universal embedding and the task-specific embedding;

[0017] Calculating the information bottleneck of the downstream classification task by combining the task loss and the cosine similarity;

[0018] The graph prompt is optimized by minimizing the information bottleneck feedback, and a graph prompt fine-tuning model is determined according to the graph prompt. Optionally, the step of determining the query node in the unlabeled graph network and the pre-trained sample pair corresponding to the query node by using a preset graph diffusion convolution function comprises the following steps:

[0019] Inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network;

[0020] determining a query node in the diffusion matrix;

[0021] Selecting the maximum value of the row where the query node is located in the diffusion matrix as a positive node index, and obtaining a pre-trained positive sample corresponding to the query node;

[0022] The structure-based sampling method uses the node with the lowest similarity in the diffusion matrix as the pre-trained negative sample corresponding to the query node;

[0023] The pre-training positive sample and the pre-training negative sample are combined into a pre-training sample pair.

[0024] Optionally, inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network comprises the following steps:

[0025] The unlabeled graph network is input into a preset graph diffusion convolution function, and the diffusion matrix of the unlabeled graph network is obtained by using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows:

[0026]

[0027] Where: S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents the diffusion step, α∈(0,1) represents the probability that a node in the unlabeled graph network transmits a message through its connection, and T represents the symmetric transfer matrix;

[0028] The formula of the symmetric transfer matrix is ​​as follows:

[0029] T=I N +D -1 / 2 AD 1 / 2

[0030] Where: I N represents the identity matrix of self-loops in the unlabeled graph network, A represents the adjacency matrix of the unlabeled graph network, and D represents the first degree matrix of the node.

[0031] Optionally, the expression formula of the graph prompt is as follows:

[0032] h φ (G′)=σ(D′ -1 / 2 A′D′ 1 / 2 X′W)

[0033] Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, W represents the weight matrix of the graph prompt, and X′ is the matrix of the transformed features.

[0034] Optionally, the pre-training loss function in the pre-training process of the graph neural network model is expressed as follows:

[0035]

[0036] Where: L pre represents the pre-training loss function, v q represents the query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, score(v1, v2) represents a score function used to measure the feature similarity between node v1 and node v2, and exp(·) represents an exponential function.

[0037] The present invention also provides a graph prompt fine-tuning system based on diffusion pre-training, the system comprising:

[0038] A graph data recognition module, which obtains graph data to be classified, and classifies the graph data to be classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified, wherein the graph data to be classified is graph data of a protein structure, and wherein the classification result includes a protein structure classification result;

[0039] The sample graph acquisition module is used to obtain unlabeled graph networks of multiple protein structures;

[0040] A sample pair selection module, used to determine a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node using a preset graph diffusion convolution function, wherein the query node is a sequence and a structural characteristic of a protein structure;

[0041] A pre-training module, used to input all of the unlabeled graph networks into a preset graph neural network model, and complete the pre-training of the graph neural network model by combining the query nodes and the pre-training sample pairs;

[0042] A downstream task acquisition module, used to acquire a task target graph network of a downstream classification task and a task specific label of the downstream classification task, wherein the downstream classification task is a protein structure classification task, wherein the task specific label is used to characterize a specified and predicted specific protein structure;

[0043] A model application module, used for inputting the task target graph network and the task feature labels into the pre-trained graph neural network model to obtain a first target graph network containing a universal embedding;

[0044] A graph hint construction module, used for constructing graph hints for the downstream classification task based on the graph convolution layer, wherein the graph hints are used for node degree, clustering coefficient, path length and centrality index of graph data of proteins;

[0045] A graph prompt application module, configured to delete a preset number of edges in the first target graph network, and use the graph prompt to complete a prompt step on the first target graph network after the edges are deleted, to obtain a second target graph network containing task-specific embeddings;

[0046] A loss calculation module, used to complete the few-sample classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task in the few-sample classification process;

[0047] A similarity calculation module, configured to calculate the cosine similarity between the universal embedding and the task-specific embedding;

[0048] An information bottleneck calculation module, used to calculate the information bottleneck of the downstream classification task by combining the task loss and the cosine similarity;

[0049] A graph hint fine-tuning module is used to optimize the graph hint by minimizing the information bottleneck feedback and determine a graph hint fine-tuning model according to the graph hint. Optionally, the sample pair selection module includes:

[0050] A matrix acquisition unit, used for inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network;

[0051] a node selection unit, configured to determine a query node in the diffusion matrix;

[0052] A positive sample selection unit, configured to select the maximum value of the row where the query node is located in the diffusion matrix as a positive node index, and obtain a pre-trained positive sample corresponding to the query node;

[0053] A negative sample selection unit, configured to select the node with the lowest similarity in the diffusion matrix as a pre-trained negative sample corresponding to the query node based on a structure-based sampling method;

[0054] The sample combination unit is used to combine the pre-training positive sample and the pre-training negative sample into a pre-training sample pair.

[0055] Optionally, the matrix acquisition unit is specifically used to input the unlabeled graph network into a preset graph diffusion convolution function, and obtain the diffusion matrix of the unlabeled graph network using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows:

[0056]

[0057] Where: S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents the diffusion step, α∈(0,1) represents the probability that a node in the unlabeled graph network transmits a message through its connection, and T represents the symmetric transfer matrix;

[0058] The formula of the symmetric transfer matrix is ​​as follows:

[0059] T=I N +D -1 / 2 AD 1 / 2

[0060] Where: I N represents the identity matrix of self-loops in the unlabeled graph network, A represents the adjacency matrix of the unlabeled graph network, and D represents the first degree matrix of the node.

[0061] Optionally, the expression formula of the graph prompt is as follows:

[0062] h φ (G′)=σ(D′ -1 / 2 A′D′ 1 / 2 X′W)

[0063] Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, W represents the weight matrix of the graph prompt, and X′ is the matrix of the transformed features.

[0064] Optionally, the pre-training loss function in the pre-training process of the graph neural network model is expressed as follows:

[0065]

[0066] Where: L pre represents the pre-training loss function, v q represents the query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, score(v1, v2) represents a score function used to measure the feature similarity between node v1 and node v2, and exp(·) represents an exponential function.

[0067] The beneficial effects of the present invention are as follows: a diffusion-based strategy is used to sample positive instance nodes similar to query nodes from graph data of protein structures, and in this way, robust node representations are generated for graph data, thereby alleviating the difference between pre-training and downstream tasks; and a graph convolution layer is used as a graph prompt to improve the effectiveness of the model in processing specific downstream tasks during training, and effectively capture the structure and node features in the graph data; the prompt layer is optimized based on the information bottleneck loss function, and the influence of irrelevant and noisy features in the previous process on generalization to downstream tasks is reduced by retaining the minimum sufficient statistics of the input data and discarding redundant information; when the graph data is classified by the obtained graph prompt fine-tuning model, the protein structure classification accuracy and classification efficiency are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 Schematic diagram of nodes for link prediction task in the pre-training phase.

[0069] Figure 2 Schematic diagram of the nodes for downstream classification tasks.

[0070] Figure 3 Schematic diagram of the process of the image prompt fine-tuning method based on diffusion pre-training in an embodiment of the present invention.

[0071] Figure 4 It is a process framework diagram of the graph prompt fine-tuning method based on diffusion pre-training in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0073] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0074] Figure 3 FIG. 1 is a flow chart of a method and system for fine-tuning graph hints based on diffusion pre-training in one embodiment. It should be understood that although Figure 3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 3 At least part of the steps in the above method may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps. Figure 3 As shown, the method and system for fine-tuning image prompts based on diffusion pre-training disclosed in the present invention specifically include the following steps:

[0075] In some embodiments, the graph data to be classified is classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified, wherein the graph data to be classified is graph data of protein structure, and the classification result includes a protein structure classification result.

[0076] S101. Obtaining unlabeled graph networks of multiple protein structures.

[0077] The graph network or graph data (hereinafter referred to as graph) can be defined as a tuple G = (V, A), where V = {v1, v2, ..., v N} represents the vertex set, N = |V| represents the number of vertices, and A = {0, 1} N×N is the adjacency matrix, which represents the connections between vertices, that is, the structure of the graph. i and v j Connected by A i,j =1 means, otherwise A i,j= 0. In addition, a property graph G = (V, A, X) has a node feature matrix where F represents the number of dimensions. Note that each node v i By vector Description, this is the i-th row in the matrix X. By preparing a large amount of unlabeled graph structure data, it can be used to help the model learn general features or domain-specific features of the graph.

[0078] In one embodiment, taking the ENZYMES dataset as an example, the dataset includes 600 molecules in 6 categories obtained from the BRENDA enzyme database: each graph network represents a protein molecule, and each node in the graph network represents a component unit (such as an amino acid) in the protein molecule. The component unit is a secondary structure element of the protein, and each node feature includes AA length, 3d length, van der Waals volume, polarity and polarizability, etc. Specifically, the dataset determines a total normalized van der Waals value for each node separately, and marks the total number of residues with low, medium and high normalized van der Waals volumes, which is called a 3-bin distribution. The length of each secondary structure element in the atomic energy spectrum is called the AA length, C α The distance between the first and last residues of an atom constitutes the 3D length. S102. Determine a query node in the unlabeled graph network and a pair of pre-trained samples corresponding to the query node using a preset graph diffusion convolution function.

[0079] In some embodiments, where the query node is a sequence and structural property of a protein structure, the choice of query node may affect the accuracy of the prediction for a protein structure. If the query node is an amino acid residue, then the chemical properties of the residue (such as charge, polarity, and size) and its position in the protein may affect its role in the three-dimensional structure. Similarly, if the query node is a larger protein fragment or an entire protein, then its sequence and structural properties (such as the distribution and sequence pattern of secondary structure elements) may affect the prediction results. Therefore, it is necessary to determine the protein structure to improve the prediction and classification effects.

[0080] In the pre-training stage, the goal is to narrow the gap between nodes in the same category and increase the distance between nodes in different categories. and the feature matrix When nodes are available but their labels are unknown, nodes with high similarity have the same label, while nodes with low similarity have different labels.

[0081] In one implementation manner, step S102 specifically includes the following steps:

[0082] Input the unlabeled graph network into the preset graph diffusion convolution function to obtain the diffusion matrix of the unlabeled graph network;

[0083] determining a query node in a diffusion matrix;

[0084] Select the maximum value of the row where the query node is located in the diffusion matrix as the positive node index, and obtain the pre-trained positive sample corresponding to the query node;

[0085] The structure-based sampling method uses the node with the lowest similarity in the diffusion matrix as the pre-trained negative sample corresponding to the query node;

[0086] The pre-training positive samples and pre-training negative samples are combined into pre-training sample pairs, where the query nodes are the sequence and structural characteristics of the protein structure.

[0087] In this embodiment, the step of inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain the diffusion matrix of the unlabeled graph network is as follows:

[0088] The unlabeled graph network is input into the preset graph diffusion convolution function, and the diffusion matrix of the unlabeled graph network is obtained by using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows:

[0089]

[0090] Where S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents the diffusion step size, α∈(0,1) represents the probability that a node in the unlabeled graph network transmits a message through its connection, and T represents the symmetric transfer matrix.

[0091] In order to strike a balance between sampling quality and spatiotemporal overhead, a diffusion-based strategy is used to sample positive node instances in the unlabeled case, which can capture long-range information in the graph to enrich the samples of the pre-training task. Formally, given a graph G, it is first input into a graph diffusion convolution function GDC(·) to obtain a diffusion matrix S. Each element s in the diffusion matrix S is ij Represents node v i and v j degree of proximity.

[0092] The formula of the symmetric transfer matrix is ​​as follows:

[0093] T=I N +D -1 / 2 AD 1 / 2

[0094] Where: I N represents the identity matrix of self-loops in the unlabeled graph network, A represents the adjacency matrix of the unlabeled graph network, and D represents the first degree matrix of the node, that is, The diffusion operation achieves linear running time O(N).

[0095] To obtain the query node v q Pre-training positive samples Select the index of the maximum value in the qth row of the diffusion matrix S as the pre-training positive sample index, and the formula is as follows:

[0096]

[0097] Where Sq,m represents the value of the mth element in the qth row of the diffusion matrix S.

[0098] For node v q Pre-training negative samples A structure-based sampling method can be used to add a self-loop to each node and prioritize the nodes that are related to v q Sampling in unconnected nodes, if v q is connected to all other nodes in the graph G, i.e. The nodes with the lowest similarity can be directly regarded as pre-training negative samples:

[0099]

[0100] Among them, score(v1, v2) represents the score function used to measure the feature similarity between node v1 and node v2.

[0101] S103. Input all unlabeled graph networks into the preset graph neural network model, and complete the pre-training of the graph neural network model by combining the query nodes and pre-trained samples.

[0102] Among them, the unlabeled graph network is input into the preset graph neural network model to obtain the transformed graph of the lth layer and the matrix of the transformed features. The elements in the matrix X′ are expressed as follows:

[0103]

[0104] UPDATE(·) represents a node feature updater, and AGGREGATE(·) is an aggregation function for neighbor information. In this embodiment, a sum function may be selected for aggregation.

[0105] Finally, by comparing the similarity between the query node and its positive and negative samples, the pre-training loss function is defined based on the matrix X′. The pre-training loss function in the pre-training process of the graph neural network model is expressed as follows:

[0106]

[0107] Where: L pre represents the pre-training loss function, v q represents a query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, score(v1, v2) represents the score function used to measure the feature similarity between node v1 and node v2, and exp(·) represents the exponential function.

[0108] S104. Obtain the task target graph network of the downstream classification task and the task specific label of the downstream classification task.

[0109] In some embodiments, wherein the downstream classification task is a protein structure classification task, task-specific labels are used to characterize specific protein structures that are assigned and predicted.

[0110] Among them, the downstream classification task is usually a graph classification task or a node classification task. For the graph classification task, there is a universal graph set G, which can be divided into two subsets. The first subset Gl is a set containing all labeled graph samples and all categories in G, and the other subset G u Contains the remaining unlabeled graph samples, that is, G = G l ∪G u and Use Y G To represent the label set of graph G. The goal of graph classification is to train a model that can accurately predict the set G u The labels of the remaining unlabeled samples in . In addition, for few-shot image classification, there are only K labeled samples for each category, which is called K-shot classification. It is formally expressed as where |Y G | represents the graph label set Y G size.

[0111] Similarly, the node classification task is performed on a separate graph G = (V, A, X), where the set of labeled nodes V l and the set of unlabeled nodes V u All of them are in the same graph G, that is, V = V l ∪V u and The node label set is Y v The goal of node classification is to train a model that can effectively predict the set V u Labels for unlabeled nodes.

[0112] S105. Input the task target graph network and the task feature labels into the pre-trained graph neural network model to obtain a first target graph network containing universal embedding.

[0113] S106. Constructing graph hints for downstream classification tasks based on graph convolutional layers.

[0114] In some embodiments, the graph provides node degree, clustering coefficient, path length, and centrality metrics for graph data of proteins.

[0115] In some embodiments, the node degree is represented in a protein structure graph, and the node degree may correspond to the coordination number of an amino acid residue (ie, the number of interactions it has with other residues). Highly coordinated residues may play a key role in the structure of the protein.

[0116] In some embodiments, the clustering coefficient is represented in a protein structure graph, and the clustering coefficient may reflect the degree of interaction between neighbors of an amino acid residue. Residues with a high clustering coefficient may exist in tightly packed regions of the protein.

[0117] In some embodiments, the path length is represented in a protein structure diagram, and the path length may correspond to the spatial distance or sequence distance between two amino acid residues. The information of the path length can help predict the three-dimensional structure of the protein.

[0118] In some embodiments, centrality indices are represented in a protein structure graph, and centrality indices (such as degree centrality, closeness centrality, and betweenness centrality) can help identify amino acid residues that play a key role in the protein structure.

[0119] Among them, due to the structural differences between graph data and text data, the original language prompt format cannot be directly adapted to graph prompts. Specifically, an obvious difference is that natural language text is mainly composed of one-dimensional, sequential structures, while graph data embodies a multi-dimensional, nonlinear network structure. From this perspective, trainable text prompts guide the language model to generate task-specific word vectors. Accordingly, a trainable graph convolutional layer h can be used φ (·) to prompt the graph model to generate task-specific graph vectors, which are expressed as follows:

[0120] h φ (G′)=σ(D′ -1 / 2 A′D′ 1 / 2 X′W)

[0121] Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, W represents the weight matrix of the graph hint. Through this message passing mechanism, h φ (G′) can use the structure of the graph to obtain the feature vector of the aggregated graph topology information.

[0122] S107. Delete a preset number of edges in the first target graph network, and use graph prompts to complete the prompting step for the first target graph network after the edges are deleted, so as to obtain a second target graph network containing task-specific embeddings.

[0123] S108. Complete the few-sample classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task during the few-sample classification process.

[0124] Among them, since the downstream classification task is usually a graph classification task or a node classification task. For the node classification task, the node set V of the graph G′ can be divided into a k-shot labeled set V l and an unlabeled set V u , that is, V = {V l , V u},in Pay attention to Y V is the label set of the node, whose size is |Y V | indicates. For node classification tasks, use The vectors in are used as node features, which are obtained by node-level specific hints In order to adapt to the small sample scenario, metric-based learning can be introduced to calculate the query node v q With class y v Prototype The distance between v ∈Y V Among them, the prototype is a virtual node used as a representation of the class level. The query node v q The label of is predicted to be the class closest to its prototype:

[0125]

[0126] where dist(·,·) describes the distance between two nodes. For convenience, we directly define dist(·,·)=1-sCore(·,·). To obtain class y v For the prototype embedding, we use class y v The average value of these label node representations in is used as its prototype embedding

[0127]

[0128] in Is Produced is a row vector in . Thus, the loss of the training node classification task can be defined as

[0129]

[0130] in is the query node v q The real prototype.

[0131] For graph classification tasks, a graph-level specific hint can be applied Its structure is the same as defined above The same. Therefore, the corresponding feature matrix can be obtained Then by using the entire matrix To obtain the query graph G q Embedding:

[0132]

[0133] Where V q It is a query graph The node set of express The corresponding node features in Readout(·) are an integration operation. Specifically, in the implementation mode, by It is realized by summing the representations of all nodes in Similar to the node classification task, the graph label y g The graph prototype embedding of can be obtained as follows:

[0134]

[0135] where Y G is the set of graph labels, and is marked as y g In addition, the loss of the graph classification task is It can be calculated as follows:

[0136]

[0137] in It is a query graph The real prototype.

[0138] S109. Compute the cosine similarity between the generic embedding and the task-specific embedding.

[0139] S110. Combining task loss and cosine similarity to calculate the information bottleneck of downstream classification tasks.

[0140] S111. Optimize graph prompts by minimizing information bottleneck feedback, and determine graph prompt fine-tuning models based on graph prompts.

[0141] In one embodiment, for steps S109 to S111, since a large number of noise features may be introduced during the prompting process, the image prompt cannot directly transmit the information of the pre-trained model. Therefore, it is possible to try to reduce the impact of irrelevant features by minimizing the information bottleneck (IB). The information bottleneck expression formula is as follows:

[0142]

[0143] in is the original input, Y represents the query label set, Z represents the embedding to be optimized, and β is a positive real number used to balance the two terms. In an embodiment, In the graph classification task, it refers to the graph set While in node-level tasks, it refers to a single graph G. I(·;·) represents the mutual information between two variables. The information bottleneck aims to learn the minimum sufficient representation for a given task by maximizing the mutual information between the embedding of the input graph and the target label. In addition, it also constrains the mutual information between the embedding of the input graph and the original features of the input graph.

[0144] Specifically, the first term of the information bottleneck expression formula represents the mutual information between the task-specific label and the graph embedding. According to the relationship between mutual information and entropy, I(Y; Z) can be expressed as:

[0145] I(Y; Z) = H(Y) - H(Y| Z),

[0146] Where H(Y) represents the entropy of Y. When Y is a single dataset, its entropy can be regarded as a fixed value, so we can focus on the entropy between task-specific labels and graph embeddings, that is, H(Y|Z). Formally, according to the definition of entropy, H(Y|Z) can be expressed as:

[0147]

[0148] For a classification task, p(z j |y i ) can be calculated by an indicator function, namely p(z j |y i )=1(z j ,y i ), where if y i Yes j The true label, then p(z j |y i )=1, otherwise p(z j |y i )=0. The number of available instances for each category is K, so p(y i ) is a uniform distribution. Therefore, the loss function That is, the loss of the training node classification task and the loss of the graph classification task, which refers to the first term I(Y; Z) of the information bottleneck expression formula.

[0149] For the second term of the information bottleneck expression formula, namely I(D;Z), it can be expressed in the desired form as follows:

[0150]

[0151] However, since p(Z) cannot be obtained, it is impractical to use the above formula directly for calculation. Therefore, an auxiliary distribution r(Z) can be introduced to separate p(Z) in order to construct a variational upper bound for the above formula:

[0152]

[0153] in is the constructed variational upper bound, and D KL (·||·) represents the KL divergence. Therefore, minimizing The process can be transformed into minimizing its variational upper bound. In detail, the pre-trained GNN is used to model the conditional distribution And r(Z) can be instantiated as any GNN model. In the implementation, in order to reduce the number of parameters, the layer h is hinted with a graph φ (·) Instantiate r(Z). In addition, to eliminate the dependence of r(Z) on the original input, a certain proportion of edges in the graph are randomly discarded before feeding the graph into the hint layer.

[0154] Note that to simplify the computation of the variational upper bound, it can be transformed into computing the cosine similarity between the embeddings obtained from the pre-trained GNN and the graph hint layer:

[0155]

[0156] where x′ represents the output of the pre-trained model, and Indicates the diagram prompt h φ (·) output. Therefore, the final downstream task loss is the information bottleneck, that is, the information bottleneck expression formula, which can be expressed as

[0157]

[0158] In one embodiment, an optimized second target graph network is determined by graph prompts, and a graph prompt fine-tuning model is determined by using the optimized second target graph network and a pre-trained GNN.

[0159] S201, obtaining graph data to be classified, and performing classification processing on the graph data to be classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified.

[0160] In one embodiment, the graph data to be classified may also be a BRENDA enzyme graph network, a protein-like graph network, a commodity recommendation network, etc.;

[0161] In one embodiment, graph data or graph network is a nonlinear data structure consisting of nodes (or vertices) and edges. Graph data structures can be used to represent network structures, such as social networks, the Internet, and transportation networks. In a graph, nodes usually represent entities, and edges represent relationships between entities.

[0162] In one embodiment, a total of 600 molecules in 6 categories obtained from the BRENDA enzyme database are taken as an example: each graph network represents a protein molecule, each node in the graph network represents a component unit in the protein molecule, and the component unit is a secondary structure element of the protein. Each node feature includes AA length, 3d length, van der Waals volume, polarity and polarizability, etc. Specifically, the data set determines a total normalized van der Waals value for each node separately, and marks the total number of residues with low, medium and high normalized van der Waals volumes, which is called a 3-bin distribution. The length of each secondary structure element in the atomic energy spectrum is called AA length, C α The distance between the first and last atoms of a residue constitutes the 3d length.

[0163] In one embodiment, the classification processing of the graph prompt fine-tuning model can obtain the node classification results of the graph data, the classification results of the relationship between nodes and the classification results of the graph data (graph network), exemplarily, the node classification results are such as the AA length, 3d length, van der Waals volume, polarity and polarizability of the above-mentioned protein structure; the classification results of the relationship between nodes, such as the length of each secondary structure element in the atomic energy spectrum is called AA length, C α The distance between the first and last atoms of a residue constitutes the 3d length.

[0164] Reference Figure 4 , Figure 4 This is a process framework diagram of the graph hint fine-tuning method based on diffusion pre-training, combined with Figure 4 The flowchart in can more clearly describe the operation process of the above steps S101 to S111.

[0165] The present invention also discloses a graph prompt fine-tuning system based on diffusion pre-training, and the system specifically comprises:

[0166] The sample graph acquisition module is used to obtain multiple unlabeled graph networks;

[0167] A sample pair selection module is used to determine the query node in the unlabeled graph network and the pre-trained sample pair corresponding to the query node by using a preset graph diffusion convolution function;

[0168] The pre-training module is used to input all unlabeled graph networks into the preset graph neural network model, and complete the pre-training of the graph neural network model by combining query nodes and pre-training sample pairs;

[0169] A downstream task acquisition module, used to obtain the task target graph network of the downstream classification task and the task specific labels of the downstream classification task;

[0170] A model application module, used to input the task target graph network and the task feature labels into the pre-trained graph neural network model to obtain a first target graph network containing a universal embedding;

[0171] A graph hint building module, which is used to build graph hints for downstream classification tasks based on graph convolutional layers;

[0172] A graph prompt application module, used for deleting a preset number of edges in the first target graph network, and using the graph prompt to complete the prompt step on the first target graph network after the edges are deleted, to obtain a second target graph network containing task-specific embeddings;

[0173] A loss calculation module, used to complete the few-sample classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task in the few-sample classification process;

[0174] A similarity calculation module, which is used to calculate the cosine similarity between the universal embedding and the task-specific embedding;

[0175] The information bottleneck calculation module is used to combine task loss and cosine similarity to calculate the information bottleneck of downstream classification tasks;

[0176] A graph hint fine-tuning module is used to optimize graph hints by minimizing information bottleneck feedback.

[0177] In one embodiment, the sample pair selection module includes:

[0178] A matrix acquisition unit, used for inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network;

[0179] a node selection unit, for determining a query node in the diffusion matrix;

[0180] A positive sample selection unit is used to select the maximum value of the row where the query node is located in the diffusion matrix as the positive node index, and obtain the pre-trained positive sample corresponding to the query node;

[0181] A negative sample selection unit, used for selecting the node with the lowest similarity in the diffusion matrix as a pre-trained negative sample corresponding to the query node based on a structure-based sampling method;

[0182] The sample combination unit is used to combine the pre-training positive samples and the pre-training negative samples into pre-training sample pairs.

[0183] In this embodiment, the matrix acquisition unit is specifically used to input the unlabeled graph network into a preset graph diffusion convolution function, and obtain the diffusion matrix of the unlabeled graph network using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows:

[0184]

[0185] Where: S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents the diffusion step size, α∈(0,1) represents the probability that a node in the unlabeled graph network transmits a message through its connection, and T represents the symmetric transfer matrix;

[0186] The formula of the symmetric transfer matrix is ​​as follows:

[0187] T=I N +D -1 / 2 AD 1 / 2

[0188] Where: I N represents the identity matrix of self-loops in the unlabeled graph network, A represents the adjacency matrix of the unlabeled graph network, and D represents the first degree matrix of the nodes.

[0189] In one implementation manner, the expression formula of the graph prompt constructed by the graph prompt construction module is as follows:

[0190] h φ (G′) = σ(D′ -1 / 2 A′D′ 1 / 2 X′W)

[0191] Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, and W represents the weight matrix of the graph prompt.

[0192] In one embodiment, the pre-training loss function in the pre-training process of the graph neural network model by the pre-training module is expressed as follows:

[0193]

[0194] Where: L pre represents the pre-training loss function, vq represents a query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, score(v1, v2) represents the score function used to measure the feature similarity between node v1 and node v2, and exp(·) represents the exponential function.

[0195] An embodiment of the present invention also includes a graph data identification module, wherein the graph data identification module is used to perform classification processing on the graph data to be classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified. It can be understood that the graph prompt fine-tuning model is obtained through the training method as described above.

[0196] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as above, which are not provided in detail for the sake of simplicity.

[0197] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A method for fine-tuning image prompts based on diffusion pre-training, characterized in that: The steps include: Acquire graph data to be classified, and classify the graph data to be classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified, wherein the graph data to be classified is graph data of a protein structure, and wherein the classification result includes a protein structure classification result; The figure shows that the training process of the fine-tuning model includes: Obtain unlabeled graph networks of multiple protein structures; Determine a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node using a preset graph diffusion convolution function, wherein the query node is a sequence and a structural characteristic of a protein structure; Inputting all of the unlabeled graph networks into a preset graph neural network model, and combining the query nodes and the pre-trained sample pairs to complete the pre-training of the graph neural network model; Obtaining a task target graph network for a downstream classification task and a task specific label for the downstream classification task, wherein the downstream classification task is a protein structure classification task, wherein the task specific label is used to characterize a specified and predicted specific protein structure; Inputting the task target graph network and the task feature labels into the pre-trained graph neural network model to obtain a first target graph network containing a universal embedding; Constructing a graph prompt for the downstream classification task based on the graph convolution layer, wherein the graph prompt is used for node degree, clustering coefficient, path length and centrality index of protein graph data; Deleting a preset number of edges in the first target graph network, and using the graph prompt to complete the prompt step on the first target graph network after the edges are deleted, to obtain a second target graph network containing task-specific embeddings; Complete the few-sample classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task in the few-sample classification process; computing a cosine similarity between the universal embedding and the task-specific embedding; Calculating the information bottleneck of the downstream classification task by combining the task loss and the cosine similarity; The graph hint is optimized by minimizing the information bottleneck feedback, and a graph hint fine-tuning model is determined according to the graph hint.

2. The method for fine-tuning image prompts based on diffusion pre-training according to claim 1, characterized in that: The method of using a preset graph diffusion convolution function to determine a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node comprises the following steps: Inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network; determining a query node in the diffusion matrix; Selecting the maximum value of the row where the query node is located in the diffusion matrix as a positive node index, and obtaining a pre-trained positive sample corresponding to the query node; The structure-based sampling method uses the node with the lowest similarity in the diffusion matrix as the pre-trained negative sample corresponding to the query node; The pre-training positive sample and the pre-training negative sample are combined into a pre-training sample pair.

3. The method for fine-tuning image prompts based on diffusion pre-training according to claim 2, characterized in that: The step of inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network comprises the following steps: The unlabeled graph network is input into a preset graph diffusion convolution function, and the diffusion matrix of the unlabeled graph network is obtained by using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows: Where: S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents the diffusion step, α∈(0,1) represents the probability that a node in the unlabeled graph network transmits a message through its connection, and T represents the symmetric transfer matrix; The formula of the symmetric transfer matrix is ​​as follows: T=I N +D -1 / 2 AD 1 / 2 Where: I N represents the identity matrix of self-loops in the unlabeled graph network, A represents the adjacency matrix of the unlabeled graph network, and D represents the first degree matrix of the node.

4. The method for fine-tuning image prompts based on diffusion pre-training according to claim 3, characterized in that: The expression formula of the graph is as follows: h φ (G′)=σ(D′ -1 / 2 A′D′ 1 / 2 X′W) Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, W represents the weight matrix of the graph prompt, and X′ is the matrix of the transformed features.

5. The method for fine-tuning image prompts based on diffusion pre-training according to claim 1, characterized in that: The pre-training loss function in the pre-training process of the graph neural network model is expressed as follows: Where: L pre represents the pre-training loss function, v q represents the query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, score(v1, v2) represents a score function used to measure the feature similarity between node v1 and node v2, and exp(·) represents an exponential function.

6. A graph prompt fine-tuning system based on diffusion pre-training, characterized in that: The system comprises: A graph data recognition module, which obtains graph data to be classified, and classifies the graph data to be classified using a graph prompt fine-tuning model to obtain a classification result of the graph data to be classified, wherein the graph data to be classified is graph data of a protein structure, and the classification result includes a protein structure classification result; a sample graph acquisition module, which is used to obtain a plurality of unlabeled graph networks of protein structures; a sample pair selection module, which is used to determine a query node in the unlabeled graph network and a pre-trained sample pair corresponding to the query node using a preset graph diffusion convolution function, wherein the query node is a sequence and structural characteristic of a protein structure; a pre-training module, which is used to input all the unlabeled graph networks into a preset graph neural network model, and complete the pre-training of the graph neural network model in combination with the query node and the pre-trained sample pair; A downstream task acquisition module, used to acquire a task target graph network of a downstream classification task and a task specific label of the downstream classification task, wherein the downstream classification task is a protein structure classification task, wherein the task specific label is used to characterize a specified and predicted specific protein structure; A model application module, used for inputting the task target graph network and the task feature labels into the pre-trained graph neural network model to obtain a first target graph network containing a universal embedding; A graph hint construction module, used for constructing graph hints for the downstream classification task based on the graph convolution layer, wherein the graph hints are used for node degree, clustering coefficient, path length and centrality index of graph data of proteins; A graph prompt application module, configured to delete a preset number of edges in the first target graph network, and use the graph prompt to complete a prompt step on the first target graph network after the edges are deleted, to obtain a second target graph network containing task-specific embeddings; A loss calculation module, used to complete the few-sample classification of the downstream classification task based on the second target graph network, and calculate the task loss of the downstream classification task in the few-sample classification process; A similarity calculation module, configured to calculate the cosine similarity between the universal embedding and the task-specific embedding; An information bottleneck calculation module, used to calculate the information bottleneck of the downstream classification task by combining the task loss and the cosine similarity; A graph hint fine-tuning module is used to optimize the graph hint by minimizing the information bottleneck feedback, and determine a graph hint fine-tuning model according to the graph hint.

7. The image prompt fine-tuning system based on diffusion pre-training according to claim 6, characterized in that: The sample pair selection module comprises: A matrix acquisition unit, used for inputting the unlabeled graph network into a preset graph diffusion convolution function to obtain a diffusion matrix of the unlabeled graph network; a node selection unit, configured to determine a query node in the diffusion matrix; A positive sample selection unit, configured to select the maximum value of the row where the query node is located in the diffusion matrix as a positive node index, and obtain a pre-trained positive sample corresponding to the query node; A negative sample selection unit, configured to select the node with the lowest similarity in the diffusion matrix as a pre-trained negative sample corresponding to the query node based on a structure-based sampling method; The sample combination unit is used to combine the pre-training positive sample and the pre-training negative sample into a pre-training sample pair.

8. The image prompt fine-tuning system based on diffusion pre-training according to claim 7, characterized in that: The matrix acquisition unit is specifically used to input the unlabeled graph network into a preset graph diffusion convolution function, and obtain the diffusion matrix of the unlabeled graph network using the diffusion algorithm in the graph diffusion convolution function. The diffusion algorithm formula is as follows: Where: S represents the diffusion matrix, GDC(·) represents the graph diffusion convolution function, G represents the unlabeled graph network, t represents the diffusion step, α∈(0,1) represents the probability that a node in the unlabeled graph network transmits a message through its connection, and T represents the symmetric transfer matrix; The formula of the symmetric transfer matrix is ​​as follows: T=I N +D -1 / 2 AD 1 / 2 Where: I N represents the identity matrix of self-loops in the unlabeled graph network, A represents the adjacency matrix of the unlabeled graph network, and D represents the first degree matrix of the node.

9. The image prompt fine-tuning system based on diffusion pre-training according to claim 8, characterized in that: The expression formula of the graph is as follows: h φ (G′)=σ(D′ -1 / 2 A′D′ 1 / 2 X′W) Where: h φ (·) represents the graph prompt, G′ represents the first target graph network, σ(·) represents the activation function, A′=A+I N , D′ represents the second degree matrix of the node, W represents the weight matrix of the graph prompt, and X′ is the matrix of the transformed features.

10. The image prompt fine-tuning system based on diffusion pre-training according to claim 6, characterized in that: The pre-training loss function in the pre-training process of the graph neural network model is expressed as follows: Where: L pre represents the pre-training loss function, v q represents the query node, represents the pre-training positive sample in the pre-training sample pair, represents the pre-trained negative sample in the pre-trained sample pair, score(v1, v2) represents a score function used to measure the feature similarity between node v1 and node v2, and exp(·) represents an exponential function.

Citation Information

Patent Citations

  • Critical protein identification method based on NCCO (Neighbor Closeness Centrality and Orthology) information

    CN106372458A

  • Protein interaction relationship prediction method based on label guidance

    CN115206423A

  • Distance enhancement-based hypergraph learning dynamic node classification method and system

    CN116821769A

  • Three-dimensional virtual fitting method and system based on custom diffusion model

    CN118096980A

  • Automated Variational Inference using Stochastic Models with Irregular Beliefs

    US20230419075A1