A graph deep model adaptive parameter adjustment method for graph classification

Abstract

The application discloses a kind of graph depth model adaptive parameter adjustment methods for graph classification, comprising the following steps: step 1: extracting the global structure feature of input sample graph i;Step 2: extracting the local structure feature of input sample graph i, and the global structure feature and the local structure feature are integrated to form structure feature vector S (i);Step 3: selecting shared GNN model;Step 4: structure feature vector S (i) is converted into modulation information by graph modulator;Step 5: given a graph sample, the parameter θ (k) of shared GNN model is adjusted by the structure information modulation factor φ (k) generated by graph modulator, and the shared GNN model after modulation parameter is called customized GNN model;Step 6: parameter training: the parameters of graph modulator and shared GNN model are trained, and cross-entropy loss is selected as loss function.The method of the application dynamically selects or adjusts model parameters, improves the accuracy of classification and the applicable scope of model.

Description

Technical Field

[0001] This invention belongs to the field of computer graph neural network technology, and specifically relates to an adaptive parameter adjustment method for graph depth models for graph classification. Background Technology

[0002] In today's rapidly evolving technological landscape, the problem of out-of-distribution (OOD) scenarios in the real world is becoming increasingly prominent. How to handle instances not encountered during the training phase has become a focal point of research. Recent studies show that deep neural networks are highly sensitive to changes in data distribution; even small differences in distribution between training and test data can lead to a significant decline in model predictive performance. Therefore, exploring methods that can maintain generalization ability under different data distributions is particularly important. Although significant progress has been made in research on Euclidean data (such as images and text), insufficient attention has been paid to non-Euclidean data, especially graph data.

[0003] Graph data is ubiquitous in real life, forming the core of many fields, from microscopic biomolecular structures and particle interactions to macroscopic transportation networks and social networks. The distribution shift phenomenon of these graph data structures is frequently observed in practical applications. For example, in the citation network of academic literature, the topic distribution of papers evolves over time; in social networks, the relationship network between users changes with personal experiences and social changes. Given the widespread application of graph data in various fields and the prevalence of its distribution shift, researching how to improve the generalization ability of graph neural networks in the face of distribution changes is of significant theoretical and practical importance for promoting the development of artificial intelligence technology. Therefore, more research investment and innovative methods are urgently needed to address the out-of-distribution generalization problem of non-Euclidean data.

[0004] Graph Neural Networks (GNNs) have demonstrated significant performance in graph representation learning, achieving excellent results in various graph-related tasks such as node classification, graph classification, graph node clustering, and link prediction. As an innovative application of deep learning technology in the field of graph data, GNNs effectively learn deep representations of graphs by fusing node attributes and graph structural information. Considering the central role of GNNs in processing graph-related tasks, improving their generalization ability and robustness is crucial, as it is of great significance for addressing the out-of-distribution generalization challenge prevalent in real-world graph data. Therefore, in-depth research on how to enhance the generalization and robustness of GNN models to solve the generalization problem of graph data under unknown distributions is an important research direction in the field of graph representation learning.

[0005] Graph classification is a fundamental and widely applied task in graph analysis, with broad real-world applications. For example, proteins can be abstracted as graph structures, and determining whether a protein has enzymatic activity can be considered a graph classification problem. Furthermore, graph classification can be used to predict Alzheimer's disease, where an individual's brain is modeled as graph data. In reality, even within the same dataset, the structural characteristics of graphs can vary significantly. For instance, the D&D dataset, widely used for protein structure prediction and classification, exhibits a wide range of node counts in its protein graphs, ranging from 30 to 5488. Randomly selecting two sample graphs from this dataset will reveal drastically different structural features, including the number of nodes, graph shape, and diameter. These differences underscore the necessity of considering the structural features of input samples in graph classification tasks.

[0006] Currently, numerous studies have addressed the out-of-distribution (OOD) problem in graph data. While these studies have proposed various techniques, most are heuristic-based and require extensive experimentation to identify effective data augmentation techniques. Besides data augmentation, some researchers have explored adversarial training and invariance principles to improve model generalization. However, these methods all follow the paradigm of "one model fits all data," while in graph data, the structural features of input samples have a decisive impact on the classification performance of GNNs. When data distribution shifts, these methods cannot adjust or customize model parameters based on the specific structural features of the input samples, leading to a decline in predictive performance. Traditional methods typically use all graphs in the dataset to train a uniform GNN model, then apply this model to all unknown graphs in the test set for label prediction. However, even within the same dataset, graph structures can vary significantly, meaning that a uniform model may not be optimal when dealing with a single graph. Adjusting and customizing model parameters based on the structural features of the input samples is crucial for solving the out-of-distribution problem in graph classification tasks. This requires designing models that can adaptively adjust parameters based on the structural characteristics of the graph to improve classification performance under different data distributions. Summary of the Invention

[0007] The technical problem to be solved by this invention is to provide an adaptive parameter adjustment method for graph depth models for graph classification.

[0008] The technical solution adopted by this invention to solve the above-mentioned technical problems is: an adaptive parameter adjustment method for graph depth models for graph classification, comprising the following steps:

[0009] Step 1: Given an input sample image i, extract the global structural features of the input sample image i;

[0010] Step 2: Extract the local structural features of the input sample image i, and integrate the global structural features and local structural features to form the structural feature vector S(i);

[0011] Step 3: Select a shared GNN model;

[0012] Step 4: The structural feature vector S(i) is transformed into modulation information through a graph modulator. The input of the graph modulator is the structural feature vector S(i), and the output is a set of structural information modulation factors φ(k). These structural information modulation factors φ(k) are used to adjust the weights or biases or other trainable parameters of the shared GNN model.

[0013] Step 5: Shared GNN model parameter modulation: Given a graph sample, the parameters θ(k) of the shared GNN model are adjusted by the structural information modulation factor φ(k) generated by the graph modulator. The shared GNN model after parameter modulation is called a customized GNN model.

[0014] Step 6: Parameter Training: Train the parameters of the graph modulator and the shared GNN model, choosing cross-entropy loss as the loss function, whose mathematical expression is: ,in, It's a real label. It predicts labels; a customized GNN model outputs predicted labels. .

[0015] Preferably, in step 1, global structural features are extracted using statistical methods. These global structural features include, but are not limited to, the number of nodes, the number of edges, node distances, graph diameter, average degree, and average clustering coefficient. Node distance refers to the minimum value of all paths between two nodes, graph diameter refers to the maximum value of the distance between any two nodes in the graph, average degree refers to the average value of the degrees of all nodes, and the clustering coefficient of a node refers to the ratio between the number of edges between its neighbors and the maximum possible value.

[0016] Preferably, in step 2, local structural features are extracted using graph theory algorithms. These local structural features include, but are not limited to, paths, cycles, and cliques. A K-path is a sequence containing K+1 non-repeating adjacent nodes. A K-cycle is a sequence containing K non-repeating adjacent nodes, but with the first and last nodes adjacent. A K-clique is a graph in which every two nodes are connected by an edge.

[0017] Preferably, in step 3, the shared GNN model is selected from any of the following models: GCN model, GAT model, GraphSAGE model, DiffPool model, and GPool model.

[0018] Preferably, in step 4, the graph modulator generates customized structural information modulation factors based on the structural characteristics of each sample graph. The structural information modulation factors are a set of weight adjustment factors used to scale or offset the weight matrices of each layer in the shared GNN model.

[0019] Preferably, in step 5, the shared GNN model for graph classification includes an aggregation layer and a pooling layer; the aggregation layer receives the graph structure and node representation as input and generates the aggregated node representation as output; the pooling layer receives the graph structure and node representation as input to generate a new graph and node representation.

[0020] When an aggregation layer or pooling layer is introduced, the adjacency matrix A∈Rn×n and the node representation X∈Rn×d are used to represent the input of these layers, where n is the number of nodes and d is the dimension of the node features;

[0021] Assumption These are the adapter parameters corresponding to this aggregation layer. Adjust the parameters as follows: : ;in This represents the parameters in the aggregation layer; These are the parameters for customizing the GNN model, and ο is the adaptation operator;

[0022] The adjusted aggregation layer is as follows: ;in, This represents the node representation generated by the aggregation layer, with a dimension of dnew.

[0023] Preferably, in step 6, the gradient of the loss function is calculated through backpropagation, and the parameters of the graph modulator and the shared GNN model are adjusted using the gradient descent method.

[0024] Preferably, in step 6, the Adam optimizer is used to select the optimal hyperparameters of the graph modulator and the shared GNN model.

[0025] The beneficial effects of this invention are as follows: This invention provides an adaptive parameter adjustment method for graph depth models in graph classification. The model parameters are adaptively adjusted based on the structural features of the input samples. This means that the model can dynamically select or adjust its parameters according to the structural information of each input sample, such as the total number of nodes, the total number of edges, homogeneity, and graph diameter, thereby improving classification accuracy and the model's applicability. The method of this invention is applicable to datasets of various sizes and distribution characteristics and is compatible with the skeletons of various basic graph neural network models. This method can function in different application scenarios and model architectures, thus improving its universality and practicality in real-world applications. Attached Figure Description

[0026] Figure 1This is a schematic diagram of the principle framework of steps 1-3 in the method of the present invention;

[0027] Figure 2 This is a schematic diagram of the principle framework of steps 4-5 in the method of the present invention;

[0028] Figure 3 This is a schematic diagram of the principle framework of step 6 in the method of the present invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and preferred embodiments. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0030] An adaptive parameter adjustment method for graph depth models for graph classification includes the following steps:

[0031] like Figure 1 As shown, step 1: Given an input sample image i, extract the global structural features of the input sample image i;

[0032] like Figure 1 As shown, step 2: extract the local structural features of the input sample image i, and integrate the global structural features and local structural features to form the structural feature vector S(i).

[0033] The structural information of a graph can be divided into global structural features and local structural features. Specifically, in an optional embodiment, in step 1, global structural features are extracted using statistical methods. Global structural features include, but are not limited to: number of nodes, number of edges, node distance, graph diameter, average degree, and average clustering coefficient. Node distance refers to the minimum value of all paths between two nodes; graph diameter refers to the maximum value of the distance between any two nodes in the graph; average degree refers to the average value of the degrees of all nodes; and the clustering coefficient of a node refers to the ratio between the number of edges between its neighbors and the possible maximum value.

[0034] Specifically, in one optional embodiment, in step 2, local structural features are extracted using graph theory algorithms. These local structural features include, but are not limited to, paths, cycles, and cliques. A K-path is a sequence containing K+1 non-repeating adjacent nodes. A K-cycle is a sequence containing K non-repeating adjacent nodes, but with the first and last nodes adjacent. A K-clique is a graph in which every two nodes are connected by an edge.

[0035] In graph classification tasks, the extracted global and local structural features are crucial for accurately distinguishing different types of graphs. These features capture high-level information and patterns in the graph, reflecting its structural characteristics and topological relationships, and providing the model with key information to differentiate between different graph categories. For example, the community structure in a social network may exhibit unique characteristics in its clustering coefficient, while protein-protein interaction networks may show significant differences in the frequency of occurrence of specific substructures.

[0036] The construction of the structural feature vector S(i) is crucial for subsequent graph-level classification tasks. It transforms the structural information of the graph into a numerical vector, providing a concise and efficient way for machine learning models to process and classify graph data. The introduction of the structural feature vector S(i) can improve the model's classification performance and enhance its robustness to changes in graph structure.

[0037] like Figure 1 As shown, step 3: Select a shared GNN model. Specifically, in an optional embodiment, in step 3, the shared GNN model is selected from any of the following models: GCN model, GAT model, GraphSAGE model, DiffPool model, and GPool model.

[0038] For medium-sized graphs with a relatively simple structure and a moderate number of nodes, the Graph Convolutional Network (GCN) model is a good choice, effectively handling homogeneous graphs. When the number of nodes is large and the relationships between them are complex, the Graph Attention Network (GAT) model is a better choice. By assigning different weights to neighboring nodes using an attention mechanism, the GAT model can capture complex relationships between nodes. For large-scale graphs with a large number of nodes, the GraphSAGE model is a good choice. It reduces computation by sampling and aggregating features of neighboring nodes, making it suitable for scenarios with a very large number of nodes. For graphs with highly variable structures or sizes, the DiffPool model is an effective hierarchical graph neural network. It learns a soft assignment matrix to cluster nodes into different subgraphs and learns a hierarchical embedding for these subgraphs. The GPool model is a graph neural network based on graph structure pooling. It reduces the graph size by selecting representative nodes and is suitable for tasks requiring graph size reduction for classification. In practice, a shared GNN model is selected based on the structural characteristics of the dataset. The shared GNN model is selected based on the global and local structural features of the extracted samples. Factors such as the number of nodes, the complexity of the graph structure, and the size variation of the graph are taken into account. The shared GNN model suitable for the data features is selected to improve the classification performance and training efficiency of the model and to better handle out-of-distribution graph-level classification tasks.

[0039] like Figure 2 As shown, step 4: The structural feature vector S(i) is converted into modulation information by a graph modulator. The input of the graph modulator is the structural feature vector S(i), and the output is a set of structural information modulation factors φ(k). These structural information modulation factors φ(k) will be used to adjust the weights or biases or other trainable parameters of the shared GNN model.

[0040] Specifically, in one optional embodiment, in step 4, the graph modulator generates a customized structural information modulation factor based on the structural characteristics of each sample graph. The structural information modulation factor is a set of weight adjustment factors used to scale or offset the weight matrix of each layer in the shared GNN model.

[0041] Graph Neural Networks (GNNs) typically consist of multiple consecutive aggregation and pooling layers, which can be viewed as different GNN blocks. The graph structure properties of a given sample may have different effects on different blocks, and each GNN block requires modulation parameters. In a GNN with K blocks (i.e., a GNN model with K layers), there are K layers of graph modulators corresponding to the K blocks. These graph modulators share the same input S(i), but their outputs are different.

[0042] After obtaining the structural feature vector S(i) of the input sample, the next step is to transform the structural feature vector into modulation information using a graph modulator. The purpose is to adjust the parameters of the shared GNN model so that it can better adapt to the structural characteristics of the sample graph, thereby improving classification accuracy and generalization ability. The graph modulator can employ various machine learning techniques, such as multilayer perceptrons, recurrent neural networks, or convolutional neural networks. These networks can learn the complex mapping relationship between the structural feature vector S(i) and modulation factors. The input of the graph modulator is the structural feature vector S(i), and the output is a set of structural information modulation factors φ(k) as modulation information. These factors can be scalars, vectors, or matrices, depending on the specific architecture of the shared GNN model. After receiving the transmitted structural feature vector S(i), the graph modulator generates a structural information modulation factor φ(k), which is used to change the corresponding matrix multiplication and bias term weights of each layer in the shared GNN model. If the shared GNN model includes an attention mechanism, the structural information modulation factor φ(k) can be used to adjust the attention weights, affecting the contribution of different nodes or edges in graph representation learning. For some shared GNN models, such as the DiffPool model or the GPool model, the structural information modulation factor φ(k) can be used to adjust the parameters of the topology, such as the number of clusters, the learning rate of the soft assignment matrix, the window size and stride of the pooling layer, etc.

[0043] The graph modulator generates modulation information based on the structural characteristics of each sample graph, enabling the shared GNN model to adaptively adjust its parameters to better handle input samples. This adaptive adjustment not only improves the model's classification performance but also enhances its robustness to changes in graph structure.

[0044] like Figure 2 As shown, step 5: Modulation of shared GNN model parameters: Given a graph sample, the parameters θ(k) of the shared GNN model are adjusted by the structural information modulation factor φ(k) generated by the graph modulator. φ(k) represents the structural information modulation factor generated by the kth graph modulator, and θ(k) represents the parameters of the kth layer of the shared GNN model. The shared GNN model after parameter modulation is called a customized GNN model.

[0045] Specifically, in one optional embodiment, in step 5, the shared GNN model for graph classification includes an aggregation layer and a pooling layer; the aggregation layer receives the graph structure and node representation as input and generates the aggregated node representation as output; the pooling layer receives the graph structure and node representation as input to generate a new graph and node representation.

[0046] When an aggregation layer or pooling layer is introduced, the adjacency matrix A∈Rn×n and the node representation X∈Rn×d are used to represent the input of these layers, where n is the number of nodes and d is the dimension of the node features;

[0047] Assumption These are the adapter parameters corresponding to this aggregation layer. Adjust the parameters as follows: : ;in This represents the parameters in the aggregation layer; These are the parameters for customizing the GNN model, and ο is the adaptation operator;

[0048] The adjusted aggregation layer is as follows: ;in, This represents the node representation generated by the aggregation layer, with a dimension of dnew.

[0049] like Figure 3 As shown, step 6: Parameter training: Train the parameters of the graph modulator and the shared GNN model, choosing cross-entropy loss as the loss function, whose mathematical expression is: ,in, It's a real label. It predicts labels; a customized GNN model outputs predicted labels. .

[0050] Specifically, in one optional embodiment, the training of the graph modulator is typically performed synchronously with the training of the shared GNN model. The parameters of the graph modulator and the shared GNN model are jointly optimized using backpropagation and gradient descent. This end-to-end training strategy ensures that the modulation information effectively guides the parameter adjustment of the shared GNN model, thereby achieving adaptive classification for each sample graph. Specifically, in one optional embodiment, this method employs an end-to-end training strategy where the training of the graph modulator and the shared GNN model is performed synchronously. This training strategy ensures that the modulation information of the graph modulator effectively guides the parameter adjustment of the shared GNN model.

[0051] Cross-entropy loss measures the difference between the true and predicted labels, and its goal is to find a set of parameters that minimizes the value of the loss function. Specifically, in one optional embodiment, in step 6, the gradient of the loss function is calculated via backpropagation, and the parameters of the graph modulator and the shared GNN model are adjusted using gradient descent. Gradient descent and backpropagation are common strategies in graph neural network training. Gradient descent iteratively adjusts parameters to minimize the loss function (i.e., the cross-entropy mentioned above). Backpropagation is a method for calculating the gradient of the loss function used to update weights and biases in a multi-layer neural network. It calculates the gradient layer by layer, starting from the output layer and working up to the input layer, enabling the network to learn data features and improve prediction accuracy.

[0052] Specifically, in one optional embodiment, in step 6, the Adam optimizer is used to select the optimal hyperparameters of the graph modulator and the shared GNN model. Hyperparameters such as the learning rate and regularization coefficients are determined using a combination of optimizers, such as the Adam Optimizer.

[0053] To address the Out-of-Depth (OOD) problem in graph classification tasks, this invention provides an adaptive parameter adjustment method for graph depth models in graph classification, focusing on solving the following issues:

[0054] The first issue is the adaptive adjustment of parameters. In graph classification tasks, different graph structures may require different parameter settings for optimal performance. Using a uniform model to classify all input samples may not be the best choice. To address this problem, the method of this invention aims to adaptively adjust model parameters based on the structural features of the input samples. This means that the model will be able to dynamically select or adjust model parameters based on the structural information of each input sample, such as the total number of nodes, the total number of edges, homogeneity, and graph diameter, thereby improving classification accuracy and the applicability of the model.

[0055] Secondly, there is the issue of high availability. The method of this invention is applicable to datasets of various sizes and distribution characteristics, and is compatible with the skeletons of various basic graph neural network models. This method can play a role in different application scenarios and model architectures, thereby improving its universality and practicality in practical applications.

[0056] The above description is only a specific embodiment of the present invention. Various examples and illustrations do not constitute a limitation on the substantive content of the present invention. Those skilled in the art can make modifications or variations to the above-described specific embodiments after reading the specification without departing from the substance and scope of the invention.

Claims

1. An adaptive parameter adjustment method for graph depth models for graph classification, characterized in that: Includes the following steps: Step 1: Given an input sample image i, extract the global structural features of the input sample image i; Step 2: Extract the local structural features of the input sample image i, and integrate the global structural features and local structural features to form the structural feature vector S(i); Step 3: Select a shared GNN model; Step 4: The structural feature vector S(i) is transformed into modulation information through a graph modulator. The input of the graph modulator is the structural feature vector S(i), and the output is a set of structural information modulation factors φ(k). These structural information modulation factors φ(k) are used to adjust the weights or biases or other trainable parameters of the shared GNN model. Step 5: Shared GNN model parameter modulation: Given a graph sample, the parameters θ(k) of the shared GNN model are adjusted by the structural information modulation factor φ(k) generated by the graph modulator. The shared GNN model after parameter modulation is called a customized GNN model. Step 6: Parameter Training: Train the parameters of the graph modulator and the shared GNN model, choosing cross-entropy loss as the loss function, whose mathematical expression is: L=-∑y i *log(y′ i ), where y i It's a real label, y′ i It predicts the label; the customized GNN model outputs the predicted label y′. i .

2. The adaptive parameter adjustment method for graph depth model for graph classification according to claim 1, characterized in that: In step 1, global structural features are extracted using statistical methods. These global structural features include, but are not limited to, the number of nodes, the number of edges, node distances, graph diameter, average degree, and average clustering coefficient. Node distance refers to the minimum value of all paths between two nodes. Graph diameter refers to the maximum value of the distance between any two nodes in the graph. Average degree refers to the average value of the degrees of all nodes. The clustering coefficient of a node refers to the ratio between the number of edges between its neighbors and the maximum possible value.

3. The adaptive parameter adjustment method for graph depth model for graph classification according to claim 1, characterized in that: In step 2, local structural features are extracted using graph theory algorithms. These local structural features include, but are not limited to, paths, cycles, and cliques. A K-path is a sequence containing K+1 non-repeating adjacent nodes. A K-cycle is a sequence containing K non-repeating adjacent nodes, but with the first and last nodes adjacent. A K-clique is a graph in which every two nodes are connected by an edge.

4. The adaptive parameter adjustment method for graph depth model for graph classification according to claim 1, characterized in that: In step 3, the shared GNN model is selected from any of the following models: GCN model, GAT model, GraphSAGE model, DiffPool model, and GPool model.

5. The adaptive parameter adjustment method for graph depth model for graph classification according to claim 1, characterized in that: In step 4, the graph modulator generates customized structural information modulation factors based on the structural characteristics of each sample graph. The structural information modulation factors are a set of weight adjustment factors used to scale or offset the weight matrix of each layer in the shared GNN model.

6. The adaptive parameter adjustment method for graph depth model for graph classification according to claim 1, characterized in that: In step 5, the shared GNN model for graph classification includes an aggregation layer and a pooling layer; the aggregation layer receives the graph structure and node representation as input and generates the aggregated node representation as output; the pooling layer receives the graph structure and node representation as input to generate a new graph and node representation. When an aggregation layer or pooling layer is introduced, the adjacency matrix A∈Rn×n and the node representation X∈Rn×d are used to represent the input of these layers, where n is the number of nodes and d is the dimension of the node features; Assume φ f These are the adapter parameters corresponding to this aggregation layer. Adjust parameter θ as follows: f : Where θ f θ represents the parameters in the aggregation layer. mf These are the parameters for customizing the GNN model. It is an adaptor operator; The adjusted aggregation layer is as follows: Among them, X new This represents the node representation generated by the aggregation layer, with a dimension of dnew.

7. The adaptive parameter adjustment method for graph depth model for graph classification according to claim 1, characterized in that: In step 6, the gradient of the loss function is calculated through backpropagation, and the parameters of the graph modulator and the shared GNN model are adjusted using the gradient descent method.

8. The adaptive parameter adjustment method for graph depth model for graph classification according to claim 1, characterized in that: In step 6, the Adam optimizer is used to select the optimal hyperparameters for the graph modulator and the shared GNN model.

Images