Graph neural network method based on graph adaptive mutual exclusion decomposition
By performing mutual exclusion decomposition on the graph structure, splitting the node neighborhood, and using the mutual exclusion sub-adjacency matrix to obtain differential node feature representations, the problems of poor node discriminability and neglect of relational information in traditional graph convolution models are solved, and richer graph structure information utilization and node feature representation are achieved.
Patent Information
- Application Number
- CN202310057972.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-13
- Publication Date
- 2026-02-03
- Estimated Expiration
- 2043-01-13
AI Technical Summary
When increasing the number of layers in a traditional graph convolutional model, the neighbor information aggregated by nodes almost covers the entire graph, resulting in poor node distinguishability and oversmoothing. At the same time, it ignores the entangled relationship information behind the graph structure.
By designing a method for mutually exclusive decomposition of the original adjacency matrix, the neighborhood of each node is split, different types of edge information hidden in a single edge are decomposed and mixed, and several sets of node feature representations with differences are obtained by using mutually exclusive sub-adjacency matrices. Finally, these representations are spliced together to enrich the information of the original graph structure.
It alleviates the oversmoothing problem of deep graph convolutional models, obtains more differentiated node feature representations, makes full use of the original graph structure information, and improves the discriminativeness and diversity of nodes.
Smart Images
Figure CN116258179B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of graph learning, and particularly relates to a graph neural network method based on graph adaptive mutual exclusion decomposition. BACKGROUND
[0002] In recent years, artificial intelligence has gradually emerged, and some technologies have been applied to production and life. Among them, deep learning has received extensive attention. Although traditional deep learning methods have made great progress and achieved great success in the representation learning of features of Euclidean space data, non-Euclidean space data exists in many scenarios, such as social networks and recommendation systems. Traditional deep learning methods cannot well process irregular non-Euclidean space data. The graph convolutional neural network ingeniously designs a representation learning method for feature extraction of graph data, which is of great significance for the research of the graph convolutional neural network.
[0003] The graph convolution model is a feature extractor designed for graph data. The essence of the traditional graph convolution model is to aggregate neighbor information by weighted summation of the neighbor nodes of the symmetric normalized adjacency matrix to obtain new node representation. However, with the increase of the number of graph convolution layers, the aggregation radius of each node, i.e. the order of the highest neighbor node, also rises, and rises to a certain value, and the neighbors connected by the node cover almost all the nodes in the graph. This will make the representation vector of each node tend to be consistent, and the distinguishability of the node becomes worse and worse, and the diversity of the node features is lost. In addition, the graph structure often mixes different types of relationships as an edge, and most existing graph convolution operations do not pay attention to the multiplicity between node relationships.
[0004] In summary, the existing graph convolution model has the following problems:
[0005] (1) When the number of model layers of the traditional graph convolution model is increased, the neighbor information aggregated by the node almost covers the whole graph, resulting in poor node distinguishability and over-smoothing problem.
[0006] (2) The existing graph convolution operation is directly defined on the original graph structure and node features, ignoring the relationship information hidden behind the graph structure. SUMMARY
[0007] Invention purposes: The purpose of the present application is to solve the problems existing in the prior art, provide a graph neural network method based on graph adaptive mutual exclusion decomposition, the present application starts from two aspects of decomposing the information behind the entanglement of the graph structure and relieving overfitting, designs a method for mutual exclusion decomposition of the original adjacency matrix, aims to split the neighborhood of each node, decomposes the different types of edge information hidden in the single side, and reduces the convergence of node representation to the nodes of the whole graph caused by the growth of model layers, simultaneously obtains several groups of node feature representations with differences by using the mutually exclusive sub-adjacency matrix, and finally splices them to enrich the information obtained from the original graph structure.
[0008] Technical scheme: The present application is a graph neural network method based on graph adaptive mutual exclusion decomposition, comprising the following steps:
[0009] Step (1), input the data to be classified, and pre-process the adjacency matrix and node features therein;
[0010] Step (2), construct a learnable edge feature, and decompose the original adjacency matrix according to the features by using a decomposition module, and the specific process is as follows:
[0011] Step (2.1), use a feedforward neural network to compress the pre-processed node features to 8 dimensions to improve the efficiency of the model,
[0012] Step (2.2), set the number K of sub-adjacency matrices generated by model decomposition, splice the node features to construct a 2*8-dimensional edge feature, and use a feedforward neural network to represent the compressed node feature as K-dimensional, to obtain the learned K-dimensional edge feature, at this time, the obtained low-dimensional edge feature is equal to the set number K of sub-adjacency matrices;
[0013] Step (2.3), the K-dimensional edge feature is one-to-one corresponding to the K sub-adjacency matrices, and it is judged which dimension in the edge feature has the maximum value, and the dimension with the maximum value is given to the corresponding sub-adjacency matrix as the standard; and the value of the edge in the sub-adjacency matrix is consistent with the original adjacency matrix, thereby obtaining K sub-adjacency matrices for mutual exclusion decomposition of the original adjacency matrix;
[0014] Step (3), for the sub-adjacency matrices obtained in step (2.3) and the node features generated by the first compression in step (2.3), a graph convolution module is used to obtain K groups of independent node features with differences according to the mutually exclusive sub-adjacency matrices;
[0015] Step (4), the K groups of independent node features obtained in step (3) are fused by an aggregation module; then the fused node features are used for node classification, and the final classification result is obtained through multiple iteration training.
[0016] Further, the step (1) preprocessing refers to converting the original adjacency matrix into a symmetric matrix, and normalizing the symmetric processed adjacency matrix and node features to maintain the original distribution of the features.
[0017] Further, the method for obtaining K-dimensional edge features in step (2.2) is shown in formula (1):
[0018] He ij =f(Wh i ||Wh j ) Formula (1)
[0019] In formula (1), h i represents the node feature of the i-th node, W represents the weight parameter of the compressed node feature, || represents the splicing function, f(·) represents the feedforward neural network, He ij represents the K-dimensional edge feature constructed by the node features of the two connected nodes i and j; the feedforward neural network is updated during the training process, and the number of decompositions to the sub-adjacency matrix is obtained by learning during the training process;
[0020] The method for obtaining the sub-adjacency matrix in step (2.3) is shown in formula (2):
[0021] D n =max(He n ) Formula (2)
[0022] In formula (2), max(·) represents taking the maximum value, D n represents the sub-adjacency matrix serial number corresponding to the n-th edge;
[0023] Thus, K sub-adjacency matrices for mutually exclusive decomposition of the original adjacency matrix are obtained;
[0024] This step can alleviate the problem of over-smoothing caused by the multi-layer graph convolution model to a certain extent by performing mutually exclusive decomposition on the non-Euclidean graph structure, compared with directly using the original graph structure for node information propagation, and can obtain more differentiated node representation and better utilize the original graph structure information.
[0025] Further, the specific method for the graph convolution module to generate K groups of independent node features in step (3) is:
[0026] K independent GCN branches are used to combine K mutually exclusive sub-adjacency matrices for differential information propagation, and a dropout layer with a rate of 0.6 is added after each GCN layer.
[0027] The number of hidden layer nodes in the above GCN intermediate layer is set to 16, and the dimension of the output node feature of the last layer is equal to the number of classifications.
[0028] Different sub-adjacency matrices are input into independent GCN layers combined with node features, so that the mutual exclusive graph structure information is fully utilized, and richer node feature expression is obtained.
[0029] Further, the aggregation module in the step (4) obtains fused node features by summing functions on the K groups of independent node features Where c is the number of classifications, and the summing function is defined as follows:
[0030]
[0031] The fused node features are subjected to softmax to obtain the final classification result.
[0032] The early-stop method is used in the iterative training process, the maximum number of iterations is 10000, the learning rate of the model is set to 0.005; the validation set result is used as the stopping standard, when the validation set loss does not decrease within 100 epochs, the model stops training, the model with the minimum validation set loss is saved for model testing; the model is trained and tested in the manner of 10 groups of random division of training sets, and finally the average precision obtained by 10 times of training is used as the classification result.
[0033] The node feature representations maintaining diversity are fused together to obtain the classification result, the node information is enriched, and the effect of one plus one is greater than two is achieved.
[0034] Beneficial effects: the original adjacency matrix is used as the basis, the original graph structure information is decomposed in a mutual exclusive manner, the over-smoothing problem of the graph convolution model is alleviated to a certain extent, the information behind the graph structure is separated by decoupling the graph structure, and the node feature representation with difference is obtained. Meanwhile, the various node feature representations generated by the mutual exclusive sub-adjacency matrix are aggregated, and the original graph structure information is fully utilized. Compared with the prior art, the present application has the following advantages:
[0035] (1) The decomposition module of the present application is a novel learnable decomposition module, which can perform mutual exclusive decomposition on the original adjacency matrix through learning.
[0036] (2) The graph convolution module of the present application uses K independent GCN layers, and can obtain independent and distinguishable node feature representations by combining the mutual exclusive sub-adjacency matrix.
[0037] (3) The aggregation module of the present application fuses different groups of node feature representations, so that the original graph structure information is fully utilized. DETAILED DESCRIPTION
[0038] Figure 1This is a flowchart illustrating the overall classification process of the present invention;
[0039] Figure 2 This is a schematic diagram of the mutual exclusion decomposition and convolution fusion of the present invention. Detailed Implementation
[0040] The technical solution of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the embodiments described.
[0041] like Figure 1 As shown, the graph neural network method based on graph adaptive mutual exclusion decomposition of the present invention includes the following steps:
[0042] Step (1): Input the data to be classified and preprocess the adjacency matrix and node features.
[0043] Step (2): Construct learnable edge features, and perform mutual exclusion decomposition on the original adjacency matrix based on the features using the decomposition module. The specific process is as follows:
[0044] Step (2.1): Compress the preprocessed node features to 8 dimensions using a feedforward neural network;
[0045] Step (2.2): Set the number K of sub-adjacency matrices generated by model decomposition, concatenate node features to construct 2*8 dimensional edge features, and use a feedforward neural network to represent the compressed node features as K-dimensional, obtaining the learned K-dimensional edge features. At this time, the obtained low-dimensional edge features are equal to the set number K of sub-adjacency matrices. By reducing the number of neighbors during convolution, the problem of poor node discriminability and oversmoothing caused by the aggregation of neighbor information of nodes almost covering the entire graph when the number of model layers of the existing graph convolution model is increased can be solved.
[0046] Step (2.3): Assign K edge features to K sub-adjacency matrices one by one, and determine which dimension of the edge features has the largest value. Use the dimension with the largest value as the standard for assigning the edge to the corresponding sub-adjacency matrix. The edge values in the sub-adjacency matrices are consistent with those in the original adjacency matrix. This yields K sub-adjacency matrices that perform mutually exclusive decomposition on the original adjacency matrix.
[0047] Step (3): For the sub-adjacency matrix obtained in step (2) and the node features generated by the first compression, the graph convolution module generates K sets of independent node features with differences obtained from the mutually exclusive sub-adjacency matrix.
[0048] Step (4): The K sets of independent node features obtained in step (3) are fused by the aggregation module; then the fused node features are used for node classification, and the final classification result is obtained through multiple iterations of training.
[0049] This invention considers fusing discriminative node feature representations together to obtain classification results, and then training on the result obtained by fusing all information, thus fully utilizing the original graph structure information. By decomposing the entangled information behind the graph structure, the decomposed sub-adjacency matrices yield more diverse, richer, and more differentiated node feature representations compared to the original structure information. Before node fusion, nodes using sub-adjacency matrices have fewer neighboring nodes than those using the original adjacency matrix. Therefore, the node features generated by the convolution of each sub-adjacency matrix converge to the entire graph more slowly, preserving differentiated information even after fusion. This alleviates the oversmoothing problem of deep graph convolutions to some extent. Furthermore, the model sharpens the fused node feature sets using softmax to ensure data dispersion.
[0050] Example 1:
[0051] This embodiment includes the following steps:
[0052] Step (1): First, use NumPy to process the input data, symmetricize the adjacency matrix, add an identity matrix to enhance the features of the nodes, and finally normalize the symmetric adjacency matrix and feature matrix.
[0053] Step (2): Construct learnable edge features, and perform mutual exclusion decomposition on the original adjacency matrix based on the features, specifically:
[0054] First, the preprocessed node features are compressed to 8 dimensions;
[0055] Then, set the number of sub-adjacency matrices generated by the model decomposition to K, concatenate the node features to construct 16-dimensional edge features, and use a feedforward neural network to represent the compressed node features as K-dimensional, thus obtaining the learned K-dimensional edge features. Here, the learned low-dimensional edge features are equal to the set number of sub-adjacency matrices K.
[0056] Next, the K-dimensional edge features are mapped one-to-one with the K sub-adjacency matrices, and it is determined which dimension of the edge features has the largest value. This dimension is used as the criterion for assigning the edge to its corresponding sub-adjacency matrix. Simultaneously, the edge weights in the sub-adjacency matrices remain consistent with those in the original adjacency matrix. This yields K mutually exclusive sub-adjacency matrices derived from the original adjacency matrix, such as... Figure 2 As shown.
[0057] Step (3): Generate K sets of independent node features with differences obtained from the mutual exclusion adjacency matrix through the graph convolution module, specifically:
[0058] K independent GCN branches are used, with each intermediate GCN layer containing 16 hidden nodes. Differential information propagation is achieved using K mutually exclusive sub-adjacency matrices, and a dropout layer with a rate of 0.6 is appended to each GCN layer. Finally, a last GCN layer compresses the K sets of independent node feature representations to the number of categories.
[0059] Step (4): The K sets of independent node features obtained in step (3) are fed into the summation function through the aggregation module for fusion, and then the final classification result is obtained through softmax. Then the fused node features are used for node classification, and the final classification result is obtained through multiple iterations of training.
[0060] The summation function is defined as follows:
[0061]
[0062] c represents the number of categories, h i For the fused node features
[0063] The classification results in this embodiment include seven categories: case-based, genetic algorithm, neural network, probabilistic method, reinforcement learning, rule learning, and theory. The model classifies machine learning papers in the dataset into different fields.
[0064] In this embodiment, early-stop iteration is used during iterative training, with a maximum of 10,000 iterations and a learning rate of 0.005. The validation set result is used as the stopping criterion. When the validation set loss no longer decreases within 100 epochs, the model training stops, and the model with the smallest validation set loss is saved for model testing. The model is trained and tested using a method of generating training sets of 10 random partitions. Finally, the average accuracy obtained from 10 training iterations is used as the classification result.
[0065] As can be seen from the above embodiments, the present invention, by performing mutually exclusive decomposition on the original graph structure information, forcibly preserves the differences between subgraphs. By decoupling the graph structure, it separates the entangled information behind the graph structure, obtains differentiated node feature representations, and alleviates the oversmoothing problem of graph convolution models to some extent. Simultaneously, it aggregates the diverse node feature representations generated by the mutually exclusive sub-adjacency matrix, ensuring full utilization of the original graph structure information. In contrast, the decomposition module in existing technologies requires iterative calculations to generate decoupled subgraphs, necessitating additional calculations of the loss function region subgraph.
Claims
1. A graph neural network method based on graph adaptive mutually exclusive decomposition, characterized in that: Includes the following steps: Step (1): Input the paper data to be classified and preprocess the adjacency matrix and node features. Step (2): Construct learnable edge features. The preprocessed adjacency matrix is decomposed into mutually exclusive features using the decomposition module. The specific process is as follows: Step (2.1): Compress the preprocessed node features to 8 dimensions; Step (2.2): Set the number of sub-adjacency matrices generated by the model decomposition. The node features are concatenated to construct 2*8 dimensional edge features, and a feedforward neural network is used to represent the compressed node features as follows: Dimension, after learning Low-dimensional edge features, where the resulting low-dimensional edge features are related to the number of sub-adjacency matrices set. equal; Step (2.3), will Dimensional features and Each sub-adjacency matrix is mapped one-to-one, and the dimension with the largest value in the edge feature is determined. This dimension is then used as the criterion for assigning the edge to its corresponding sub-adjacency matrix. Furthermore, the edge values in the sub-adjacency matrices remain consistent with those in the original adjacency matrix. This process yields... A sub-adjacency matrix that performs mutual exclusion decomposition on the original adjacency matrix; Step (3): For the sub-adjacency matrix obtained in step (2.3) and the node features generated by the first compression in step (2.3), the graph convolution module is used to generate differential features obtained from the mutually exclusive sub-adjacency matrix. Groups of independent node features; graph convolution module generates The specific method for identifying independent node features is as follows: use A separate GCN branch, combined Differential information propagation is performed using mutually exclusive sub-adjacency matrices, and a dropout layer with a rate of 0.6 is added after each GCN layer; The number of hidden nodes in the intermediate layers of the GCN above is set to 16, and the feature dimension of the output nodes in the last layer is equal to the number of categories; Step (4): The data obtained in step (3) is processed through the aggregation module. The independent node features are fused together; then the fused node features are used for node classification. The final classification result is obtained through multiple iterations of training. The classification results of the papers to be classified include seven categories: case-based, genetic algorithm, neural network, probabilistic method, reinforcement learning, rule learning, and theory.
2. The graph neural network method based on graph adaptive mutual exclusion decomposition according to claim 1, characterized in that: The preprocessing step (1) refers to converting the original adjacency matrix into a symmetric matrix and normalizing the symmetric adjacency matrix and node features.
3. The graph neural network method based on graph adaptive mutual exclusion decomposition according to claim 1, characterized in that: The step (2.2) obtains The method for defining edge features is shown in formula (1): Equation (1) In equation (1), Indicates the first The node characteristics of each node. Weight parameters representing the features of compressed nodes. Represents the concatenation function. This represents a feedforward neural network. Two nodes representing a connection and Constructed from node features Dimensional edge features; The method for obtaining the sub-adjacency matrix in step (2.3) is shown in formula (2): Equation (2) In equation (2), This represents taking the maximum value. Indicates the first The sub-adjacency matrix index corresponding to the edge feature.
4. The graph neural network method based on graph adaptive mutual exclusion decomposition according to claim 1, characterized in that: In step (4), the aggregation module will The fused node features are obtained by summing the independent node features using a summation function. ,in The summation function is defined as follows to calculate the number of categories: Equation (3) The fused node features are then processed using softmax to obtain the final classification result.
5. The graph neural network method based on graph adaptive mutual exclusion decomposition according to claim 1, characterized in that: In step (4), the iterative training process uses an early-stop method for iteration, with a maximum number of iterations of 10,000 and a learning rate of 0.
005. The validation set result is used as the stopping criterion. When the validation set loss no longer decreases within 100 epochs, the model stops training and the model with the smallest validation set loss is saved for model testing. The model is trained and tested by generating training sets in 10 random partitions, and the average accuracy obtained from 10 training iterations is used as the classification result.