Relation node mining method based on adaptive information propagation and aggregation hypergraph model
Through the adaptive hypergraph neural network combined with dual-view information propagation and decoupling convolution, the limitations of hypergraph model in mining paired node relationships and high-hop nearest neighbor information extraction are solved, and efficient and accurate hypergraph node classification is achieved, which significantly improves the Model performance.
Patent Information
- Application Number
- CN202411092435.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-15
AI Technical Summary
The existing hypergraph model has limitations in mining potential paired node relationships and extracting high-hop nearest neighbor information, resulting in information loss and noise, affecting model expression capabilities and performance.
Adaptive hypergraph neural network (AdaHGNN), including a dual-view information propagation mechanism, decoupled hypergraph convolution and an adaptive aggregation method based on the improvement strategy, uses graph view and hypergraph view to simplify hypergraph Convolution and adaptively aggregate high-hop nearest neighbor information.
Effectively capture high-order node relationships and explore potential paired node relationships, improve the accuracy and efficiency of hypergraph node classification, significantly improve the performance of the model, especially in the accuracy of standard node classification data, which reaches 11%.
Smart Images

Figure CN118965115A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of node classification, and in particular to a node classification method for adaptive information propagation and aggregation of hypergraph models. Background Art
[0002] In the past few years, graph convolutional networks (GCNs) have gained widespread attention due to their outstanding performance in real-world applications such as text classification, image recognition, and anomaly detection. Most of these methods capture pairwise relationships between nodes in a graph. However, in many real-world scenarios, the relationships between nodes are not only binary (pairwise), but may also be ternary, quaternary, or higher order. Therefore, reducing these high-order relationships to pairwise relationships may lead to a loss of information and hinder the model's expressiveness.
[0003] To overcome this limitation, the hypergraph structure was proposed and achieved significant improvements. Hypergraphs allow each hyperedge to connect multiple nodes (usually 3 or more nodes) at the same time to represent and model group relationships (high-order node relationships) between nodes. To further process hypergraph structured data, the hypergraph neural network (HGNN) was proposed, which includes two key steps: information propagation and information aggregation. In the information propagation step, HGNN uses the "node-hyperedge-node" mechanism (i.e., hypergraph information propagation mechanism (hypergraph IPM)) to interact information between multiple nodes in the same hyperedge, thereby capturing high-order node relationships. In the information aggregation step, HGNN aggregates the information of neighboring nodes to learn a comprehensive representation of the target node. These two steps enable HGNN to show excellent performance in the node classification task of hypergraph data and promote the development of various related hypergraph models.
[0004] Although hypergraph models have achieved certain results in the early stage, there is still significant room for improvement. Among them, their main limitation is that hypergraph information propagation mechanisms (hypergraph IPMs) may not be sufficient to mine potential pairwise relationships in real-world hypergraph data. For example, in a hypergraph-based social network, users A, B, and C in the same group are connected by a hyperedge and establish an explicit high-order relationship. However, users A and B may have a close relationship in this group, but there is no hyperedge to accurately describe this. However, this potential pairwise relationship may be valuable for node classification tasks. In previous hypergraph IPMs, the information of nodes with potential pairwise relationships (such as A and B) needs to be propagated to the hyperedge together with other nodes in the same hyperedge (such as C). After the hyperedge-level group interaction, the information is then propagated back to each node to capture the high-order node relationship. This interaction does not take into account the differences between paired nodes and treats all paired nodes equally, which makes it more difficult to discover and mine potential pairwise node relationships. Therefore, how to effectively mine potential pairwise node relationships in hypergraph data and effectively integrate them with explicit high-order node relationships remains to be explored.
[0005] In addition, due to the over-smoothing problem, most hypergraph models have limited ability to extract information from high-hop neighbors (i.e., nodes that are multiple hops away from the target node, where the hop count between two nodes is defined as the number of hyperedges in the shortest path connecting them). This limitation restricts the expressiveness and performance of hypergraph models. Therefore, some studies have proposed to use skip connections to facilitate the extraction and aggregation of information from high-hop neighbors, thereby enhancing hypergraph models. However, the following issues remain unresolved:
[0006] 1) Extracting information from high-hop neighbors requires multiple rounds of information propagation, feature transformation, and activation mapping, which incurs significant computational overhead.
[0007] 2) Not all information from high-hop neighbors contributes positively to the classification task. Directly aggregating this information may introduce noise and degrade model performance.
[0008] Therefore, how to effectively extract and aggregate valuable information from high-hop neighbors remains to be explored. Summary of the invention
[0009] To solve the above problems, the present invention discloses a node classification method for adaptive information propagation and aggregation of hypergraph models to overcome the limitations in mining potential pairwise node relationships and extracting and aggregating information from high-hop neighbors.
[0010] A node classification method for adaptive information propagation and aggregation of hypergraph models, the node classification method for adaptive information propagation and aggregation of hypergraph models is based on a new adaptive hypergraph neural network (AdaHGNN), including a dual-view information propagation mechanism (IPM), a decoupled hypergraph convolution (DHC) and an adaptive aggregation method based on a lifting strategy, the node classification method for adaptive information propagation and aggregation of hypergraph models includes:
[0011] A dual-view information propagation mechanism (IPM) is used to process the input hypergraph data, where the graph view IPM is used to mine potential pairwise node relationships, and the hypergraph view IPM is used to capture explicit high-order node relationships. The above node representations are fused using a learnable matrix and combined with the original node features to obtain updated node features.
[0012] The traditional hypergraph convolution is simplified to decoupled hypergraph convolution (DHC) by removing the activation function and reparameterizing the learnable parameters to extract information from high-hop neighbors.
[0013] Multiple DHCs are trained using a boosting strategy to adaptively aggregate valuable information from high-hop neighbors for classification according to their information weights.
[0014] Beneficial effects of the present invention:
[0015] 1. A new type of adaptive hypergraph neural network is proposed, which can effectively capture high-order node relationships, mine potential pairwise node relationships, and efficiently and adaptively utilize the beneficial information of the target node's high-hop neighbors to achieve accurate hypergraph node classification.
[0016] 2. A new dual-view IPM is proposed, which can not only capture the high-order node relationships that are explicitly present in the hypergraph data, but also progressively and dynamically mine the potential pairwise relationships in the nodes and adaptively fuse the mined information.
[0017] 3. A new DHC with adaptive aggregation method is proposed to efficiently extract information from high-hop neighbors and aggregate them in an adaptive manner. Experiments show that the proposed model has significant superiority in accuracy, and on standard node classification data, it has achieved an 11% accuracy improvement compared to other methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 Schematic diagram of the adaptive hypergraph neural network structure for this application.
[0019] Figure 2 This is a flow chart of the node classification method for adaptive information propagation and aggregation of hypergraph models in this application. DETAILED DESCRIPTION
[0020] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.
[0021] Figure 1 The adaptive hypergraph neural network constructed in this application is demonstrated, including an information propagation module, a node feature acquisition and update module, and an information aggregation module.
[0022] Among them, the information propagation module uses the graph view IPM to mine potential pairwise node relationships and the hypergraph view IPM to capture explicit high-order node relationships. Through these two processes, the dual-view node representation is obtained. In the acquisition and update node feature module, these representations are fused using a learnable matrix and combined with the original node features to obtain updated node features. The information aggregation module applies multiple DHCs to extract valuable information from high-hop neighbors on the updated node features, and uses the boosting strategy to adaptively aggregate this information and make the final prediction based on the information weight.
[0023] In one embodiment, Figure 2 As shown in the figure, a node classification method for adaptive information propagation and aggregation of hypergraph models is proposed, including:
[0024] Step S1: After inputting the hypergraph data, use the graph view IPM to mine potential pairwise node relationships, and use the hypergraph view IPM to capture explicit high-order node relationships. Through these two processes, a dual-view node representation is obtained. An example of the lth dual-view IPM is shown below.
[0025] First, this embodiment uses the IPM of the hypergraph neural network (HGNN) as the hypergraph view IPM to capture high-order node relationships, as shown below: represents a high-order node representation, represents the normalized hypergraph association matrix, X (l-1) represents the output of the network (l-1) layer, X (0) is the original feature of the node.
[0026]
[0027] Secondly, a graph view IPM is designed to mine pairwise node relationships. To this end, the original hypergraph H is first converted to a graph G by star expansion. This method can effectively establish explicit pairwise node relationships by adding edges between representative nodes within each hyperedge. m , according to the node characteristics and Select a pair of nodes (v k ,v p )∈N(e m ), where i and j represent node indices, N(e m ) represents the hyperedge e m The node indexes k and p are determined by formula (2), where SumPooling is used to compress the feature vector into a scalar value. Then, weighted edges are added between each pair of selected nodes to construct the graph To establish explicit pairwise node relationships, represented as its weighted adjacency matrix and Based on this graph, GCN’s IPM is applied to capture pairwise node relationships, as shown in formula (3), where Represents a pairwise node representation.
[0028]
[0029] Step S2: Use a learnable matrix to fuse the above representations and combine them with the original node features to obtain updated node features, as shown in formula (4) and formula (5). Here, X (l) represents the updated node feature matrix, μ represents the element coefficient matrix used to weigh the importance of the two view information. The symbol || represents the connection operation, and ⊙ represents the element-wise product. is a learnable matrix, d represents the dimension of the node feature, and f() represents the dual view representation to be connected Linear transformation that projects to the appropriate dimension.
[0030]
[0031] This embodiment adopts the gating mechanism because it can adaptively weigh the importance of high-order node relationships captured by the hypergraph view and pairwise node relationships mined by the graph view in information fusion. By fusing the information of these two views, high-order and pairwise node representations can complement each other and fully capture the complex node relationships in the hypergraph data.
[0032] Step S3: Simplify the traditional hypergraph convolution into a new decoupled hypergraph convolution (DHC) by removing the activation function and reparameterizing the learnable parameters to extract information from high-hop neighbors.
[0033] Existing hypergraph convolutions inherit certain complexities due to their deep learning pedigree, and have become a burden to extract information from high-hop neighbors. In previous hypergraph models, in order to extract information from the l-hop neighbors of the target node, l information propagations, feature transformations, and activation mappings are required. However, this process may result in significant computational overhead. Therefore, this embodiment simplifies the traditional hypergraph convolution into a new DHC. Starting from DHC using hypergraph view IPM, a version combined with dual-view IPM is then derived as shown below.
[0034] First, the nonlinear activation function σ(·) between the traditional hypergraph convolutional layers is removed. In addition, this embodiment adds initial node features after each information propagation so that the nodes are still distinguishable after multiple iterations, as shown below.
[0035]
[0036] Then, the learnable parameters before the output layer are reparameterized into a single matrix, P (hidden) =(P (0) P (1) …P (l-1) ).
[0037]
[0038] Therefore, the output of the graph convolutional network is approximated as a two-layer fully connected neural network classifier The input is And Y (l) is the information extracted from the l-hop neighbors as follows.
[0039]
[0040] This embodiment effectively extracts information from neighbors by removing the activation function and re-parameterizing the learnable parameters. Finally, dual-view IPM is used Replace SuperGraph IPM To enhance the information extraction process, the formula of DHC combined with dual-view IPM for extracting information from l-hop neighbors can be written as follows.
[0041]
[0042] Step S4: Use a boosting strategy to train multiple DHCs, and adaptively aggregate valuable information from high-hop neighbors for classification according to their information weights.
[0043] At present, a practical method for extracting information from high-hop neighbors has been obtained based on DHC. However, not all information from high-hop neighbors is beneficial for classification. Therefore, an adaptive aggregation method is customized according to the boosting strategy to aggregate the valuable information extracted by multiple DHCs.
[0044] In a specific embodiment, multiple base classifiers are used to extract information, and the extracted information is subsequently aggregated for final prediction. This process can adaptively filter out noise and highlight valuable information by iteratively updating information weights. Here, multiple DHCs are used as base classifiers, and different base classifiers are used to extract information from different hop neighbors. For example, the lth base classifier is used to extract information from the lth hop neighbor. These base classifiers are trained in sequence. In order to train any number of DHCs using an adaptive aggregation method, this embodiment provides an example of training the lth base classifier.
[0045] In the training phase, we first minimize the loss function Loss (l) To train the lth base classifier. Then, the weighted error rate err of the trained base classifier is calculated in sequence. (l) and information weight α (l) , as shown in formulas (10)-(12). Here, n represents the number of training samples, represents the indicator function, y i is the true label of the i-th training sample, and C represents the number of label categories. It refers to the probability prediction of the lth base classifier for the i-th training sample. It is through The predicted labels are obtained by applying the Argmax operation. In addition, represents the weight value of the ith training sample when training the lth base classifier, which can be calculated based on the prediction of the previous base classifier. This weight is used to encourage the current base classifier to pay attention to the samples that were previously misclassified. In the first base classifier, each training sample has the same weight, that is,
[0046]
[0047] After the lth base classifier is trained, the weights are updated using formula (13): and renormalized using formula (14). This updating and renormalization step also ensures that will not become zero in a finite number of iterations.
[0048]
[0049] Finally, when all base classifiers have been trained, they can be classified according to the information weight α(l) , the final prediction result Y is obtained by aggregating the information extracted by the base classifier (out) This operation can filter out bad information from high-hop neighbors and highlight valuable information. In addition, compared with the original boosting method, this embodiment additionally multiplies the weighted accuracy score on the validation set in the decision aggregation step. To ensure the generalization performance of the model, as shown below.
[0050]
[0051] In order to comprehensively evaluate the node classification ability of AdaHGNN, this application conducted experiments on four real-world hypergraph datasets. The information of the experimental datasets is shown in Table 1:
[0052] Table 1
[0053]
[0054] First, as can be seen from Table 1, the Cora-CA dataset contains a total of 2708 scientific publications and 1072 authors. Publications and their corresponding authors are represented by nodes and hyperedges, respectively. Publications involving the same author are connected by hyperedges. The abstract of each publication is encoded by a One-Hot encoder to form a 1433-dimensional node feature. The publications are divided into seven categories, and 20 publications are used as training data for each category. Second, the Cora-CC dataset contains the same nodes, node features, and categories as Cora-CA. Cora-CC contains 1579 hyperedges, which exceeds Cora-CA. Citations of publications are represented by hyperedges. Publications that cite the same paper are connected. Third, the Citseer-CC dataset includes 3312 scientific publications and 1079 citations. Publications and their corresponding citations are represented by nodes and hyperedges, respectively. The abstract of each publication is encoded by a One-Hot encoder to form a 3703-dimensional node feature. The publications are divided into six categories, and 23 publications are used as training data for each category. Finally, the Pubmed-CC dataset contains 19,717 scientific publications and 7,963 citations. Publications and corresponding citations are represented by nodes and hyperedges, respectively. Each publication is described by term frequency-inverse document frequency (TF-IDF), forming a 500-dimensional node feature. The publications are divided into three categories, each with 53 publications, which are used as training data.
[0055] AdaHGNN is implemented in Pytorch and optimized using the Adam optimizer. The experiment trained and tested AdaHGNN on a 32G Tesla-V100 GPU. For the hyperparameter settings and network structure, AdaHGNN uses 300 hidden units and 10 layers. Here, the 10-layer AdaHGNN means that 10 DHCs are used as base classifiers in the training. In addition, a 5×10 -3 L2 regularization, 1×10 -3 Weight decay and 500 validation sets were used. 500 training epochs were set for all models with early stopping at a patience value of 150 epochs. Each experiment was run with 20 data splits and 20 runs were performed for each experiment. The experiments used accuracy as the evaluation metric and calculated the standard deviation.
[0056] The experiment compares the performance of AdaHGNN with the following baseline models: FastHGCN, HyperGCN, HGCN, UniGCNII, Allsettransformer, PhenomNN, UniG-Encoder. Among the baseline models, the results of FastHGCN and HyperGCN are directly cited from the original papers. The codes of UniGCNII, PhenomNN, and UniG-Encoder are taken from the official website. Allsettransformer is implemented according to the settings of the official library. In addition, the experiment modifies the dataset division of these implementations to follow the experimental settings of this paper, while the rest remains unchanged.
[0057] The experimental results are shown in Table 2:
[0058] Table 2
[0059]
[0060]
[0061] Table 3 lists the performance of different AdaHGNN variants:
[0062] Table 3
[0063]
[0064] Table 3 shows the average accuracy (%) results of these variants on the four datasets with different training data ratios. It is obvious that excluding dual-view IPM, DHC and adaptive aggregation methods will reduce the performance of the model in all cases, which proves the effectiveness of the proposed method. It is worth noting that on the Citeseer-CC dataset, when the labeled data ratio is 4.2%, the performance gap between AdaHGNN excluding dual-view IPM and the complete AdaHGNN is significantly larger than that of other datasets. A possible explanation is that each hyperedge of the Citeseer-CC dataset contains more nodes than other datasets, which leads to more complex node relationships within each hyperedge. In this case, the ability of dual-view IPM to capture high-order information and mine pairwise node relationships enables it to provide more significant performance improvements. In addition, when the labeled data ratio is halved (e.g., from 5.2% to 2.6%, 4.2% to 2.1%, and 0.8% to 0.4%), the performance gap between Cora-CA, Cora-CC, and Citeseer-CC excluding dual-view IPM and AdaHGNN decreases. The potential reason is that the reduction of available information causes the dual-view IPM in AdaHGNN to be unable to fully explore the potential pairwise node relationships.
[0065] In addition, the experiment also observed that the performance of AdaHGNN dropped significantly when DHC was removed. The possible reason is that too many network parameters lead to overfitting. This phenomenon further shows that DHC plays a key role in the effectiveness of the model. In addition, when the labeled data ratio decreases, the average accuracy gap between the variants that do not include aggregation and AdaHGNN increases in most cases. These results prove that when aggregation is not included, the high-hop neighbor information is not fully utilized as in AdaHGNN, resulting in a decrease in model performance. In addition, it can be seen that when compared with Tables 2 and 3, the variants that do not include dual-view IPM and aggregation outperform most baseline models. These results show the robustness of the proposed model. Comparison with the baseline model of the variant that does not include dual-view IPM and aggregation also shows that in most cases, effectively utilizing the information of high-hop neighbors can bring greater performance improvements than only capturing high-hop node relationships or only mining pairwise node relationships.
[0066] In this application, we propose the AdaHGNN model that achieves excellent performance in node classification tasks. First, a novel dual-view IPM method is proposed to adaptively capture and mine high-order node relationships and pairwise node relationships in hypergraph data. In addition, a new decoupled hypergraph convolution is proposed to efficiently extract information from high-hop neighbors. Finally, a new adaptive aggregation method is designed to effectively filter out noise and flexibly utilize valuable information from high-hop neighbors to achieve accurate classification.
[0067] Extensive experiments demonstrate that AdaHGNN outperforms comparison models in terms of accuracy and efficiency on four standard hypergraph datasets, showing great promise for real-world node classification applications.
[0068] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above technical features.
Claims
1. A method for mining relational nodes based on adaptive information propagation and aggregation hypergraph model, It is characterized in that Based on the newly constructed adaptive hypergraph neural network, adaptive information propagation and aggregation of relationship node mining are realized, wherein the adaptive hypergraph neural network includes an information propagation module, an acquisition and update node feature module and an information aggregation module; specifically, the following steps are included: S1: After inputting the hypergraph data, use the graph view IPM to mine the potential pairwise node relationships in the hypergraph data, and use the hypergraph view IPM to capture the explicit high-order node relationships. Through these two processes, the dual-view node representation is obtained; S2: Use a learnable gating matrix to learn the weights of different view information, and fuse the dual-view representations according to the weights, so as to adaptively filter out noise information and highlight useful information; the fused dual-view representation will be added to the original node features as a new node feature; S3: By removing the activation function and compressing the model parameters, the classic hypergraph convolution is extended to a new type of decoupled hypergraph convolution (DHC), which has the ability to quickly extract the high-hop neighbor information of the target node; S4: Use the boosting strategy to train multiple DHCs, adaptively learn the information weights of different high-hop neighbors of the target node, and aggregate the rich information of the target node according to the weights for classification.
2. According to claim 1, a method for mining relationship nodes based on adaptive information propagation and aggregation hypergraph model, Features: The S1 specifically includes the following steps: S11: The IPM of Hypergraph Neural Network (HGNN) is adopted as the hypergraph view IPM to capture high-order node relationships, as shown below: in represents a node representation that captures high-level information, represents the normalized hypergraph association evidence, X (l-1) represents the output of the (l-1)th layer of the network, X (0) is the original node feature; S12: Design a graph view IPM to mine pairwise node relationships; transform the original hypergraph H into graph G by star expansion; for each hyperedge e m , according to the node characteristics and Select a pair of nodes (v k ,v p )∈N(e m ), where k and p represent node indices; node indices k and p Determined by formula (2), N(e m ) represents the hyperedge e m A collection of nodes; Among them, SumPooling is used to compress the feature vector into a scalar value; S13: Add weighted edges between each pair of selected nodes to build the graph To establish explicit pairwise node relationships, represented as its weighted adjacency matrix and Based on this graph, GCN’s IPM is applied to capture pairwise node relationships, as shown in formula (3): in Captures pairwise representations of nodes.
3. According to claim 1, a method for mining relationship nodes based on adaptive information propagation and aggregation hypergraph model, Features: By fusing the information of these two views, the high-order and paired node representations of S2 can complement each other and fully capture the complex node relationships in the hypergraph data; as shown in formula (4) and formula (5); HereX (l) represents the updated node feature matrix, μ represents the element coefficient matrix used to weigh the importance of the two view information; the symbol || represents the connection operation, ⊙ represents the element-level product; W 1 , is a learnable matrix, d represents the dimension of the node feature, and f() represents the dual view representation to be connected Linear transformation that projects to the appropriate dimension.
4. According to claim 1, a method for mining relationship nodes based on adaptive information propagation and aggregation hypergraph model, Features: The S3 simplifies the traditional hypergraph convolution into a new DHC; starting from DHC using hypergraph IPM, and then deriving a version combined with dual-view IPM; specifically: S31: First, the nonlinear activation function σ(·) between the traditional hypergraph convolutional layers is removed; the initial node features are added after each information propagation so that the nodes are still distinguishable after multiple iterations, as shown below, where P (l-1) Represents the parameters of the network (l-1) layer; S32: Reparameterize the learnable parameters before the output layer into a single matrix, namely P (hidden) =(P (0) P (1) …P (l-1) ); Therefore, the output of the graph convolutional network is approximated as a two-layer fully connected neural network classifier The input is And Y (l) is the information extracted from the l-hop nearest neighbor as follows; S32: Efficiently extract information from the neighbors of the target node by removing the activation function and re-parameterizing the learnable parameters; finally, use the dual-view IPM Replace SuperGraph IPM To enhance the information extraction process, the formula of DHC combined with dual-view IPM for extracting information from l-hop neighbors can be written as follows:
5. According to claim 1, a method for mining relationship nodes based on adaptive information propagation and aggregation hypergraph model, Features: The S4 uses multiple base classifiers to extract information, and then aggregates the extracted information for final prediction; this process can adaptively filter out noise and highlight valuable information by iteratively updating information weights; multiple DHCs are used as base classifiers, and different base classifiers are used to extract information of different hop neighbors.
