Graph neural network characterization method and device based on high-order motif, and medium
By counting the frequency of the model in complex networks and building a high-order model graph neural network representation model, the problem of existing methods ignoring the node collaboration mode and high computational complexity is solved, and more accurate and robust graph representation is achieved, and complex networks are improved. modeling ability and application performance.
Patent Information
- Application Number
- CN202410903696.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-08
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-08
AI Technical Summary
Existing high-order interaction representation methods ignore the differences in collaboration patterns between nodes, resulting in insufficient information on message delivery and representation accuracy, especially based on sampling estimation methods, potential model candidate subgraphs may be ignored, and computational complexity High precise counting methods lead to computational challenges.
By counting the six model frequencies of size 4 in a complex network, a graph neural network representation model based on high-order model is constructed, using advanced co-occurrence views and semantic attention fusion components, combining contrast learning and random perturbation technology to fuse advanced orders and low-order structural information to alleviate weak connectivity and computational complexity of higher-order views.
Improve the accuracy and robustness of graph representation, enhance the modeling ability of complex networks, provide more distinctive and generalizable network representation, reduce computing complexity, and improve graph neural network application in actual applications performance in.
Smart Images

Figure CN118940793A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of machine learning, and specifically relates to a graph neural network characterization method, device and medium based on a high-order motif. Background Art
[0002] Many systems in the real world exist in the form of networks, such as interpersonal networks, transportation networks, and epidemic transmission networks in social systems; neural networks, protein interaction networks, and biological gene networks in ecosystems; and computer networks and driverless car signal communication networks in scientific and technological systems. Complex networks have become an emerging research hotspot in social sciences, biological sciences, information sciences, statistical physics, computer science, and other disciplines. In recent years, graph neural networks have set off a wave of applying deep learning methods to network data analysis.
[0003] Graph neural networks use low-order interactions to perform neighbor aggregation and feature updates to capture local structural features and patterns of graphs, so as to learn representations of nodes, edges, subgraphs, or the entire graph as input for downstream machine learning tasks. Graph neural network methods help analyze the topological structure of complex networks, understand their functions, discover hidden laws, and predict their behaviors, and have important application value in research in biology, social sciences, information sciences, and other fields.
[0004] However, there are not only low-order point-edge interactions between the entities that make up the network, but also complex interactions between multiple entities, which are called high-order interactions. Representing the network system based on high-order interactions can better understand, analyze and predict the functional modules and dynamic laws of subgraphs, and enhance the graph representation ability. Existing high-order interaction representation methods include hypergraphs, degenerate graphs, simplicial complexes, high-order adjacency tensors, etc. These methods usually regard a high-order interaction as a set of nodes that jointly perform a specific function, without considering the differences in the interaction patterns between the nodes in the set.
[0005] Network motifs are specific substructures that frequently appear in the network, reflecting the collaboration patterns and interaction relationships between multiple nodes in the network. They focus more on specific interaction patterns between multiple nodes. It can be seen that using network motifs to represent high-order interactions can not only describe the set of nodes that make up the high-order interactions, but also reflect the collaboration patterns between nodes. Therefore, by identifying and analyzing network motifs, we can deeply understand and utilize high-order interactions, thereby achieving more meaningful results in complex network research.
[0006] At present, there are existing methods to identify motifs by counting the frequency of non-isomorphic subgraphs with k nodes in the network. These methods can be mainly divided into two categories: methods based on sampling estimation and methods based on exact counting. Motif statistical methods based on sampling estimation judge isomorphism by sampling connected subgraphs with k nodes, estimate the frequency of occurrence of motifs using the sampled subgraph frequencies, and infer the overall frequency of motifs in the entire network based on the sampling results. This type of method may ignore some potential motif candidate subgraphs, thus affecting the accuracy of the final result. In addition, sampling estimation methods have certain limitations in displaying high-order interactions between nodes. Methods based on exact counting systematically enumerate all subgraphs in the network and calculate the frequency of each subgraph. This method can find all frequent subgraphs, accurately display high-order interactions in the network, and provide meaningful high-order interaction information for accurately analyzing high-order potential patterns in the network and revealing the structural characteristics of the network. However, methods based on exact counting require traversing the entire network, resulting in a sharp increase in computational complexity. Therefore, it is necessary to design efficient motif exact statistical methods to meet computational challenges.
[0007] In terms of graph neural network representation, there are methods that use high-order motifs to characterize network interactions to improve network representation. Existing methods regard multiple motif adjacency matrices as network multi-view information, use motif adjacency matrices for convolutional coding to obtain multi-view representations of the network, and select view representations that are more in line with node preferences to improve network representation performance. When constructing a high-order network using a motif adjacency matrix, low-order interactions in the network are often ignored, resulting in low connectivity of the high-order network and nodes in the network being divided into multiple connected branches. The weak connectivity of this network will affect the message passing mechanism in the graph neural network. In addition, when selecting a preferred view for each node, the existing method does not take into account the differences in node preferences for different high-order structures, and ignores the diverse semantic information in different views. Therefore, it is necessary to design a graph neural network method that can effectively integrate the high-order structure and low-order structure information of the graph, alleviate the message passing limitations caused by the weak connectivity of the high-order graph, and produce a more discriminative and generalizable network representation, so as to provide better support for the performance of the graph neural network in practical applications. In view of this, the present invention is proposed. Summary of the invention
[0008] The purpose of the present invention is to provide a graph neural network characterization method, device and medium based on high-order motifs, count the frequency of high-order motifs in complex networks, use high-order interactions between nodes to characterize network characteristics, and improve the accuracy of graph characterization.
[0009] The objective of the present invention is achieved through the following technical solutions:
[0010] A graph neural network representation method based on a high-order motif comprises the following steps:
[0011] Step 1, obtaining a network data set, wherein the data set includes entity attribute features and interaction relationships between entities in a complex network, and counting the frequencies of six motifs of size 4 in the complex network, specifically: 4-path, 4-circle, 3-star, tail triangle, chord circle, and 4-cluster, to obtain a corresponding motif instance set;
[0012] Step 2: construct a graph neural network representation model based on a high-order motif, wherein the graph neural network representation model based on a high-order motif includes a high-order view construction module, a feature perturbation view construction module and a network coding module;
[0013] Step 3: Use the network dataset to train a graph neural network representation model based on a high-order motif to obtain a graph representation based on a high-order motif as a vectorized input for downstream machine learning tasks.
[0014] Furthermore, step 1 specifically includes:
[0015] Get the network data set G = (V, E, X), whose adjacency matrix is A0; sort the nodes in the complex network G = (V, E, X) in non-increasing order by degree, and let where v1, v2, ..., v n Represent n nodes in the complex network, Represents node v i degree; for each node v i ,like Select three nodes from its neighborhood {v x ,v y ,v z}, consisting of the node set {v i ,v x ,v y ,v z}Get the induced subgraph G[v i ,v x ,v y ,v z ], let its edge number be j: if j = 0, then the 3-star structure frequency of the motif is increased by 1, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 3-star corresponding motif instance set I[M0]; if j=1, the frequency of the motif tail triangle structure statistics is increased by 1, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the motif instance set I[M1] corresponding to the tail triangle; if j=2, the statistical frequency of the motif chord structure is increased by 1 / 2, and the derived subgraph G[v i ,vx ,v y ,v z ] is stored in the motif instance set I[M2] corresponding to the chord circle; if j=3, the frequency of the motif 4-cluster structure statistics is increased by 1 / 4, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 4-cluster corresponding motif instance set I[M3]; for each node v i ,like Node v i The neighborhoods are v x With v y , v z For node v x or v y Neighborhood of and not v i , using the optional three nodes {v x ,v y ,v z}, consisting of the node set {v i ,v x ,v y ,v z}Get the induced subgraph G[v i ,v x ,v y ,v z ]:If v z Non-node v x With v y The common neighborhood of the motif 4-path structure is increased by 1 / 2, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 4-path corresponding motif instance set I[M4]; if v z For node v x With v y The common neighborhood of the motif 4-loop structure is increased by 1 / 4, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 4-circle corresponding motif instance set I[M5].
[0016] Furthermore, the step of constructing a graph neural network representation model based on a high-order motif in step 2 includes:
[0017] Step 2.1, using the co-occurrence frequency of motifs between node pairs to construct the corresponding high-order co-occurrence view as the anchor sample view;
[0018] Step 2.2, randomly perturb the complex network feature matrix to simulate the noise and changes in the real world to obtain negative sample views;
[0019] Step 2.3, construct a network coding module, wherein the network coding module includes a graph neural network of an encoder component, a contrastive learning component, a semantic attention fusion component and a decoder component.
[0020] Furthermore, the step of constructing a corresponding high-order co-occurrence view as an anchor sample view using the co-occurrence frequency of motifs between node pairs in step 2.1 includes:
[0021] For any motif M k , count the number of co-occurrences of the motif between node pairs, and get the k A high-order co-occurrence view of as the anchor sample view, where V represents the node set, Represents the relationship between nodes regarding the motif M k The co-occurrence relationship, X represents the attribute characteristics of the node; specifically, the adjacency matrix of the high-order co-occurrence view is:
[0022]
[0023] where i≠j, is an indicator function. If statement s is true, then otherwise The high-order co-occurrence view clusters the subgraphs that are isomorphic to the given motif; thus, k high-order co-occurrence views can be obtained.
[0024] Furthermore, the step of randomly perturbing the complex network feature matrix in step 2.2 to simulate noise and changes in the real world to obtain a negative sample view includes:
[0025] The complex network G has a feature matrix X containing n node features. Generate a random sequence of length n, keep the overall distribution and properties of node features unchanged, and exchange the features of different nodes according to the generated random sequence to obtain the perturbed feature matrix Simulate actual noise and changes to obtain negative sample views Where V represents the node set of the complex network G, E represents the interaction relationship between nodes, Represents the attribute characteristics of the node after perturbation.
[0026] Furthermore, the step of constructing the network coding module in step 2.3 includes:
[0027] The encoder component extracts local and global features of nodes through layer-by-layer convolution and aggregation operations, and maps these features to a low-dimensional space to obtain a compact representation of the graph:
[0028]
[0029] in, and are the feature information of node v and node u at the l-1th iteration, AGGREGATION(·) is the function of extracting node neighborhood features, and COMBINE(·) is the function of aggregating node neighborhood features; normalization preprocessing is performed on each view in I is the unit diagonal matrix, express The degree matrix of ; the specific node representation of the k-th view is captured by:
[0030]
[0031] in, is the weight matrix from the input layer to the hidden layer of the first layer of graph convolutional encoder, is the weight matrix from the hidden layer to the output layer of the second-layer graph convolutional encoder; for each view, the graph convolutional encoder can be used to transform the positive sample view G and the anchor sample view And negative sample view Iteratively aggregate the feature information of the node neighborhood to obtain the compact embedding representation matrix corresponding to the view
[0032] The contrastive learning component uses contrastive loss to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs, while combining attribute noise to enhance robustness, and learns highly discriminative node feature representations to reflect the similarities and differences between nodes;
[0033] The semantic attention fusion component assigns different weights to each high-order structure co-occurrence view information, and uses attention-based semantic fusion to learn the encoding information of high-order views and low-order views to obtain the feature representation of information fusion; specifically, by learning the weights of the views (w0,...,w k ) Analyze the different significance of different views to the graph representation; the importance of the kth view α k The calculation is as follows:
[0034]
[0035] Among them, V is the node set, q represents the attention vector, C is the weight matrix, and b is the bias vector; is the kth view node v iThe representation vector is the graph encoding Z k The semantic attention fusion module transforms the view-specific representation through a single-layer MLP, and the importance of the view-specific embedding is measured by the attention vector q; then, the view-specific importance w k Normalized by the softmax function:
[0036]
[0037] After obtaining the weight of each view, the information fused graph is represented as:
[0038]
[0039] Through attention semantic fusion, the information fused encoding Z is obtained as the graph representation;
[0040] The decoder component is responsible for decoding the encoded representation into original data; the inner product decoder is used to reconstruct the graph topology features, and the cosine similarity of each pair of nodes is calculated to obtain the reconstructed adjacency matrix:
[0041]
[0042] in, is the graph reconstruction adjacency matrix, N is the trainable weight;
[0043] Use graph convolution decoder to restore node feature information:
[0044]
[0045] in, is the graph reconstruction attribute matrix, Z is the semantic fusion graph embedding, and W (0) and W (1) They are the two trainable weight matrices of the two convolutional layers in the decoder.
[0046] Furthermore, the step of using the network dataset to train the graph neural network representation model based on the high-order motif in step 3 to obtain the graph representation based on the high-order motif as the vectorized input of the downstream machine learning task includes:
[0047] The graph neural network representation model based on high-order motifs is used for training to obtain a complex network feature matrix Z as a graph representation. During the training process, the loss function of contrastive learning in the graph neural network representation model based on high-order motifs is:
[0048]
[0049] in, Represents the jth high-level view node v i The graph shows that Represents a low-level view node v i The graph shows that Represents the perturbation view node v i The graph representation of w (·) represents the consistency between views, where k represents the number of high-order views, and optimization learning is performed on all high-order views;
[0050] During the training process, the reconstruction loss function in the graph neural network representation model based on the high-order motif is:
[0051]
[0052] Among them, A represents the adjacency matrix of graph G, represents the topological features of the decoded graph, X represents the graph feature matrix, Represents the attribute features of the decoded graph; by minimizing L r It allows the encoding module to better learn the key features of graph nodes;
[0053] The loss function in the graph neural network representation model based on high-order motifs is:
[0054] L=λ c L c +λ r L r ,
[0055] Among them, λ c , r They are all weight parameters for balancing the global loss.
[0056] The present invention provides a graph neural network representation device based on a high-order motif, the device comprising:
[0057] at least one memory for storing one or more programs;
[0058] At least one processor can execute one or more programs stored in the memory. When the one or more programs are executed by the processor, the processor can implement the method described in the present invention.
[0059] The present invention provides a readable storage medium storing a computer program, which can implement the method described in the present invention when the computer program is executed by a processor.
[0060] Compared with the prior art, according to the graph neural network characterization method, device and medium based on high-order motifs of each solution of the present invention, the present invention has the following technical effects:
[0061] First, the present invention proposes a method based on precise counting, which can count the frequencies of six motifs of size 4 (4-path, 4-loop, 3-star, tail triangle, chord loop, 4-cluster) in complex networks, so as to accurately display the high-order interaction relationships in the network and reveal the structural characteristics of the network;
[0062] Second, the present invention designs and trains a graph autoencoder model based on semantic attention fusion of low-order and high-order graph structures, analyzes the different effects of different high-order structures on graph representation, and selects appropriate high-order structures to optimize graph representation;
[0063] Third, the present invention uses high-order co-occurrence views as anchor sample views, randomly perturbed complex network feature matrices as negative sample views, and the complex network itself as positive sample views, and uses contrastive learning to enhance the cross-view of low-order and high-order structures of nodes. Figure 1 The consistency is improved to alleviate the message transmission limitation caused by the weak connectivity of high-order views, and attribute noise is introduced to enhance the robustness of graph representation.
[0064] In summary, the present invention uses motif co-occurrence to characterize high-order interaction patterns between node pairs, enhances the modeling capabilities of complex networks, and obtains more accurate and robust graph representations for various machine learning tasks, which has great social and commercial value. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] Figure 1 It is a flowchart of a graph neural network representation method based on a high-order motif according to an embodiment of the present invention.
[0066] Figure 2 It is a structural diagram of six high-order motifs (4-path, 4-loop, 3-star, tail triangle, chord loop, 4-cluster) of six sizes of 4 counted by the present invention.
[0067] Figure 3 It is a structural diagram of a graph neural network representation method based on a high-order motif according to an embodiment of the present invention. DETAILED DESCRIPTION
[0068] In combination with the specific content of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described below; obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments, which does not constitute a limitation of the present invention. In order to more clearly show the technical scheme provided by the present invention and the technical effects produced, the graph neural network representation method based on high-order motifs provided by the embodiment of the present invention is described in detail below with a specific embodiment.
[0069] like Figure 1As shown, this embodiment provides a graph neural network characterization method based on high-order motifs, which is a method for characterizing network features by using high-order interactions between nodes to improve the accuracy of graph characterization. Specifically, it includes the following steps:
[0070] Step 1: Obtain a network data set, which includes entity attribute features and the interaction relationship between entities in a complex network, and count the frequencies of six motifs (4-path, 4-circle, 3-star, tail triangle, chord circle, 4-cluster) with a size of 4 in the complex network to obtain the corresponding motif instance set. The specific steps include:
[0071] Step 1.1, obtain the citation network dataset through the academic search engine Baidu Academic, which includes document titles, abstracts, keywords, author information and citation relationships. Perform missing value processing and cleaning on the document information, and represent each document as a node, which contains the metadata information of the document. If document A cites document B, there is a directed edge in the graph from node A to node B, indicating the citation relationship. Subsequently, attribute features are extracted from the document metadata and encoded or vectorized. Finally, a citation network G = (V, E, X) is constructed, where V represents the node set of the citation network, E represents the edge set of the citation network, and X represents the attribute features of the document. The adjacency matrix of the citation network is A0.
[0072] Step 1.2: Count the frequency of motifs of size 4 in the citation network G = (V, E, X). The relevant structure is as follows: Figure 2 As shown. Sort the nodes in the complex network in non-increasing order by degree, where v1, v2, ..., v n Represent n nodes in the complex network, Represents node v i degree; for each node v i ,like Select three nodes from its neighborhood {v x ,v y ,v z}, consisting of the node set {v i ,v x ,v y ,v z}Get the induced subgraph G[v i ,v x ,v y ,v z ], let its edge number be j: if j = 0, then the 3-star structure frequency of the motif is increased by 1, and the derived subgraph G[v i ,v x ,v y ,v z] is stored in the 3-star corresponding motif instance set I[M0]; if j=1, the frequency of the motif tail triangle structure statistics is increased by 1, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the motif instance set I[M1] corresponding to the tail triangle; if j=2, the statistical frequency of the motif chord structure is increased by 1 / 2, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the motif instance set I[M2] corresponding to the chord circle; if j=3, the frequency of the motif 4-cluster structure statistics is increased by 1 / 4, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 4-cluster corresponding motif instance set I[M3]; for each node v i ,like Node v i The neighborhoods are v x With v y , v z For node v x or v y Neighborhood of and not v i , using the optional three nodes {v x ,v y ,v z}, consisting of the node set {v i ,v x ,v y ,v z}Get the induced subgraph G[v i ,v x ,v y ,v z ]:If v z Non-node v x With v y The common neighborhood of the motif 4-path structure is increased by 1 / 2, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 4-path corresponding motif instance set I[M4]; if v z For node v x With v y The common neighborhood of the motif 4-loop structure is increased by 1 / 4, and the derived subgraph G[v i ,v x ,v y ,v z] is stored in the motif instance set I[M5] corresponding to the 4-circle. The motif instance sets I[M0], I[M1], I[M2], I[M3], I[M4], and I[M5] represent the subgraphs in the network that are isomorphic to 4-path, 4-circle, 3-star, tail triangle, chord circle, and 4-cluster, respectively, representing different sets of literature citation relationships. By displaying high-order citation relationships such as clusters, chains, and stars in the citation network, we can deeply analyze the network structure characteristics and identify hidden research trends and hotspots.
[0073] Step 2: Figure 3 The graph neural network representation model based on high-order motifs is constructed, and the graph neural network representation model based on high-order motifs includes a high-order view construction module, a feature perturbation view construction module and a network coding module. The specific steps include:
[0074] Step 2.1: Use the co-occurrence frequency of motifs between node pairs to construct the corresponding high-order co-occurrence view as the anchor sample view. k , count the number of co-occurrences of the motif between node pairs, and get the k A high-order co-occurrence view of as the anchor sample view, where V represents the node set, Represents the relationship between nodes regarding the motif M k The co-occurrence relationship, X represents the attribute characteristics of the node; specifically, the adjacency matrix of the high-order co-occurrence view is:
[0075]
[0076] Where i≠j, I[M k ] is the set of motif instances corresponding to the kth motif in the network, i m For I[M k ], is an indicator function. If statement s is true, then otherwise If a pair of nodes co-occurs in a motif instance, it is considered that there is an edge between the pair of nodes in the high-order view, so that the subgraphs isomorphic to the given motif are clustered. Based on this, k high-order co-occurrence views can be obtained:
[0077] Step 2.2: Randomly perturb the complex network feature matrix to simulate the noise and changes in the real world to obtain negative sample views. The complex network G has a feature matrix X containing n node features. Generate a random sequence of length n, keep the overall distribution and properties of node features unchanged, and exchange the features of different nodes according to the generated random sequence to obtain the perturbed feature matrix This simulates the noise and changes in the real world and obtains negative sample views Where V represents the node set in the complex network, E represents the interaction relationship between nodes, Represents the attribute characteristics of the node after perturbation. This method can simulate the uncertainty and dynamics in the real environment, thereby exploring the data space more comprehensively and enhancing the generalization ability of real data.
[0078] Step 2.3, construct a network coding module, which includes a graph neural network of an encoder component, a contrastive learning component, a semantic attention fusion component and a decoder component.
[0079] The encoder component extracts local and global features of nodes through layer-by-layer convolution and aggregation operations, and maps these features to a low-dimensional space to obtain a compact representation of the graph:
[0080]
[0081] in, and The feature information of node v and node u at the l-1th iteration respectively. The AGGREGATION(·) operation extracts the neighborhood information of the node by aggregating the neighbor features and using the weight sharing mechanism. The COMBINE(·) operation combines the aggregated information with the node's own features to generate the next level of node representation. All view-specific graph encoders consist of a two-layer graph convolutional encoder: the first layer uses a convolutional layer to extract features, uses the ReLU activation function, and the output node feature dimension is n×128. The second layer is a convolutional layer with an output node feature dimension of n×32. Normalization preprocessing is performed on each view in, I is the unit diagonal matrix, express The specific node representation of the k-th view is captured by:
[0082]
[0083] in, is the weight matrix from the input layer to the hidden layer of the first layer of graph convolutional encoder, is the weight matrix from the hidden layer to the output layer of the second-layer graph convolutional encoder. For each view, the graph convolutional encoder can be used to train the positive sample view G and the anchor sample view And negative sample view Iteratively aggregate the feature information of the node neighborhood to obtain the compact embedding representation matrix corresponding to the view
[0084] The contrastive learning component uses contrastive loss to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs, while combining attribute noise to enhance robustness, so that the model can learn more discriminative feature representations. In this embodiment, contrastive learning focuses on the difference between the real pattern and the perturbation pattern in the feature, improves the network coding module's resistance to data changes and noise, and enhances the generalization ability of the graph neural network on real data.
[0085] The semantic attention fusion component takes into account the diverse semantic information contained in the nodes in different high-order structure co-occurrence views, and the different importance of each high-order structure co-occurrence view information, and needs to assign different weights to each view. The embodiment of the present invention uses attention-based semantic fusion to learn the encoding information of high-order views and low-order views to obtain the feature representation of information fusion. Specifically, by learning the weights of the views (w0,...,w k ) Analyze the different significance of different views to the graph representation. The importance of the k-th view α k The calculation is as follows:
[0086]
[0087] Among them, V is the node set, q represents the attention vector, C is the weight matrix, and b is the bias vector; is the kth view node v i The representation vector is the graph encoding Z k The semantic attention fusion module transforms the view-specific representation through a single-layer MLP, and the importance of the view-specific embedding is measured by the attention vector q; then, the view-specific importance w k Normalized by the softmax function:
[0088]
[0089] After obtaining the weight of each view, the information fused graph is represented as:
[0090]
[0091] Through attention semantic fusion, the information fused encoding Z is obtained as the graph representation.
[0092] The decoder component is responsible for decoding the encoded representation into the original data. The inner product decoder is used to reconstruct the graph topology features, and the cosine similarity of each pair of nodes is calculated to obtain the reconstructed adjacency matrix:
[0093]
[0094] in, is the graph reconstruction adjacency matrix, and N is the trainable weight.
[0095] In order to make full use of the information contained in the multi-view graph, the graph convolution decoder is used to restore the node feature information:
[0096]
[0097] in, is the graph reconstruction attribute matrix, Z is the semantic fusion graph embedding, and W (0) and W (1) They are the two trainable weight matrices of the two convolutional layers in the decoder. The graph convolution decoder includes two graph neural network layers: the first layer uses the deconvolution layer to refine the features, uses the ReLU activation function, and the output node feature dimension is 128; the second layer is the convolution layer, and the output node feature dimension is d, and the reconstructed node feature matrix is obtained.
[0098] Step 3: Use the network dataset to train a graph neural network representation model based on a high-order motif to obtain a graph representation based on a high-order motif as a vectorized input for downstream machine learning tasks. The specific steps include:
[0099] Step 3.1. Let the citation network G = (V, E, X) constructed in step 1 and the corresponding six motif instance sets I[M0], I[M1], I[M2], I[M3], I[M4], and I[M5] be the input of the graph neural network representation model based on high-order motifs in step 2.
[0100] Step 3.2, the model described in step 2 learns the low-dimensional embedding Z of the citation network by optimizing the objective function, so that these embedding vectors can effectively reflect the global and local characteristics and relationships of each node (document). The training process uses the contrast loss term to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs. The loss function of contrastive learning in the graph neural network representation model based on high-order motifs is:
[0101]
[0102] in, Represents the jth high-level view node v i The graph shows that Represents a low-level view node v i The graph shows that Represents the perturbation view node v i The graph representation of w (·) represents the consistency between views, where k represents the number of high-order views. This embodiment performs optimization learning on all high-order views.
[0103] In addition, this embodiment uses a loss function to measure the difference between the real topological features and the reconstructed topological features, and the real attribute features and the reconstructed attribute features. The reconstruction loss function in the graph neural network representation model based on the high-order motif is:
[0104]
[0105] Among them, A represents the adjacency matrix of graph G, represents the topological features of the decoded graph, X represents the graph feature matrix, Represents the decoded graph attribute features. By minimizing L r This allows the encoding module to better learn the key features of graph nodes.
[0106] This embodiment combines contrast loss and reconstruction loss to optimize the model. The loss function in the graph neural network representation model based on the high-order motif is:
[0107] L=λ c L c +λ r L r ,
[0108] Among them, λ c , r They are all weight parameters for balancing the global loss.
[0109] This embodiment uses the Adam optimization method to adjust the contrast loss function, reconstruction loss function and network parameters, and the number of adjustment iterations is 100; during the iterative training process, the first-order moment estimation and second-order moment estimation of the gradient are used to dynamically adjust the learning rate of each parameter, so that the parameter changes are relatively stable, and finally the graph representation Z of the citation network is obtained to quantify the similarity and influence of the documents.
[0110] Step 3.3, this implementation uses probabilistic latent semantic analysis (Latent Dirichlet Allocation, LDA) to identify potential topic structures, output topic distribution, and determine the number of clusters, each cluster representing a potential topic or category. In addition, low-dimensional embeddings usually have clearer geometric structures, which are more consistent with the spherical or approximately spherical clusters assumed by K-means clustering. This embodiment uses the K-means clustering method to cluster the citation network. By clustering the citation network to obtain clustering results, it is helpful to understand the development dynamics of the academic research field by discovering groups or clusters between documents.
[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
Claims
1. A graph neural network representation method based on high-order motifs, characterized in that: The following steps are involved: Step 1, obtaining a network data set, wherein the data set includes entity attribute features and interaction relationships between entities in a complex network, and counting the frequencies of six motifs of size 4 in the complex network, specifically: 4-path, 4-circle, 3-star, tail triangle, chord circle, and 4-cluster, to obtain a corresponding motif instance set; Step 2: construct a graph neural network representation model based on a high-order motif, wherein the graph neural network representation model based on a high-order motif includes a high-order view construction module, a feature perturbation view construction module and a network coding module; Step 3: Use the network dataset to train a graph neural network representation model based on a high-order motif to obtain a graph representation based on a high-order motif as a vectorized input for downstream machine learning tasks.
2. The graph neural network representation method based on high-order motifs according to claim 1 is characterized in that: Step 1 specifically includes: Get the network data set G = (V, E, X), whose adjacency matrix is A0; sort the nodes in the complex network G = (V, E, X) in non-increasing order by degree, and let d v1 ≥d v2 ≥...≥d vn , where v1, v2, ..., v n Represents n nodes in the complex network, d vi Represents node v i degree; for each node v i , if d vi ≥3, select three nodes from its neighborhood {v x ,v y ,v z }, consisting of the node set {v i ,v x ,v y ,v z }Get the induced subgraph G[v i ,v x ,v y ,v z ], let its edge number be j: if j = 0, then the 3-star structure frequency of the motif is increased by 1, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 3-star corresponding motif instance set I[M0]; if j=1, the frequency of the motif tail triangle structure statistics is increased by 1, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the motif instance set I[M1] corresponding to the tail triangle; if j=2, the statistical frequency of the motif chord structure is increased by 1 / 2, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the motif instance set I[M2] corresponding to the chord circle; if j=3, the frequency of the motif 4-cluster structure statistics is increased by 1 / 4, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 4-cluster corresponding motif instance set I[M3]; for each node v i , if d vi =2, node v i The neighborhoods are v x With v y , v z For node v x or v y Neighborhood of and not v i , using the optional three nodes {v x ,v y ,v z }, consisting of the node set {v i ,v x ,v y ,v z }Get the induced subgraph G[v i ,v x ,v y ,v z ]:If v z Non-node v x With v y The common neighborhood of the motif 4-path structure is increased by 1 / 2, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 4-path corresponding motif instance set I[M4]; if v z For node v x With v y The common neighborhood of the motif 4-loop structure is increased by 1 / 4, and the derived subgraph G[v i ,v x ,v y ,v z ] is stored in the 4-circle corresponding motif instance set I[M5].
3. The graph neural network representation method based on high-order motifs according to claim 1 is characterized in that: Step 2 specifically includes: Step 2.1, using the co-occurrence frequency of motifs between node pairs to construct the corresponding high-order co-occurrence view as the anchor sample view; Step 2.2, randomly perturb the complex network feature matrix to simulate the noise and changes in the real world to obtain negative sample views; Step 2.3, construct a network coding module, wherein the network coding module includes a graph neural network of an encoder component, a contrastive learning component, a semantic attention fusion component and a decoder component.
4. The graph neural network representation method based on high-order motifs according to claim 3 is characterized in that: Step 2.1 specifically includes: For any motif M k , count the number of co-occurrences of the motif between node pairs, and get the k A high-order co-occurrence view of as the anchor sample view, where V represents the node set, Represents the relationship between nodes regarding the motif M k The co-occurrence relationship, X represents the attribute characteristics of the node; specifically, the adjacency matrix of the high-order co-occurrence view is: where i≠j, is an indicator function. If statement s is true, then otherwise The high-order co-occurrence view clusters the subgraphs that are isomorphic to the given motif; thus, k high-order co-occurrence views can be obtained.
5. The graph neural network representation method based on high-order motifs according to claim 3 is characterized in that: Step 2.2 specifically includes: The complex network G has a feature matrix X containing n node features. Generate a random sequence of length n, keep the overall distribution and properties of node features unchanged, and exchange the features of different nodes according to the generated random sequence to obtain the perturbed feature matrix Simulate actual noise and changes to obtain negative sample views Where V represents the node set of the complex network G, E represents the interaction relationship between nodes, Represents the attribute characteristics of the node after perturbation.
6. The graph neural network representation method based on high-order motifs according to claim 3 is characterized in that: Step 2.3 specifically includes: The encoder component extracts local and global features of nodes through layer-by-layer convolution and aggregation operations, and maps these features to a low-dimensional space to obtain a compact representation of the graph: in, and are the feature information of node v and node u at the l-1th iteration, AGGREGATION(·) is the function of extracting node neighborhood features, and COMBINE(·) is the function of aggregating node neighborhood features; normalization preprocessing is performed on each view in I is the unit diagonal matrix, express The degree matrix of ; the specific node representation of the k-th view is captured by: in, is the weight matrix from the input layer to the hidden layer of the first layer of graph convolutional encoder, is the weight matrix from the hidden layer to the output layer of the second-layer graph convolutional encoder; for each view, the graph convolutional encoder can be used to transform the positive sample view G and the anchor sample view And negative sample view Iteratively aggregate the feature information of the node neighborhood to obtain the compact embedding representation matrix corresponding to the view The contrastive learning component uses contrastive loss to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs, while combining attribute noise to enhance robustness, and learns highly discriminative node feature representations to reflect the similarities and differences between nodes; The semantic attention fusion component assigns different weights to each high-order structure co-occurrence view information, and uses attention-based semantic fusion to learn the encoding information of high-order views and low-order views to obtain the feature representation of information fusion; specifically, by learning the weights of the views (w0,...,w k ) Analyze the different significance of different views to the graph representation; the importance of the kth view α k The calculation is as follows: Among them, V is the node set, q represents the attention vector, C is the weight matrix, and b is the bias vector; is the kth view node v i The representation vector is the graph encoding Z k The semantic attention fusion module transforms the view-specific representation through a single-layer MLP, and the importance of the specific view embedding is measured by the attention vector q; then, the view-specific importance w k Normalized by the softmax function: After obtaining the weight of each view, the information fused graph is represented as: Through attention semantic fusion, the information fused encoding Z is obtained as the graph representation; The decoder component is responsible for decoding the encoded representation into original data; the inner product decoder is used to reconstruct the graph topology features, and the cosine similarity of each pair of nodes is calculated to obtain the reconstructed adjacency matrix: in, is the graph reconstruction adjacency matrix, N is the trainable weight; Use graph convolution decoder to restore node feature information: in, is the graph reconstruction attribute matrix, Z is the semantic fusion graph embedding, and W (0) and W (1) They are the two trainable weight matrices of the two convolutional layers in the decoder.
7. The graph neural network representation method based on high-order motifs according to claim 1 is characterized in that: Step 3 specifically includes: The graph neural network representation model based on high-order motifs is used for training to obtain a complex network feature matrix Z as a graph representation. During the training process, the loss function of contrastive learning in the graph neural network representation model based on high-order motifs is: in, Represents the jth high-level view node v i The graph shows that Represents a low-level view node v i The graph shows that Represents the perturbation view node v i The graph representation of w (·) represents the consistency between views, where k represents the number of high-order views, and optimization learning is performed on all high-order views; During the training process, the reconstruction loss function in the graph neural network representation model based on the high-order motif is: Among them, A represents the adjacency matrix of graph G, represents the topological features of the decoded graph, X represents the graph feature matrix, Represents the attribute features of the decoded graph; by minimizing L r It allows the encoding module to better learn the key features of graph nodes; The loss function in the graph neural network representation model based on high-order motifs is: L=λcLc+λ r L r , Among them, λc, λ r They are all weight parameters for balancing the global loss.
8. A processing device, characterized in that: include: at least one memory for storing one or more programs; At least one processor can execute one or more programs stored in the memory, and when the one or more programs are executed by the processor, the processor can implement the method described in any one of claims 1 to 7.
9. A readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 can be implemented.
Citation Information
Patent Citations
Network representation learning algorithm combining high-order structure and attention mechanism
CN114692867A
Graph representation learning method based on multi-source interactive fusion
CN114756713A
Graph neural network interpretable method based on network motif
CN115730657A
Device and method for generating network using complex network properties
US20230214672A1
Method and apparatus for learning graph representation for out-of-distribution generalization, device and storage medium
US20230289617A1
