Dynamic heterogeneous graph neural network model

Abstract

The invention discloses a dynamic heterogeneous graph neural network model, and relates to the technical field of neural network models, and the key points of the technical scheme are that node and edge feature processing is performed by adopting a type-based projection mode, and an attention matrix of an embedded node type is defined for each node and edge type; projection features for different node types are obtained through the attention matrix, and unified representation of node and edge features is achieved. According to the dynamic heterogeneous graph neural network model, dynamic modeling and learning capabilities are combined, and the dynamic heterogeneous graph neural network model can adapt to dynamic changes of a heterogeneous graph structure. Not only can feature representation of nodes and edges be extracted from graph data, but also adaptive learning can be carried out according to states at different moments, so that more accurate and more comprehensive graph analysis and prediction are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network models, and more specifically, to a dynamic heterogeneous graph neural network model. Background Art

[0002] At present, the feature extraction methods for heterogeneous graphs mainly rely on meta-path feature aggregation based on prior knowledge. However, this method has some limitations. First, due to the limitation of prior knowledge, this method often cannot adapt well to the dynamically changing graph structure. The topological structure and feature attributes of heterogeneous graphs may change over time, and it is difficult for the static meta-path feature aggregation method to effectively model this dynamicity. Second, it is difficult for this method to adaptively learn according to the state at different times. For nodes and edges in heterogeneous graphs, they may have different importance and influence at different times, and it is difficult for existing methods to flexibly capture and respond to such changes. The traditional graph attention neural network model has significant advantages. It can accept the input of dynamic graphs and mainly relies on defining attention weights based on the relationship between nodes. The relationship between nodes can be adjusted and updated in dynamic graphs, which is more flexible and has strong adaptability when processing dynamic graphs. However, for heterogeneous graphs, it ignores the heterogeneous information and attributes of the graph during calculation, cannot extract the complete feature representation of nodes and edges from the graph data, and is difficult to support accurate and comprehensive graph analysis and prediction.

[0003] Therefore, in order to overcome the limitations of the above-mentioned prior art, the present invention proposes a dynamic heterogeneous graph neural network model. Summary of the invention

[0004] The purpose of the present invention is to overcome the limitations of the above-mentioned prior art and, based on the traditional graph attention neural network, propose a dynamic heterogeneous graph neural network model that combines path embedding and adaptive embedding.

[0005] The above technical purpose of the present invention is achieved through the following technical solutions: a dynamic heterogeneous graph neural network model, which uses a type-based projection method to process node and edge features, defines an attention matrix that embeds the node type for each node and edge type, and obtains projection features for different node types through the attention matrix to achieve a unified representation of node and edge features.

[0006] The present invention is further configured as follows: the attention matrix maps the features of nodes and edges into a unified feature space, and performs weighted fusion on them, and obtains projection features for different node types by multiplying the features of nodes and edges with the attention matrix.

[0007] The present invention is further configured as follows: when fusing the features of nodes and edges, an attention mechanism is used; a corresponding attention weight is assigned to each node and edge according to the type information of the node and edge; a weighted fused feature representation is obtained by multiplying and adding the features of the node and edge with the attention weight, and then the projection features for different node types are obtained, and the features of the nodes and edges are projected to the same distribution.

[0008] The present invention is further configured to: use a path-based attention calculation method to process heterogeneous information and attributes of the graph.

[0009] The path-based attention calculation method first defines a path neighbor. The path neighbor of a node refers to the set of nodes connected to the node through a specific path. For each path neighbor node, its embedded features and the fusion features of heterogeneous information are calculated. The fusion features are obtained by combining the embedded features of the path neighbor nodes with the edge attributes in the path. The embedded features of the path neighbor nodes are fused together with the heterogeneous information into the path features to capture the structural information and association relationships between nodes, and play a role through weighted edge attribute attention to dynamically adjust the contribution of the features.

[0010] The present invention is further configured as follows: by calculating the attention coefficient between paths, the attention coefficient is obtained by calculating the similarity between the fusion features of the path neighbor nodes and the current node representation; the higher the similarity, the greater the attention coefficient, indicating that the path contributes more to the current node representation; the lower the similarity, the smaller the attention coefficient, indicating that the path contributes less to the current node representation, and the final embedded features are output.

[0011] The present invention is further configured as follows: the size and shape of the receptive field of each node depends on the number and mode of connection between the node and neighboring nodes; and information is captured by using an architecture that adaptively adjusts the degree of neighborhood aggregation information.

[0012] The present invention is further configured as follows: the architecture for adaptively adjusting the degree of neighborhood aggregation information is composed of a jump connection and an adaptive information aggregation mechanism; the jump connection allows nodes to directly access information from deeper neighbor nodes without going through a complete information transmission process; through the adaptive information aggregation mechanism, the degree of information transmission and aggregation is dynamically adjusted according to the node connection mode and context requirements.

[0013] In summary, the present invention has the following beneficial effects: 1. A dynamic heterogeneous graph neural network model of the present invention has an architecture that can accept dynamic heterogeneous graphs with changing structures and attributes, and can retain timing information at different times, thereby completely and effectively retaining the overall characteristics of the graph.

[0014] 2. Allowing flexible adjustment of information transmission and aggregation methods according to the specific conditions of the nodes. This enables the method of the present invention to better adapt to the connection methods and context requirements of different nodes.

[0015] 3. Through skip connections and adaptive information aggregation mechanism, the method of the present invention can more directly obtain contextual information that has an important impact on node embedding, thereby improving the accuracy and efficiency of embedding.

[0016] 4. The aggregation weights in the architecture reflect the importance a node places on information from neighboring nodes at different levels. This allows for a better understanding of the node embedding process and improves the interpretability of the model.

[0017] 5. Since the method of the present invention does not depend on a specific graph structure or node type, it has strong generalization ability and can be applied to various types of graph data, such as social networks, knowledge graphs, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a traditional graph neural network model heterogeneous graph;

[0019] Figure 2 It is the feature aggregation and similarity calculation based on the path in the embodiment of the present invention;

[0020] Figure 3 is the receptive field of the network node in the embodiment of the present invention;

[0021] Figure 4 is a schematic diagram of a jump connection in an embodiment of the present invention;

[0022] Figure 5 This is the heterogeneous graph neural network structure and calculation process of the present invention. DETAILED DESCRIPTION

[0023] The following is combined with Figure 1-5 The present invention is described in further detail.

[0024] Example: Traditional graph neural network models usually assume that nodes and edges in the graph have the same feature distribution, which is often not true in practical applications, such as Figure 1 As shown, different types of nodes and relationships often have different feature distributions and attributes, which requires the model to be able to process and fuse features according to different types.

[0025] In order to reduce the difficulty of training and achieve maximum parameter sharing while maintaining different relationship features, a dynamic heterogeneous graph neural network model uses a type-based projection method to process node and edge features, and defines an attention matrix embedded in the node type for each node and edge type. The function of the attention matrix is ​​to map the features of nodes and edges into a unified feature space and perform weighted fusion on them. By multiplying the features of nodes and edges with the attention matrix, the projection features for different node types are obtained, and the unified representation of node and edge features is achieved.

[0026] When fusing node and edge features, the attention mechanism is used. A corresponding attention weight is assigned to each node and edge according to the node and edge type information. The attention weight reflects the importance and relevance of the node and edge in the graph. By multiplying and adding the node and edge features with the attention weight, a weighted fused feature representation is obtained, and the projection features for different node types are obtained, and the node and edge features are projected to the same distribution. This type-based projection method and the fusion method of the attention matrix enable the model to better handle graph data with heterogeneous characteristics.

[0027] First, we define the concept of path neighbors. The path neighbors of a node refer to the set of nodes connected to the node through a specific path, such as Figure 2 As shown in the figure. For each path neighbor node, the fusion feature of its embedded features and heterogeneous information is calculated. The fusion feature is obtained by combining the embedded features of the path neighbor nodes with the edge attributes in the path. In this way, the embedded features of the path neighbor nodes are fused with the heterogeneous information into the path features to capture the structural information and association relationship between the nodes, and the weighted edge attribute attention is used to make it work and dynamically adjust the contribution of the feature.

[0028] Finally, by calculating the attention coefficient between paths, the attention coefficient is obtained by calculating the similarity between the fusion features of the path neighbor nodes and the current node representation. The higher the similarity, the greater the attention coefficient, indicating that the path contributes more to the current node representation. On the contrary, the lower the similarity, the smaller the attention coefficient, indicating that the path contributes less to the current node representation, and the final embedded features are output. The traditional node-level attention calculation method has certain limitations when processing heterogeneous information and attributes of the graph. In this way, in the model of the present invention, the path-based attention calculation method can better capture the relationship and structural information between nodes, and dynamically adjust the contribution of features, so that the model can more accurately capture the heterogeneous information and attributes in the graph. Finally, the attention coefficients of all paths are weighted and summed with the corresponding fusion features to obtain the final node representation. This node representation contains the node's own features, and also integrates the heterogeneous information and attributes in the graph, as well as the association between different paths.

[0029] In dynamic graphs, the receptive field of each node is affected by its position and connection structure, such as Figure 3 As shown in the figure. The size and shape of the receptive field depend on the number and connection mode of the node with its neighboring nodes. Therefore, different nodes face different degrees of difficulty in obtaining information about neighboring nodes. For nodes that are directly connected to many other nodes (i.e., densely connected nodes), their receptive fields are usually large and complex in shape. These nodes are able to obtain more extensive and comprehensive information because they can directly access a large number of neighboring nodes. Nodes only need to aggregate shallow neighbor information to obtain good global embedding features. This fast and efficient information processing method makes densely connected nodes have higher information sensitivity and response speed in dynamic graphs. However, for nodes with sparse connections (i.e., sparsely connected nodes), the situation is different. Due to the limited number of neighboring nodes, these nodes have greater difficulty in obtaining information. In order to capture more contextual information, sparsely connected nodes require higher levels of aggregated information. It is necessary to traverse longer paths or pass through more intermediate nodes to obtain sufficient information. This information processing method is inefficient and prone to introducing noise and errors.

[0030] In order to solve this problem, the present invention adopts an architecture that adaptively adjusts the degree of neighborhood aggregation information. The architecture mainly consists of two key parts: jump connection and adaptive information aggregation mechanism. Jump connection allows nodes to directly access information from deeper neighbor nodes without going through a complete information transfer process. The advantage is that sparsely connected nodes can better utilize contextual information and improve the accuracy of embedding. At the same time, through the adaptive information aggregation mechanism, the degree of information transmission and aggregation can be dynamically adjusted according to the connection mode and context requirements of the nodes. Jump connection is one of the core components in the architecture. It allows nodes to directly access information from deeper neighbor nodes when aggregating information without going through a complete information transfer process, and dynamically realizes adaptive embedding of information at different levels. It breaks the limitation of information transmission layer by layer in traditional graph neural networks, so that nodes can more directly obtain contextual information that has an important impact on their embedding and obtain more accurate embedding features. By establishing jump connections between different layers of the network, nodes can selectively skip some intermediate layers when aggregating information and directly utilize information from deeper neighbor nodes. Compared with the traditional fixed aggregation strategy, this method can flexibly adjust the degree of information transmission between different nodes, so as to better capture the local and global relationships. Obtain more representative and adaptive embedding features. In addition to jump connections, the architecture also introduces an adaptive information aggregation mechanism. This mechanism allows the dynamic adjustment of information transmission and aggregation according to the node connection mode and context requirements. Specifically, a learnable aggregation weight is assigned to each node, which reflects the degree of importance the node attaches to the information of neighboring nodes from different periods and levels. When aggregating information, these weights are multiplied and added with the embedded features of neighboring nodes from different levels to obtain the final embedded representation of the node. In this way, the adaptive integration of node neighborhood information is achieved, making the embedding result more representative and adaptive.

[0031] The model combines dynamic modeling and learning capabilities, uses the Attention mechanism to learn the weights of neighbor nodes, aggregates neighbor features and heterogeneous information onto paths, and then obtains the expression of the node itself based on the weighted sum of the paths. It can adapt to the dynamic changes of heterogeneous graph structures. It can not only extract feature representations of nodes and edges from graph data, but also perform adaptive learning based on the states at different times, thereby achieving more accurate and comprehensive graph analysis and prediction.

[0032] The pseudo code of a single HGAT layer program can be implemented in Python as follows:

[0033]

[0034]

[0035] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A dynamic heterogeneous graph neural network model, characterized in that: A type-based projection method is used to process node and edge features. An attention matrix that embeds the node type is defined for each node and edge type. The projection features for different node types are obtained through the attention matrix to achieve a unified representation of node and edge features.

2. A dynamic heterogeneous graph neural network model according to claim 1, characterized in that: The attention matrix maps the features of nodes and edges into a unified feature space and performs weighted fusion on them. By multiplying the features of nodes and edges with the attention matrix, projection features for different node types are obtained.

3. A dynamic heterogeneous graph neural network model according to claim 2, characterized in that When fusing the features of nodes and edges, the attention mechanism is used; a corresponding attention weight is assigned to each node and edge according to the type information of the node and edge; the weighted fused feature representation is obtained by multiplying and adding the features of the node and edge with the attention weight, and then the projection features for different node types are obtained, and the features of the nodes and edges are projected to the same distribution.

4. A dynamic heterogeneous graph neural network model according to claim 3, characterized in that: A path-based attention calculation method is used to process the heterogeneous information and attributes of the graph.

5. The path-based attention calculation method according to claim 4, characterized in that: First, a path neighbor is defined. The path neighbor of a node refers to the set of nodes connected to the node through a specific path. For each path neighbor node, its embedded features and the fusion features of heterogeneous information are calculated. The fusion feature is obtained by combining the embedded features of the path neighbor nodes with the edge attributes in the path; the embedded features of the path neighbor nodes are fused with heterogeneous information into the path feature to capture the structural information and association relationship between nodes, and it works through weighted edge attribute attention to dynamically adjust the contribution of the feature.

6. A dynamic heterogeneous graph neural network model according to claim 5, characterized in that: By calculating the attention coefficient between paths, the attention coefficient is obtained by calculating the similarity between the fusion features of the path neighbor nodes and the current node representation; the higher the similarity, the larger the attention coefficient, indicating that the path contributes more to the current node representation; the lower the similarity, the smaller the attention coefficient, indicating that the path contributes less to the current node representation, and the final embedded features are output.

7. A dynamic heterogeneous graph neural network model according to claim 6, characterized in that: The size and shape of each node's receptive field depends on the number and mode of connections between the node and its neighboring nodes; an architecture that adaptively adjusts the degree of neighborhood aggregation information is used to capture information.

8. A dynamic heterogeneous graph neural network model according to claim 7, characterized in that: The architecture for adaptively adjusting the degree of neighborhood aggregation information consists of a skip connection and an adaptive information aggregation mechanism; the skip connection allows a node to directly access information from a deeper level of neighboring nodes without going through a complete information transfer process; Through the adaptive information aggregation mechanism, the information transmission and aggregation degree are dynamically adjusted according to the node connection mode and context requirements.

Images