A large-scale heterogeneous graph-oriented graph neural network sampling method

Abstract

The application relates to a graph neural network sampling method for a large-scale heterogeneous graph. The method first loads graph data and re-labels a node set; then initializes a probability matrix of each node under different types; performs expansion of different layers according to a neighborhood expansion rule, and performs normalization according to the probability matrix while considering feature completeness to calculate a sampling node; next, a sampling subgraph is constructed according to the sampling node, and missing features are completed; finally, a sampling subgraph capable of being used for subsequent heterogeneous graph neural network training is obtained.

Description

Technical Field

[0001] This invention provides a graph neural network sampling method for large-scale heterogeneous graphs, belonging to the field of deep learning. Background Technology

[0002] As of December 2021, my country's internet user base reached 1.032 billion, with an internet penetration rate of 73.0%, and the mobile internet is showing a new development trend. With the increasing prevalence of the internet and the continuous increase in user data, higher demands are being placed on user behavior analysis and user category differentiation. In the pursuit of better classification and prediction results, the number of model parameters is constantly increasing, training time is gradually increasing, and the demand for computing resources is even greater.

[0003] In recent years, graph neural networks (Graph Neural Networks) have proven to be a powerful tool for analyzing non-Euclidean graph data, and have been widely applied in fields such as search, recommendation, and risk control. However, training a Graph Neural Network model requires calculating the feature representations of all nodes in each layer. As the scale of the graph grows, computational resources are clearly insufficient to fully load the node representations of a large-scale dataset at once. To address this, some works have proposed different sampling schemes to obtain more sampled subgraphs for training. However, most works consider obtaining subgraphs from homogeneous graphs for training the Graph Neural Network. In reality, most data is heterogeneous, adding additional node types and relational features. Because the quantity and distribution of these heterogeneous features are uneven, and homogeneous graph sampling methods often consider degree features, ignoring heterogeneous features, the sampling process for subgraphs may result in an imbalance or complete absence of certain types of features. Therefore, the obtained subgraph sampling results are unreasonable, leading to prolonged model training time and potentially poor results. Thus, better sampling of heterogeneous Graph Neural Networks under large-scale data is an important research topic.

[0004] Current sampling methods for graph neural networks are mostly based on importance sampling strategies. They compute an importance representation for each node and then perform importance sampling based on probability. Most works rely on sampling neighboring nodes and utilize various types of aggregation functions to improve the model's expressive power. Typical methods include: node sampling, which simulates a random walk starting from the target node to achieve a subset of nodes in the subgraph; layer sampling, which retains a small number of nodes for aggregation during neighborhood exploration at different layers to control the expansion factor and perform importance sampling; and subgraph sampling, which samples multiple subgraphs and generates subgraphs using graph clustering.

[0005] The aforementioned works are all based on the assumption of homogeneous graphs. There are few sampling methods for heterogeneous graphs. The known heterogeneous graph sampling method is HG Sampling, which mainly aims to set separate weights for each type and sample the same number of nodes for each type to obtain a subgraph. This approach performs poorly with low sampling layers and low batch sizes. The method aims for roughly the same number of nodes for each type, but ignores the difference between the sampled subgraph and the actual scale in reality. This results in subgraphs with shallow neighborhood expansion layers having structural differences compared to the full graph. On real-world distributed datasets, it ignores the mutual influence between different types of nodes, requiring more iterations to achieve similar accuracy. The sampling process only considers homogeneity information, leading to a certain degree of isolation in the relationships between different types. Therefore, this paper proposes a heterogeneous graph sampling method that considers global structural features, the interaction between different types, and has certain data feature completion characteristics for large-scale data. Summary of the Invention

[0006] Technical problems to be solved

[0007] To avoid the shortcomings of existing technologies, this invention provides a graph neural network sampling method for large-scale heterogeneous graphs.

[0008] Technical solution

[0009] A graph neural network sampling method for large-scale heterogeneous graphs, characterized by the following steps:

[0010] Step 1: Load graph data and initialize node set;

[0011] Step 2: Initialize the weight matrix of the sampling probability for each node;

[0012] Step 3: In the current layer, normalize and accumulate the sampling probability matrix obtained in Step 2; the calculation method is as follows: in The cumulative normalized probability is represented by τ, the type is represented by len, the maximum length of the adjacency matrix is ​​represented by φ, and the edge type is represented by φ. Multiple rounds of neighborhood exploration are performed until the depth reaches the user-defined number of layers. Then, the importance of the graph is sampled based on the probability of the existing nodes.

[0013] Step 4: Construct a sampling graph based on the point set of the sampling nodes;

[0014] Step 5: Based on the point set of the sampled nodes, obtain the feature information of the corresponding nodes from the original data and construct a feature matrix; the node features in most datasets are not complete, so feature completion is performed for the missing node feature information;

[0015] Step 6: Combine the sampled graph obtained in Step 4-3 and the feature completion information obtained in Step 5, which are the processed subgraph and feature information.

[0016] A further technical solution of the present invention: Step 1 is specifically as follows:

[0017] Step 1-1: Set a unique ID for all nodes, numbered starting from 0;

[0018] Step 1-2: Initialize the node set and denote it as sample_nodes: According to the value batch_size given by the user, randomly select an initial number of nodes equal to batch_size, and store the initial nodes in the form of <ID, number of features included in the node> into the sampling node result set sample_nodes of the current layer as the initial random sampling result;

[0019] Step 1-3: Count the number of edges of each type in the whole graph and the proportion of each type in the total number of edges in the whole graph, and store them in the form of a dictionary, which is called the initial node proportion set sample_dict, and the recorded information is: <starting node type, edge type, ending node type, proportion value>.

[0020] A further technical solution of the present invention: Step 2 is specifically as follows:

[0021] Step 2-1: Calculate the sampling number of probability matrices of each type in the current layer

[0022] Step 2-1-1: According to the initial node set sample_nodes obtained in Step 1-2, count the proportion of the number of edge types of the first-order neighbors around the node set and store it in the form of a dictionary, which is called the layer node proportion set neighbor_dict, and the recorded information is: <starting node type, edge type, ending node type, proportion value>;

[0023] Step 2-1-2: According to the initial node proportion set sample_dict obtained in Step 1-3 and the records of the layer node proportion set neighbor_dict obtained in Step 2-1-1, calculate the arithmetic average of the corresponding proportion values according to <starting node type, edge type, ending node type> to obtain the number of samples of each type of edge in this layer; In low-order sampling, using an equilibrium method, the sampling proportions of each type can better approximate the real situation of the original graph, can improve the type proportion, and the lower-layer model training can reach the convergence target faster;

[0024] Step 2-2: Process the neighbors of each node according to their categories, determine the weights based on the degree information of the nodes, and calculate the probability matrix for each node type; process the edges of each type separately, and evaluate the probability of each node. The evaluation method is to accumulate the degree centrality degree in sequence, and the accumulation method is the reciprocal of the number of surrounding neighbors.

[0025] Further technical solution of the invention: Step 3 is as follows:

[0026] Step 3-1: Normalize the probability matrices for each node type obtained in Step 2-2;

[0027] Step 3-2: Take a weighted average of the normalized probabilities and the number of features of each node. By reducing the bias through a weighted estimate of the mean, a new sampling probability for each node is obtained.

[0028] Step 3-3: Based on the sampling probabilities of each node obtained in Step 3-2 and the number of samples of each type, perform sampling; and update the sampling probability matrix based on the currently obtained point set. The matrix update method is sequential accumulation, i.e. Where n represents the current node, τ represents the type, and S represents the cumulative sampling probability of the current node n under type τ. This represents the accumulated normalized probability. The matrix value is updated repeatedly until a specified depth is reached, at which point the matrix update ends.

[0029] A further technical solution of the present invention: Step 4 is as follows:

[0030] Step 4-1: In step 3-3, the sampled point set is obtained. Each node has an identifier ID, which is uniquely determined in step 1-1. Recording these IDs will yield the sampled node set.

[0031] Step 4-2: Based on the node set in Step 4-1, and according to the connectivity between two points, the corresponding edge relationship, i.e., the edge set, can be obtained;

[0032] Step 4-3: Based on the node set obtained in Step 4-1 and the edge set obtained in Step 4-2, the sampling graph can be constructed.

[0033] A further technical solution of the present invention: Step 5 is as follows:

[0034] Step 5-1: Based on the sampling map obtained in Step 4, find the nodes with missing feature information and list them as a point set;

[0035] Step 5-2: Based on the point set obtained in Step 5-1, complete the set according to the actual dependency relationship and feature transfer method.

[0036] Beneficial effects

[0037] A graph neural network sampling method for large-scale heterogeneous graphs provided by the present invention first loads graph data and relabels the node set; then initializes the probability matrix of each node under different types; performs expansion of different layers according to the neighborhood expansion rule, normalizes according to the probability matrix, and considers feature completeness to calculate the sampled nodes; next, constructs a sampled subgraph based on the sampled nodes and completes the missing features; finally, obtains a sampled subgraph that can be used for subsequent training of the heterogeneous graph neural network.

[0038] The present invention can be used under large-scale data, taking into account the global structural features, enabling better performance under the limitation of a smaller neighborhood exploration range, considering the interactions between different types of nodes and edges at the same time, and being able to quickly complete some missing data features after obtaining the sampled subgraph, and finally providing it to different heterogeneous graph neural network models for subsequent training. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings are only for the purpose of illustrating specific embodiments and are not considered to be a limitation of the present invention. Throughout the drawings, the same reference signs represent the same components.

[0040] Figure 1 It is a schematic diagram of the distribution of heterogeneous graph nodes before and after sampling;

[0041] Figure 2 It is a schematic diagram of data completion of the realistic dependency relationship in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0043] Step 1: Load the graph data and complete the initialization of the result node set;

[0044] Step 1-1: Set a unique ID for all nodes, starting from 0;

[0045] Step 1-2: Initialize the node set as sample_nodes: randomly select an initial node with a number of batch_size according to the value batch_size given by the user, and store the initial nodes in the form of <ID, the number of features included in the node> into the sampled node result set sample_nodes of the current layer as the initialization random sampling result;

[0046] Steps 1-3: Count the number of edges of each type in the entire graph and the proportion of each type to the total number of edges in the entire graph. Store the data in the form of a dictionary, called the initial node proportion set sample_dict. The information recorded is: <starting node type, edge type, ending node type, proportion value>.

[0047] Step 2: Initialize the weight matrix of the sampling probability for each node;

[0048] Step 2-1: Calculate the number of samples for each type of probability matrix in the current layer. Since the proportion of nodes of different types is uneven, sampling a fixed number of nodes of each type will lead to an imbalance in the number of neighbors of different categories for a single node when the sampling depth is low. Therefore, based on the neighbor proportions of the initial node set in the current layer, i.e., the sample_dict obtained in Step 1-3, we need to adjust the proportion of different types of edges during each sampling process using the methods described in Steps 2-1-1 and 2-1-2 below. This will ensure that the graph contains more type information, resulting in better sampling results even at low sampling depths and improving computational efficiency.

[0049] Step 2-1-1: Based on the initial node set sample_nodes obtained in Step 1-2, count the proportion of edge types occupied by the first-order neighbors around the node set, and store it in the form of a dictionary, called the layer node proportion set neighbor_dict. The recorded information is: <starting node type, edge type, ending node type, proportion value>.

[0050] Step 2-1-2: Based on the initial node proportion set `sample_dict` obtained in Step 1-3 and the layer node proportion set `neighbor_dict` obtained in Step 2-1-1, calculate the arithmetic average of each record according to the corresponding proportion values ​​of <starting node type, edge type, ending node type> to obtain the number of edges of each type that should be sampled in this layer. In low-order sampling, using a balanced approach, the sampling proportions of each type can better approximate the real situation of the original image, improving the type proportions and allowing the lower-layer model training to reach the convergence target faster.

[0051] Step 2-2: Process the neighbors of each node according to their categories, determine the weights based on the degree information of the nodes, and calculate the probability matrix for each node type. Process the edges of each type separately, and evaluate the probability of each node. The evaluation method is to accumulate the degree centrality degree in sequence, and the accumulation method is the reciprocal of the number of surrounding neighbors.

[0052] Step 3: In the current layer, normalize and accumulate the sampling probability matrices for each category obtained in Step 2-2. The calculation method is as follows: in The function represents the accumulated normalized probability, τ represents the type, len represents the maximum length of the adjacency matrix, and φ represents the edge type. Multiple rounds of neighborhood exploration are performed until the depth reaches the user-defined level. Then, weighted sampling of the graph is performed on the existing nodes based on the probability.

[0053] Step 3-1: Normalize the probability matrices for each node type obtained in Step 2-2;

[0054] Step 3-2: Calculate a weighted average of the normalized probabilities obtained for each node and the number of node features. This weighted estimate, based on the mean, reduces bias and yields new sampling probabilities for each node. The main purpose of this approach is to reduce the sampling probability of nodes with missing information through a weighted method.

[0055] Step 3-3: Based on the sampling probabilities of each node obtained in Step 3-2 and the number of samples of each type, perform sampling. Then, update the sampling probability matrix based on the currently obtained point set. The matrix update method is sequential accumulation, i.e. Where n represents the current node, τ represents the type, and S represents the cumulative sampling probability of the current node n under type τ. Represents the cumulative normalized probability. The specific calculation method is as described in step 3. Repeat the matrix value update multiple times until the specified depth is reached, then end the matrix update process.

[0056] Step 4: Construct a sampling graph based on the point set of the sampling nodes;

[0057] Step 4-1: In step 3-3, the sampled point set is obtained. Each node has an identifier ID, which is uniquely determined in step 1-1. Recording these IDs will yield the sampled node set.

[0058] Step 4-2: Based on the node set in Step 4-1, and according to the connectivity between two points, the corresponding edge relationship, i.e., the edge set, can be obtained;

[0059] Step 4-3: Based on the node set obtained in Step 4-1 and the edge set obtained in Step 4-2, the sampling graph can be constructed;

[0060] Step 5: Based on the point set of the sampled nodes, extract the feature information of the corresponding nodes from the original data and construct a feature matrix. Since node features in most datasets are incomplete, feature completion is performed for missing node feature information.

[0061] Step 5-1: Based on the sampling map obtained in Step 4, find the nodes with missing feature information and list them as a point set;

[0062] Step 5-2: Based on the point set obtained in Step 5-1, complete the set according to the actual dependency relationship and feature transfer method;

[0063] Step 5-2-1: Feature completion based on real-world dependencies. This method is only applicable to some datasets and requires completion based on real-world logical relationships to achieve high efficiency. It is often related to time characteristics. For example, in social networks, the main text and comments have a chronological order. If the time information of the main text node is missing, the earliest time information of the comment node can be used for feature completion. In citation datasets, if the time information of the article node is missing, the time can be approximated by the relationship between the citation and cited paper nodes.

[0064] Assuming that in the citation dataset, the in-degree relationship is citation and the out-degree relationship is being cited; then the specific feature completion method is as follows: take the node with missing information and find all surrounding nodes with corresponding time information; record the value with the largest time among the in-degree nodes as A, which is equivalent to the latest time of the cited article; record the value with the largest time among the out-degree nodes as B, which is equivalent to the earliest time of being cited; take the average of A and B, which can be used as the time feature information of the current node; in other cases, different reality dependency methods need to be designed.

[0065] Step 5-2-2: Information Completion Based on Feature Propagation: Since neighbors often have similar feature vectors, the propagation property of feature matrices of similar nodes can be quantified using Dirichlet Energy. The specific completion method is as follows: measure the quadratic form of the squared difference between the node's feature and the average of its neighbors; reconstruct missing features using the correlation of feature propagation; the propagation process involves initializing with unknown features, then propagating the features by applying a normalized adjacency matrix, and finally resetting the known features to their true values. Repeat these two operations until the feature vector converges.

[0066] Step 6: Combining the sampled image obtained in Step 4-3 with the feature completion information obtained in Step 5, we get the processed sub-image and feature information.

[0067] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention.

Claims

1. A large-scale heterogeneous graph-oriented graph neural network sampling method, characterized in that The steps are as follows: Step 1: load graph data, initialize node set; the graph data is citation data; the details are as follows: Step 1-1: set a unique ID for all nodes, starting from 0; Step 1-2: initialize the node set represented as sample_nodes: according to the user-given value batch_size, randomly select batch_size initial nodes, and store the initial nodes in the form of <ID, node feature number> to the sample node result set sample_nodes of the current layer as the initial random sampling result; Step 1-3: count the number of each type of edge in the whole graph, and the proportion of each type in the total number of edges in the whole graph, stored in the form of a dictionary, called the initial node proportion set sample_dict, recording information as: <starting node type, edge type, ending node type, proportion value>; Step 2: initialize the weight matrix of the sampling probability of each node; the details are as follows: Step 2-1: calculate the sampling number of each type of probability matrix in the current layer Step 2-1-1: according to the initial node set sample_nodes obtained in step 1-2, count the proportion of the type of edge occupied by the first-order neighbor of the node set, stored in the form of a dictionary, called the layer node proportion set neighbor_dict, recording information as: <starting node type, edge type, ending node type, proportion value>; Step 2-1-2: according to the initial node proportion set sample_dict obtained in step 1-3 and the layer node proportion set neighbor_dict obtained in step 2-1-1, the corresponding proportion values of each record according to <starting node type, edge type, ending node type> are calculated, and the number of each type of edge that should be sampled in this layer is obtained; in low-order sampling, the sampling proportion of each type can better approach the real situation of the original graph, and the type proportion can be improved, and the lower model training can reach the convergence target faster; Step 2-2: process the neighbors according to the category, decide the weight according to the degree information of the node, and calculate the probability matrix under each node type; process each type of edge and evaluate the probability of each node, and the evaluation method is to accumulate in turn according to the degree centrality, and the accumulation method is the reciprocal of the number of peripheral neighbors; Step 3: in the current layer, according to the sampling probability matrix obtained in step 2, normalize and accumulate; The calculation method is: wherein represents the cumulative normalized probability, represents the type, represents the maximum length of the adjacency matrix, represents the edge type; the neighborhood exploration is performed in multiple rounds until the depth reaches the user-set number of layers; and then the importance sampling of the graph is performed on the current existing nodes according to the probability; the specific process is as follows: Step 3-1: normalize the probability matrix under each node type obtained in step 2-2; Step 3-2: weight average the normalized probability of each node and the node feature number, reduce the bias through weighted estimation of the mean, and obtain the new sampling probability of each node; Step 3-3: Based on the sampling probabilities of each node obtained in Step 3-2 and the number of samples of each type, perform sampling; and update the sampling probability matrix based on the currently obtained point set. The matrix update method is sequential accumulation, i.e. Where n represents the current node, Indicates type, Indicates the current node ,exist Cumulative sampling probability under type This represents the accumulated normalized probability. The matrix value is updated repeatedly until a specified depth is reached, at which point the matrix update ends. Step 4: construct a sample graph according to the point set of the sampled nodes; the details are as follows: Step 4-1: in step 3-3, the sampled point set is obtained, each node has an identification ID, which is uniquely determined in step 1-1, and the ID is recorded to obtain the sampled node set; Step 4-2: According to the node set in step 4-1, the corresponding edge relationship is obtained according to the connection relationship between two points, that is, the edge set; Step 4-3: According to the node set obtained in step 4-1 and the edge set obtained in step 4-2, a sampling graph can be constructed; Step 5: According to the point set of the sampling node, the feature information of the corresponding node is obtained from the original data to construct a feature matrix; Most node features in the data set are not complete, and the missing node feature information is completed; Specifically as follows: Step 5-1: According to the sampling graph obtained in step 4, find the node with missing feature information and list it as a point set; Step 5-2: According to the point set obtained in step 5-1, complete it according to the real dependence relationship and feature transmission mode; Step 5-2-1: According to the feature completion of the real dependence relationship, this method is only suitable for part of the data set, and the data set needs to be completed according to the real logic relationship, which has the characteristics of high efficiency; It is related to time characteristics, in the citation data set, if the article node time information is missing, the time approximation is performed according to the relationship between the cited and cited paper nodes; In the citation data set, the in-degree relationship is reference, and the out-degree relationship is cited; Then the specific feature completion method is: taking the current node with missing information, find all nodes with corresponding time information in the neighborhood; Record the maximum value of the time in the in-degree node as A, which is equivalent to the latest time of the cited article; Record the maximum value of the time in the out-degree node as B, which is equivalent to the earliest time of the cited article; Take the average of A and B as the time feature information of the current node; In other cases, different real dependence methods need to be designed; Step 5-2-2: According to the feature transmission mode, the information is completed: since the neighbors often have similar feature vectors, the Dirichlet energy is used to quantify the propagation of the feature matrix of the same type node; The specific completion method is: measure the quadratic form of the square difference between the node feature and its neighbor average value; The correlation of feature propagation can reconstruct the missing feature; The propagation process is to use the unknown feature to initialize, and then propagate the feature by applying the normalized adjacency matrix, and then reset the known feature to its true value; Repeat the above two operations until the feature vector converges; Step 6: Combine the sampling graph obtained in step 4-3 and the feature completion information obtained in step 5 to obtain the processed subgraph and feature information.

Images