Heterogeneous graph sampling method and accelerator
By maintaining the metapath dictionary tree during heterogeneous graph sampling and using semantic redundancy detection and reusable grouping, the problem of inefficient sampling of heterogeneous graph data is solved, and efficient sampling and inference performance is achieved.
Patent Information
- Application Number
- CN202510768919.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-10
- Publication Date
- 2025-09-02
AI Technical Summary
The prior art is difficult to efficiently process the sampling process of heterogeneous pattern data, resulting in low sampling efficiency and becoming a performance bottleneck for small-batch inference of heterogeneous pattern neural networks.
By maintaining the metapath dictionary tree, recording the sampled metapaths, using semantic redundancy detectors to detect overlapping parts, multiplexing the sampled results, and optimizing the sampling process with a reusability-driven semantic grouping method.
It effectively reduces the overhead of off-chip memory access, improves sampling efficiency and performance, and realizes efficient processing of heterogeneous graph data.
Smart Images
Figure CN120579643A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computing technology, in particular to the field of heterogeneous graph processing, and more particularly to a method and accelerator for heterogeneous graph sampling. Background Art
[0002] The introduction of graph neural networks (GNNs) addresses the need for processing irregular data in non-Euclidean spaces. However, in real-world scenarios, many real-world graphs naturally represent heterogeneous graphs, meaning they contain more than one type of node and edge. Traditional GNNs cannot efficiently process these graphs. Therefore, heterogeneous graph neural networks (HGNNs) were proposed to efficiently process both the structural and semantic characteristics of heterogeneous graph data.
[0003] As the scale of graph data in the real world continues to grow, it is becoming increasingly unrealistic to perform full-graph reasoning on complete graph data due to the limitations of hardware resources, especially storage resources. This makes the reasoning paradigm based on mini-batch sampling the de facto standard for HGNN reasoning for large-scale graph data.
[0004] Most mainstream HGNNs use a semantic representation composed of multiple adjacency relationships, also known as a meta-path, to enhance the model's ability to capture semantic information. Mini-batch HGNN inference requires first performing a meta-path-based sampling process to construct a set of operations for inference. Existing techniques typically perform this sampling process on traditional computing platforms (such as CPUs and GPUs). However, due to significant memory bottlenecks and complex operation branches, this sampling process is difficult to efficiently handle on traditional computing platform architectures, making sampling a performance bottleneck for small-batch HGNN inference. GNN acceleration technologies, such as CoGNN (An Algorithm-Hardware Co-Design Approach to Accelerate GNN Inference With Minibatch Sampling), propose a method to uniformly sample target nodes with more common neighbors to improve on-chip data locality, thereby accelerating the GNN sampling process. However, because the HGNN sampling process involves multiple adjacency relationships and multiple semantics, it is more complex than the sampling process of traditional GNNs. Therefore, traditional GNN sampling optimization methods cannot be effectively applied directly to HGNN sampling optimization.
[0005] The above-mentioned existing technical means cannot be directly used to accelerate small-batch HGNN reasoning due to the lack of efficient processing capabilities and optimization methods for heterogeneous graph data. Therefore, there is a technical gap in the field of small-batch sampling optimization for HGNN, which cannot meet the efficient processing requirements for HGNN.
[0006] The heterogeneous graph sampling paradigm used on traditional platforms employs a per-element path execution process. On the one hand, due to the overlap between longer and shorter meta-paths, this leads to a large number of redundant traversal paths during the sampling process. On the other hand, the per-element path sampling paradigm fails to exploit the inherent semantic parallelism between different semantics. These shortcomings of the existing technology lead to inefficient heterogeneous graph sampling and become a major performance bottleneck in end-to-end HGNN small-batch execution.
[0007] It should be noted that this background information is intended solely to introduce relevant information related to the present invention to facilitate understanding of the present invention's technical solution. It does not necessarily constitute prior art. Relevant information submitted and disclosed together with the present invention's solution should not be considered prior art unless there is evidence that the relevant information was disclosed prior to the filing date of the present invention. Summary of the Invention
[0008] Therefore, the object of the present invention is to overcome the above-mentioned defects of the prior art and provide a method and accelerator for heterogeneous graph sampling.
[0009] The purpose of the present invention is achieved through the following technical solutions:
[0010] According to a first aspect of the present invention, a method for sampling a heterogeneous graph is provided, comprising: obtaining a batch of target nodes and a set of meta-paths; maintaining a meta-path dictionary tree for recording sampled meta-paths for each batch of target nodes; selecting meta-paths from the meta-path set according to a predetermined order, sampling the target nodes on the graph data in sequence, and updating the meta-path dictionary tree, wherein the predetermined order includes ordering the paths from short to long. Before each sampling, determining whether there is semantic overlap based on the currently selected meta-path and the meta-path dictionary tree; if so, completing the sampling process corresponding to the currently selected meta-path based on the sampling results of the overlapping portion. This solution can at least achieve the following beneficial technical effects: by maintaining a meta-path dictionary tree to record shorter meta-paths that have been sampled, and directly reusing the sampling results of the shorter meta-path when semantic redundancy is found, the present invention can effectively reduce the off-chip memory access overhead of the sampling process and improve sampling efficiency. In addition, because the sampling process reuses the sampling results of the short path, the memory access overhead is also reduced when the graph data is subsequently processed based on the sampled results.
[0011] Optionally, completing the sampling process corresponding to the currently selected meta-path based on the sampling results of the overlapping portion includes: obtaining each sampling result that has been sampled in the overlapping portion, extracting the terminal node of each sampling result as the target alternative node; starting from the target alternative node, sampling backward based on the non-overlapping portion of the currently selected meta-path and the meta-path dictionary tree to obtain the sampling result corresponding to the target node. This solution can at least achieve the following beneficial technical effects: when the current meta-path is found to have an overlapping portion with the meta-path dictionary tree, the solution no longer starts sampling from the original target node, but instead samples from the terminal node target alternative node in the previously sampled sampling result, thereby reducing repeated sampling overhead.
[0012] Optionally, the method further includes: after completing sampling for each target node based on all meta-paths in the meta-path set, obtaining the next batch of target nodes and meta-path sets, and initializing a meta-path dictionary trie. This solution can achieve at least the following beneficial technical effects: maintaining a separate dictionary trie for each batch of target nodes can prevent a single dictionary trie from being too large, reduce the hardware overhead required to maintain the dictionary trie, and ensure the efficiency of discovering semantic overlap based on the dictionary trie.
[0013] Optionally, the method further includes: obtaining a total set of metapaths, which includes all metapaths used to guide the sampling process, each metapath being a semantic path pattern composed of a node type and an edge type; and dividing the total set of metapaths into multiple groups with the optimization goal of maximizing the reusability of the paths within the group, with the metapaths in each group constituting a metapath set. This solution can at least achieve the following beneficial technical effects: after grouping in this manner, the metapaths in each metapath set can be made as reusable as possible, realizing a new type of reusability-driven semantic grouping method. Compared with random grouping, off-chip memory accesses are further reduced, thereby improving sampling performance.
[0014] Optionally, the process of dividing the total set of meta-paths into multiple groups includes: determining a reusability index for each two meta-paths in the total set of meta-paths, where a larger value of the index indicates that the shorter meta-path matches closer to the front of the longer meta-path; and using a grouping algorithm to group the total set of meta-paths and the reusability indexes between the meta-paths to obtain multiple groups. This solution can at least achieve the following beneficial technical effects: for meta-paths with a reuse relationship, a closer matching position means greater reusability, for example, PAP has greater reusability for PAPSP than PSP; therefore, by setting this reusability index, the subsequent grouping algorithm can be made more inclined to group meta-paths with higher reusability into the same group, thereby making the meta-paths within the group have higher reusability.
[0015] Optionally, the reusability index can be calculated by:
[0016]
[0017] in, Indicates the position coordinates of the element in the longer meta-path where the shorter meta-path matches the prefix of the longer meta-path. Indicates the length of the longer meta-path. This solution can achieve at least the following beneficial technical effects: the reusability index can make the index value of the shorter meta-path at a closer matching position larger than that of the longer meta-path, so that subsequent grouping can be more accurate.
[0018] According to a second aspect of the present invention, an accelerator for implementing the method described in the first aspect is provided, comprising: a semantic redundancy detector, configured to: obtain a batch of target nodes and meta-path sets, and maintain a meta-path dictionary tree for recording the sampled meta-paths for each batch of target nodes; select meta-paths from the meta-path set according to a predetermined order requirement, determine whether there is semantic overlap based on the currently selected meta-path and the meta-path dictionary tree, and obtain a judgment result; and after the sampling process corresponding to the currently selected meta-path is completed, record it in the meta-path dictionary tree; a neighbor sampling unit, configured to: perform sampling according to the judgment result, comprising: if the judgment result is yes, completing the sampling process corresponding to the currently selected meta-path based on the sampling results of the sampled overlapping part; if the judgment result is no, gradually completing the corresponding sampling process according to the complete sampling path indicated in the currently selected meta-path.
[0019] Optionally, the semantic redundancy detector includes a semantic decoder and a type matcher, wherein: the semantic decoder is used to convert each meta-path in the dictionary tree and the currently selected meta-path into a bit string; the type matcher is used to determine whether there is semantic overlap based on the numerical comparison of the bit strings. If the bit string of the currently selected meta-path is consistent with the bit string of a meta-path in the dictionary tree, there is semantic overlap; if the bit string of the currently selected meta-path is different from the bit strings of all meta-paths in the dictionary tree, there is no semantic overlap. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:
[0021] Figure 1 Schematic diagram of heterogeneous graph and small batch sampling process;
[0022] Figure 2 Schematic diagram of the HGNN inference prediction execution process;
[0023] Figure 3 1 is a flow chart of a method for heterogeneous graph sampling according to an embodiment of the present invention;
[0024] Figure 4 A schematic diagram of a semantic hypergraph modeling and grouping example according to an embodiment of the present invention;
[0025] Figure 5 Schematic diagram of a technical overview of a method for heterogeneous graph sampling according to an embodiment of the present invention;
[0026] Figure 6 A schematic diagram of an example of sampling of a redundancy elimination algorithm according to an embodiment of the present invention;
[0027] Figure 7 A schematic diagram of the hardware structure of an accelerator according to an embodiment of the present invention;
[0028] Figure 8 Schematic diagram of experimental results according to an embodiment of the present invention. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0030] As mentioned in the background technology section, the sampling process of the data required by heterogeneous graph neural networks is inefficient and becomes the main performance bottleneck in the end-to-end HGNN small-batch execution process. In response to this, the method of the present invention maintains a meta-path dictionary tree for the sampled meta-paths of a batch of target nodes to be sampled, and is used to analyze whether the subsequently selected meta-path has semantic overlap with the meta-path dictionary tree. If so, the sampling process corresponding to the currently selected meta-path is completed based on the sampling results of the overlapping part, so as to significantly reduce the off-chip random memory access caused by semantic redundancy.
[0031] Reasoning on large-scale heterogeneous graphs mainly includes two stages: sampling process and neural network reasoning. In order to facilitate the understanding of the solution of the present invention, the schematic execution flow of the two stages is first given, as shown below. Figure 1 and Figure 2 shown.
[0032] Sampling process: Figure 1In this paper, we use a simple heterogeneous graph as an example to illustrate the mini-batch sampling process. This heterogeneous graph contains three types of nodes: paper (P), author (A), and subject (S), as well as three types of edges, or adjacency relationships: author---paper, paper---cited---paper, and paper---belongs---subject. In addition to direct adjacency relationships, mainstream HGNN models typically use a combination of multiple adjacency relationships to form meta-paths to capture higher-order semantic information. For example, the meta-path PSP (abbreviated as PSP) indicates that the starting and ending papers belong to the same topic; PAP indicates that the starting and ending papers share a common author; and PAPSP indicates that the starting and ending papers share a common author and belong to the same subject area. The process of mini-batch sampling a heterogeneous graph according to meta-paths involves starting from a target node, following the meta-path definition, and searching for a corresponding type of neighbor each time through the traversal. For example, the sampling process based on the PSP meta-path starts from a target node P (marked in red in the figure), finds all its S-type neighbors and randomly selects one (marked in red in the figure). ), and then from the selected neighbor node Start, find all its P-type neighbors and randomly select one (in the figure ), a sampling traversal path based on the PSP meta-path is obtained. Figure 1 The sampling results with a maximum number of sampled neighbors of 2 under the PSP / PAP / PAPSP meta-path are given respectively. The end node of the traversal path is the semantic neighbor node sampled according to the corresponding meta-path.
[0033] In the entire HGNN small batch inference process described above, the stage of sampling according to the meta-path is the main performance bottleneck, accounting for more than 60% of the execution time on average. The main behavioral characteristics of this stage are traversing the graph structure according to the predefined meta-path, with a large amount of random memory access and subject to memory access constraints. However, according to Figure 1 It can be seen that semantic redundancy exists between different meta-paths. For example, the longer meta-path PAPSP actually contains the shorter meta-path PAP. Therefore, there are duplicate memory accesses during the sampling process of the longer meta-path and the shorter meta-path. Furthermore, the traditional execution paradigm uses a semantic-by-semantic execution paradigm, sampling all semantics sequentially, sampling PSP after PSP, ignoring the inherent sampling parallelism between semantics.
[0034] Neural network reasoning: After sampling the corresponding neighbors, the reasoning and prediction process of HGNN is as follows Figure 2Generally speaking, the reasoning execution process of an HGNN can be expressed as follows:
[0035]
[0036] in, Target node The eigenvector of Neighbor nodes The eigenvector of is a set of meta-paths, For a specific meta-path, Target node In the meta path The neighbor set below, is the weight matrix of the feature map. Schematically, for each meta-path, several semantic graphs are sampled, each of which corresponds to a semantic. For example, the first semantic graph corresponds to the semantics of PSP. In each semantic graph, the first execution stage of HGNN, namely the feature mapping stage, is responsible for transforming the original feature vectors of all nodes through a linear transformation, that is, transforming the original feature vectors with the weight matrix Matrix multiplication is performed to convert the feature vectors into new ones in the same vector space. The second execution phase, the neighbor aggregation phase, aggregates the feature vectors of all neighbor nodes to the target node through element-by-element or weighted addition. After the neighbor aggregation phase is completed across all semantic graphs, the final execution phase, semantic aggregation, fuses the aggregated neighbor feature vectors of the target node obtained from different semantic graphs to form the final embedded representation of the target node. This concludes the inference execution process of the HGNN itself. The subsequent prediction process is similar to that of other neural networks. For example, for the simple downstream task of binary node classification, after obtaining the embedded representation of the target node, a multi-layer perceptron is first used to convert the embedded representation into a classification vector with only two elements. The node's class is then determined based on the value distribution of the two elements.
[0037] According to one embodiment of the present invention, see Figure 3 , provides a method for heterogeneous graph sampling, including steps S1, S2, S3, S4, and S5. In order to better understand the present invention, each step is described in detail below in conjunction with specific embodiments.
[0038] Step S1: Obtain the total set of meta-paths, which includes all meta-paths used to guide the sampling process. Each meta-path is a semantic path pattern composed of node types and edge types.
[0039] According to one embodiment of the present invention, the meta-path set is a collection of all meta-paths used to guide the sampling process of related heterogeneous graph data. Schematically, in the scenario of capturing different semantic relationships between papers, the meta-path set is a collection of semantic path patterns between papers that need to be paid attention to. For example, some common meta-paths include: PAP: stands for paper-author-paper, PSP: stands for paper-topic-paper, PAPSP: stands for paper-author-paper-topic-paper, and PASP: stands for paper-author-topic-paper.
[0040] Step S2: Taking maximizing the reusability of the paths within the group as the optimization goal, the total set of meta-paths is divided into multiple groups, and the meta-paths in each group constitute a meta-path set.
[0041] Due to the large number of meta-paths, if the existing approach of directly randomly grouping the entire meta-path set and then sampling each path individually is used, this simple random grouping method cannot guarantee the good reusability of the meta-paths within the group. The sampling results corresponding to some shorter meta-paths in the early stage will not be able to be used in the sampling process for longer meta-paths, resulting in a large number of repeated memory accesses, which may affect the effectiveness of the semantic redundancy-aware sampling method. To exploit the inherent parallelism between semantics, this step considers grouping based on semantic reusability to achieve semantic parallel processing.
[0042] According to one embodiment of the present invention, the process of dividing the total set of meta-paths into multiple groups includes: determining a reusability index for each two meta-paths in the total set of meta-paths, where a larger value of the index indicates that the shorter meta-path matches closer to the front of the longer meta-path; and using a grouping algorithm to group the total set of meta-paths and the reusability indexes between the meta-paths to obtain multiple groups.
[0043] According to one embodiment of the present invention, the calculation method of the reusability index includes:
[0044]
[0045] in, Indicates the position coordinates of the element in the longer meta-path where the shorter meta-path matches the prefix of the longer meta-path. Represents the length of the longer meta-path. Of course, those skilled in the art can also use other alternative calculation methods, such as:
[0046]
[0047] in, represents the length of the shorter meta-path, This metric represents the position coordinates of the element within the longer meta-path that matches the prefix of the shorter meta-path. This metric takes into account the impact of both "early matches" and "complete matches." The earlier the match and the longer the match, the greater the contribution. For two shorter meta-paths of the same length, the earlier the match, the higher the score (reusability index value).
[0048] According to one embodiment of the present invention, the grouping algorithm may be the Louvain algorithm. Alternatively, those skilled in the art may also employ other grouping algorithms, such as the Leiden algorithm. The Louvain algorithm is used as an example for illustration.
[0049] The grouping process based on the Louvain algorithm includes:
[0050] The initialization phase includes: initializing each node in the total meta-path set as an independent community;
[0051] The local optimization phase includes: attempting to maximize modularity (abbreviated as Q) by moving nodes to adjacent communities, including calculating the incremental modularity (denoted as ΔQ) after moving a node (node i) to an adjacent community (community C). If the incremental modularity is greater than 0, the node is moved to the adjacent community; otherwise, the node remains in the current community. The above movement process is repeated until no nodes can be moved to further improve modularity, and the network reconstruction phase begins.
[0052] The network reconstruction phase includes: treating each community as a new node, retaining the edges between communities, and assigning the weight of the edges the sum of the weights of the edges between communities in the original network. Repeating the local optimization phase until the modularity no longer increases, the final community partition is output, and each divided community is regarded as a group, that is, a meta-path set.
[0053] Schematically, the incremental calculation formula for modularity can be as follows:
[0054]
[0055] The following simplified forms can be used:
[0056]
[0057] in, represents the modularity increment brought by moving node i into the adjacent community C, represents the total number of edges in the graph corresponding to the total set of meta-paths, represents the sum of the weights of the edges within the adjacent community C, represents the total edge weight of nodes associated with the adjacent community C, represents the total edge weight associated with node i, It represents the sum of the weights of the edges connecting node i with the nodes in the adjacent community C. The above reusability index is the weight of the edge in the graph.
[0058] In terms of the execution process, the above scheme first constructs a hypergraph for the meta-path based on a newly proposed reusability indicator, and performs semantic grouping with the goal of maximizing reusability within the group. Then, the sampling process is executed in parallel for multiple semantic groups, and the semantic redundancy-aware sampling method proposed in the present invention is used in each group to reduce off-chip memory access.
[0059] For meta-paths with reuse relationships, a higher matching position means greater reusability. For example, PAP is more reusable for PAPSP than PSP. Based on this indicator, the meta-path can be regarded as a node, and a hypergraph can be established based on the reuse relationship, such as Figure 4 As shown in a in , each edge is assigned a reusability index ( 、 equilateral weights), such as Figure 4 As shown in b, since the reusability of the two meta-paths PAP and PSP for PAPSP is different, the edge weight between node PAP and node PAPSP is is greater than the edge weight between nodes PSP and PSPAP After the above modeling process, the optimal grouping problem can be converted into a hypergraph community partitioning problem that maximizes reusability. That is, the modeled hypergraph is partitioned into communities so that the sum of the weights within a community is large, while the sum of the weights between communities is small. This is shown in Algorithm 1.
[0060] Algorithm 1: Reusability-driven semantic grouping
[0061]
[0062] First, in the first stage, i.e., the stage of improving reusability, each meta-path node is considered as an independent group. Each adjacency group (Line 5), try to add the node to the adjacency group and calculate the reusability increment (Line 6), which is the increase in the sum of the total reusability weights within each group, record this value and iterate this process until the group with the largest reusability increment is obtained. (Lines 5-10), if the maximum reusability increment is greater than 0, the node Join the group (lines 11-13). Repeat this process until all nodes are added to the group that can obtain the largest reusability increment. Then in the second stage, which is the grouping and merging stage, each obtained group is regarded as a node, and a hypergraph is constructed based on reusability (line 17). The first stage is repeated until the number of groups is less than the maximum number of parallel channels that the hardware can support (lines 16-20). At this point, the grouping of all meta-paths is completed, and the meta-paths within the group are guaranteed to have the maximum reusability. Figure 4 As shown in Figure c, the meta-paths PAP, PAPP, and PAPSP are grouped together, while the other meta-paths are grouped together. These meta-path groupings are then used as the basis for task scheduling and sent to the subsequent Heterogeneous Graph Neural Network Sampling Accelerator (ESR-HGNN) for execution. This Heterogeneous Graph Neural Network Sampling Accelerator is primarily used to accelerate the computational sampling process. The resulting sampling results can be sent to any Heterogeneous Graph Neural Network inference platform, including CPUs, GPUs, and dedicated inference accelerators, for further inference execution.
[0063] It should be noted that ESR-HGNN itself is a multi-channel design that can support the parallel sampling process of multi-path groups, while the reusability-driven semantic grouping method is a data preprocessing process performed on the CPU.
[0064] Step S3: Obtain a batch of target nodes and meta-path sets.
[0065] According to one embodiment of the present invention, due to the large size of the graph data, the sampling process divides the nodes in the graph data into multiple batches, each time only sampling the target nodes in a preset batch. Furthermore, the total set of meta-paths is also divided into multiple meta-path sets. These sets are then used to guide the sampling of neighboring nodes of the target node in the current batch.
[0066] Step S3 may be executed on a computing device having a single processing channel, where the computing device obtains a batch of target nodes and meta-path sets at a time for sampling.
[0067] Alternatively, step S3 may be performed on an accelerator having multiple parallel processing channels. One channel of the accelerator may acquire a batch of target nodes and meta-path sets. A different channel may acquire another batch of target nodes and meta-path sets, thereby performing sampling in parallel on each channel.
[0068] After the current channel completes sampling for each target node of the batch according to all meta-paths of the current meta-path set, the next batch of target nodes and meta-path sets are obtained.
[0069] Step S4: Maintain a meta-path dictionary tree for recording sampled meta-paths for each batch of target nodes.
[0070] According to one embodiment of the present invention, since different target nodes in each batch of target nodes undergo sampling processes under a particular meta-path almost simultaneously, a meta-path dictionary tree is maintained for each batch of target nodes, recording the sampled meta-paths. After sampling is completed for each target node based on all meta-paths in the meta-path set, the meta-path dictionary tree is initialized to re-establish and maintain the meta-path dictionary tree corresponding to the next batch of target nodes.
[0071] Step S5: Select meta-paths from the meta-path set according to the predetermined order requirements, sample the target nodes on the graph data in turn, and update the meta-path dictionary tree, where the predetermined order requirements include the order of paths from short to long. Before each sampling, determine whether there is semantic overlap based on the currently selected meta-path and the meta-path dictionary tree. If so, complete the sampling process corresponding to the currently selected meta-path based on the sampling results of the overlapping part.
[0072] According to one embodiment of the present invention, completing the sampling process corresponding to the currently selected meta-path based on the sampling results that have been sampled in the overlapping part includes: obtaining each sampling result that has been sampled in the overlapping part, extracting the end node of each sampling result as the target alternative node; starting from the target alternative node, sampling backward according to the non-overlapping part of the currently selected meta-path and the meta-path dictionary tree to obtain the sampling result corresponding to the target node. For example, see Figure 5 Assume that two meta-path sets (one containing PAP, PAPP, and PAPSP, and the other containing PSP, PSPP, and PSPAP) are divided into different channels and processed in parallel. If n first sampling results have already been sampled for target node P under the guidance of meta-path PAP, then n second sampling results need to be sampled for the same target node P under the guidance of meta-path PAPP. In this case, the n second sampling results do not need to be collected starting from the target node P. Instead, each terminal node in the n first sampling results can be used as a target alternative node. Sampling from this point forward will yield the sampling result for the corresponding target node.
[0073] The following is a schematic sampling process using Algorithm 2:
[0074] Algorithm 2 Redundancy-aware sampling method
[0075]
[0076] For a given heterogeneous graph and metapath collection , and the number of sampled neighbors , you can build a meta-path dictionary tree For prefix matching of meta-paths, the dictionary tree is a commonly used data structure, which is often used to record and quickly find whether the prefix of a long sequence exists. , for each target node , sample its neighbors under all meta-paths. For each meta-path , first in Check if there is a matching prefix meta-path in (line 4, for example, PAP is the prefix meta-path of PAPSP, and PSP is the prefix meta-path of PSPAP). If not, Direct Insert and at the same time based on the original target node in the meta path Downsampling neighbors (lines 12-17); if exist Zhongyu Successful matching indicates that the target node is in Shorter prefix metapath If neighbor sampling has already been performed, the traversal path can be reused directly, and the sampling task is adjusted as follows: (1) The sampling target is no longer the original target node , but In the meta path The neighbor set Each neighbor node in ; (2) The metapath on which sampling is based is no longer , but With prefix The remaining part after matching The original sampling task is transformed into a meta-path-based , for each neighbor Sample a neighbor (Lines 4-10), and in parallel Updated to In the above process, the update of the dictionary tree in line 6 and the insertion of the dictionary tree in line 14 are both basic management operations of the dictionary tree data structure. The sampling functions in lines 8 and 16 are the processes of traversing neighbors under different adjacency relationships based on the meta-path to obtain the semantic neighbors under different meta-paths.
[0077] See also Figure 6, which shows a simple example of redundancy-aware sampling. The algorithm can be understood in conjunction with the legend. For a set containing PSP, PSPP, and PSPAP, sampling is performed as follows. Initially, the dictionary tree is initialized with only the root node, and the target node (the red node P) is sampled based on the meta-path PSP. First, the dictionary tree is queried. If it is found that PSP does not exist, it is sampled according to PSP and inserted into the dictionary tree. Then, assuming that the next meta-path is PSPAP sampling, the dictionary tree is queried at this time, and a prefix meta-path match is performed. If PSP exists, it is actually sampled according to PAP after matching, and the dictionary tree is updated based on PSPAP.
[0078] According to one embodiment of the present invention, this embodiment differs from the above embodiments in that this step employs a scheme for randomly grouping the total set of meta-paths, rather than semantic grouping (i.e., step S2 is different). This embodiment provides a method for heterogeneous graph sampling, comprising: step S2': randomly dividing the total set of meta-paths into multiple groups, with the meta-paths in each group constituting a meta-path set; the number of groups can be greater than or equal to the number of parallel computing channels of the accelerator. The remaining steps are similar to those of the above embodiments and are not further described here. While this solution performs slightly worse than the above embodiments, subsequent experiments demonstrate that it still offers performance improvements over existing solutions.
[0079] According to one embodiment of the present invention, see Figure 7 , an accelerator (abbreviated as ESR-HGNN) is provided, comprising: a scheduler for scheduling overall processing tasks; a memory access controller for controlling access to memory data; multiple channels (such as channels 0-3) capable of performing parallel computing, each channel comprising: a semantic redundancy detector, which is configured to: obtain a batch of target nodes and meta-path sets, and maintain a meta-path dictionary tree for recording sampled meta-paths for each batch of target nodes; select a meta-path from the meta-path set according to a predetermined order requirement, determine whether there is semantic overlap based on the currently selected meta-path and the meta-path dictionary tree, and obtain a judgment result; and after the sampling process corresponding to the currently selected meta-path is completed, record it in the meta-path dictionary tree; a neighbor sampling unit, which is configured to: perform sampling according to the judgment result, including: if the judgment result is yes, completing the sampling process corresponding to the currently selected meta-path based on the sampling results of the overlapping part; if the judgment result is no, gradually completing the corresponding sampling process according to the complete sampling path indicated in the currently selected meta-path.
[0080] According to one embodiment of the present invention, a semantic redundancy detector includes: a local scheduler for scheduling tasks related to semantic redundancy detection, a metapath trie (metapath prefix tree) for recording sampled metapaths, and a type matcher for determining whether there is semantic overlap based on the currently selected metapath and the metapath trie. A trie is a tree-like data structure that can efficiently store and retrieve sequences such as strings. Each node represents a character, and the path from the root node to the leaf node corresponds to a different string or prefix. In the design of this embodiment, each node stores a code (code) for the current node type, a storage area starting address (data_addr) corresponding to the sampled neighbor index, a pointer to the subsequent node (next_addr), and an end flag (end_flag). The end flag is used to indicate whether the node is the terminal node of a complete metapath. This structure supports prefix matching of subsequent metapaths. If a complete path is successfully matched, the traversal path of the metapath can be directly reused. To achieve efficient prefix matching, binary encoding can be used for meta-paths, and a matching unit based on a multi-bit (e.g., 4-bit) XOR tree is designed. Of course, those skilled in the art can also design XOR trees with other bit sizes based on the type of meta-path. Since meta-paths are strings and type matchers are bit strings, a semantic decoder can also be included in the semantic redundancy detector to convert the meta-path into a bit string.
[0081] The Neighbor Sampler repeatedly samples the target node's neighbors based on the meta-path (semantic sequence), storing the neighbor information in Compressed Sparse Row (CSR) format until the required number of neighbors has been sampled. The Neighbor Sampler consists of a Neighbor Loader, which loads the current target node's neighbor list from memory, and a Neighbor Selector, which randomly selects neighbor nodes using a Linear Feedback Shift Register (LFSR). In addition, each channel is configured with a private Adjacency List Cache to temporarily store neighbor nodes for later reuse. The Neighbor Sampler also includes a local controller, which schedules the sampling process for each target node. The Neighbor Sampler also includes a target selector, a vertex buffer, and a semantic path cache. The vertex buffer is used to pre-cache the target nodes (original target nodes) to be processed in the batch. The semantic path cache is used to cache sampled results. For example, it caches the target node (corresponding to the target ID) and neighbor nodes (corresponding to the neighbor ID) sampled for each meta-path (corresponding to the meta-path code). When semantic redundancy is detected, the target selector searches for an alternative target node to continue sampling, thus avoiding repeated traversal. Specifically, if the target selector determines that the result is yes, it searches the semantic path cache for each sampled result in the overlapping portion and extracts the terminal node of each sampled result as the target replacement node. If the result is no, the target selector retrieves the target node from the node cache to start sampling. Illustratively, when the semantic redundancy detector identifies a reusable subpath that has already been sampled in the currently processed meta-path, it uses a matching mechanism to match this subpath with the recorded path, sets the target selector's control signal to 1, and directly extracts the sampled neighbors corresponding to this subpath from the semantic path cache as the new starting point for subsequent semantic sampling, thus eliminating redundant operations.
[0082] It should be noted that the ESR-HGNN structure retains a hardware-implemented dictionary tree structure for the semantic redundancy detector and a type matcher for matching operations; the neighbor sampling unit is responsible for performing neighbor sampling based on the specific target node and task meta-path to be sampled.
[0083] According to one embodiment of the present invention, a method for classifying nodes in graph data is also provided, the method comprising: obtaining a sampling result for a target node according to a method for heterogeneous graph sampling; training a graph neural network based on the sampling result to obtain an embedding feature of the target node; and classifying the target node based on the embedding feature to obtain a classification result. The graph data is a knowledge graph, such as an academic network relationship graph, a social relationship graph, a medical knowledge graph, an e-commerce knowledge graph, or a financial knowledge graph.
[0084] In order to verify the effect of the present invention, the inventors also conducted experiments.
[0085] The dataset used in the experiment is:
[0086] 1) ACM (Association for Computing Machinery Digital Library): Academic dataset in the computer field, including papers, authors, conferences / journals, keywords, etc., used for academic network analysis (such as co-authorship network and citation network).
[0087] 2) IMDB (Internet Movie Database): A movie dataset containing movies, actors, directors, genres, ratings, etc., often used in recommendation systems or heterogeneous graph analysis (such as actor-movie bipartite graphs).
[0088] 3) MAG (Microsoft Academic Graph): This is a large-scale academic knowledge graph that covers interdisciplinary papers, authors, institutions, conferences / journals, etc. (updates have been stopped, but historical data is still widely used).
[0089] The methods used in the experiment are:
[0090] ESR-HGNN-B: is the most basic single-channel configuration;
[0091] ESR-HGNN-R: Based on ESR-HGNN-B, the redundant perception sampling method proposed in this paper is used;
[0092] ESR-HGNN-P: Based on ESR-HGNN-R, ESR-HGNN is configured with four channels, but with random semantic grouping.
[0093] ESR-HGNN: Based on ESR-HGNN-P, the reusability-aware semantic grouping method proposed in this invention is added.
[0094] like Figure 8 As shown in the experimental results,
[0095] 1. Reduced off-chip memory access during sampling: Figure 8 This paper demonstrates the off-chip memory accesses under different experimental configurations during meta-path-based heterogeneous graph sampling on three different datasets: ACM, IMDB, and MAG. As shown, the redundancy-aware sampling method proposed in this paper reduces the average off-chip memory access (Geometric Mean, GM) by 71.56% compared to traditional sampling methods, while the reusability-driven semantic grouping method proposed in this paper reduces off-chip memory accesses by 44.42% compared to random grouping methods.
[0096] 2. Improvement of sampling performance: such as Figure 8 As shown in the figure, the redundancy-aware sampling method proposed in the present invention improves the sampling performance by an average of 2.94 times compared with the traditional sampling method, while the reusability-driven semantic grouping method proposed in the present invention improves the sampling performance by an average of 3.21 times compared with the random grouping parallel method.
[0097] In general, the embodiments of the present invention can achieve at least one of the following effects:
[0098] (1) A novel HGNN sampling paradigm is proposed, which eliminates redundancy based on the perception of semantic redundancy to reduce off-chip random memory access, and supports semantic parallelism through reusability-driven semantic grouping, while ensuring the effectiveness of the redundancy-aware sampling method in semantic parallel scenarios.
[0099] (2) By maintaining a meta-path dictionary tree to record the shorter meta-paths that have been sampled, and when semantic redundancy is found, the traversal results of the shorter meta-path are directly reused, and the target node of the sampling is dynamically adjusted. This method effectively reduces the off-chip memory access in the sampling process on multiple data sets and improves the sampling performance.
[0100] (3) The random grouping method that supports semantic parallelism may lead to a decrease in the effectiveness of the semantic redundancy-aware sampling method. The embodiment of the present invention proposes a reusability index through the redundant overlapping relationship of meta-paths, and constructs a hypergraph by treating meta-paths as nodes and adding edges between meta-paths with reusable relationships. The semantic grouping problem of maximizing the reusable traversal paths within the group can be modeled as a problem of dividing the hypergraph into communities according to reusability.
[0101] (4) Based on the classic Louvain algorithm, a new reusability-driven semantic grouping method is proposed, which effectively reduces off-chip memory access and improves sampling performance compared to the random grouping method.
[0102] It should be noted that although the above describes the various steps in a specific order, it does not mean that the steps must be performed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.
[0103] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.
[0104] A computer-readable storage medium may be a tangible device that holds and stores instructions used by an instruction execution device. Computer-readable storage media may include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove having instructions stored thereon, and any suitable combination thereof.
[0105] While various embodiments of the present invention have been described above, the above descriptions are intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A method for heterogeneous graph sampling, characterized in that: include: Get a batch of target nodes and meta-path sets; Maintain a meta-path dictionary tree for recording the sampled meta-paths for each batch of target nodes; Select meta-paths from the meta-path set according to the predetermined order requirements, sample the target nodes on the graph data in turn, and update the meta-path dictionary tree. The predetermined order requirements include the order of paths from short to long. Before each sampling, determine whether there is semantic overlap based on the currently selected meta-path and the meta-path dictionary tree. If so, complete the sampling process corresponding to the currently selected meta-path based on the sampling results of the overlapping part.
2. The method according to claim 1, characterized in that The sampling process corresponding to the currently selected meta-path is completed based on the sampling results of the overlapping part, including: Obtain each sampling result that has been sampled in the overlapping part, and extract the end node of each sampling result as the target replacement node; Starting from the target alternative node, sampling is performed based on the non-overlapping part of the currently selected meta-path and the meta-path dictionary tree to obtain the sampling result corresponding to the target node.
3. The method according to claim 1, characterized in that The method further comprises: After completing the sampling of each target node according to all meta-paths of the meta-path set, the next batch of target nodes and meta-path sets are obtained, and the meta-path dictionary tree is initialized.
4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Obtain a total set of meta-paths, which includes all meta-paths used to guide the sampling process, each meta-path is a semantic path pattern composed of node types and edge types; Taking maximizing the reusability of the paths within the group as the optimization goal, the total set of meta-paths is divided into multiple groups, and the meta-paths in each group constitute a meta-path set.
5. The method according to claim 4, characterized in that The process of dividing the total set of meta-paths into multiple groups includes: For every two meta-paths in the total meta-path set, a reusability index is determined. The larger the value of the index, the closer the shorter meta-path is to the matching position within the longer meta-path. According to the total set of meta-paths and the reusability index between each meta-path, a grouping algorithm is used to perform grouping to obtain a plurality of groups.
6. The method according to claim 5, characterized in that The calculation method of reusability index includes: in, Indicates the position coordinates of the element in the longer meta-path where the shorter meta-path matches the prefix of the longer meta-path. Indicates the length of the longer meta-path.
7. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 6.
8. An electronic device, characterized in that: include: one or more processors; as well as a memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1 to 6 by executing the executable instructions.
9. An accelerator for implementing the method according to any one of 1 to 6, comprising: A semantic redundancy detector, which is configured to: Obtain a batch of target nodes and meta-path sets, and maintain a meta-path dictionary tree for each batch of target nodes to record the sampled meta-paths; Select a meta-path from the meta-path set according to a predetermined order, determine whether there is semantic overlap based on the currently selected meta-path and the meta-path dictionary tree, and obtain a judgment result; as well as After the sampling process corresponding to the currently selected meta-path is completed, it is recorded in the meta-path dictionary tree; The neighbor sampling unit is configured to perform sampling according to the judgment result, including: If the judgment result is yes, the sampling process corresponding to the currently selected meta-path is completed based on the sampling results of the overlapping part; If the judgment result is no, the corresponding sampling process is gradually completed according to the complete sampling path indicated in the currently selected meta-path.
10. The accelerator according to claim 9, characterized in that The semantic redundancy detector includes a semantic decoder and a type matcher, where: The semantic decoder is used to convert each meta-path in the dictionary tree and the currently selected meta-path into a bit string; The type matcher is used to determine whether there is semantic overlap based on the numerical comparison of bit strings. If the bit string of the currently selected meta-path is consistent with the bit string of a meta-path in the dictionary tree, there is semantic overlap; if the bit string of the currently selected meta-path is different from the bit strings of all meta-paths in the dictionary tree, there is no semantic overlap.