Inference calculation method and system of graph attention mechanism

Abstract

This invention belongs to the technical field of reasoning based on a specific computational model. It discloses a reasoning computation method and system for a graph attention mechanism. The reasoning computation method includes: dividing the original graph data into multiple partitions and constructing a local tree structure; constructing the associated edges for each original graph vertex; performing boundary vertex sampling, boundary edge sampling, internal edge sampling, and graph topology simplification for each partition; obtaining the key-value data of the original graph vertices; for each fused vertex, obtaining the key-value data of its neighboring vertices based on its fused edges and performing attention calculation to obtain the updated embedding of the corresponding fused vertex; using the associated edges and graph topology sampling edges as information acquisition edges, for each original graph vertex, obtaining the embeddings of its neighboring vertices based on its corresponding information acquisition edges and performing attention calculation to obtain the updated embedding of the corresponding original graph vertex. This approach can reduce computational overhead and memory consumption while maintaining the computational accuracy of the graph attention mechanism.

Description

Technical Field

[0001] This invention belongs to the technical field of reasoning based on specific computational models, and more specifically, relates to a reasoning computation method and system based on graph attention mechanism. Background Technology

[0002] Graph attention mechanisms (Graph Transformer) are an emerging architecture derived from the introduction of the Transformer architecture into the field of graph learning in natural language processing and computer vision processing. Representative works include Graphormer and Graph Transformer.

[0003] While Graph Transformer has demonstrated superior performance compared to traditional GNNs on various graph learning benchmarks, it still faces technical challenges. Specifically, the computational complexity of the standard self-attention mechanism is proportional to the square of the length of the input sequence. When dealing with large-scale real-world graph data, this quadratic complexity leads to unacceptable computational overhead and memory consumption, forming a serious "attention computation bottleneck." According to relevant research, attention computation accounts for more than 80% of the time spent in inference computation of Graph Transformer models, becoming the absolute dominant part of the inference computation process.

[0004] While researchers have proposed various sparse attention mechanisms to reduce computational complexity, these methods ignore the inherent structural information of the graph, potentially disrupting important long-range dependencies, leading to decreased model performance, and may also introduce additional overhead. Chinese patents such as "CN120087414B - Optimization method, device, storage medium and program product for attention mechanism computation" and "CN118333167B - An attention mechanism computation method, computation system and storage medium" propose some efficiency improvement schemes, but they are all solely focused on the computation of the attention mechanism, not on the implementation of graph attention mechanisms.

[0005] How to reduce computational overhead and memory consumption while ensuring the computational accuracy of graph attention mechanisms is a pressing technical problem that needs to be solved. It has become the key to promoting the application of graph Transformers in large-scale real-world scenarios. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a reasoning computation method and system for graph attention mechanisms. Its purpose is to reduce computational overhead and memory consumption while ensuring the computational accuracy of graph attention mechanisms, thereby promoting the application of graph Transformers in large-scale real-world scenarios.

[0007] To achieve the above objectives, this invention is proposed.

[0008] According to a first aspect of the present invention, a reasoning computation method using a graph attention mechanism is provided, comprising: The original graph data is divided into multiple partitions. For each partition, the original graph vertices within it are used as the bottom-level leaf nodes to generate parent nodes layer by layer until the top-level root node is generated, thus obtaining the local tree structure of the partition. All the parent nodes generated in the local tree structure are fused vertices. For each fused vertex, each original graph vertex it depends on is used as the source vertex to add a fused edge pointing to the fused vertex. For each original graph vertex, select its corresponding set of associated vertices and construct the associated edges from each associated vertex to the original graph vertex. The set of associated vertices includes other original graph vertices and some merged vertices within the partition to which the original graph vertex belongs, as well as some merged vertices within other partitions. The original graph vertices that each merged vertex in the set of associated vertices do not overlap, and the union of the original graph vertices that each merged vertex depends on and the other original graph vertices in the set of associated vertices is all other original graph vertices. For each partition, boundary vertex sampling, boundary edge sampling, internal edge sampling, and graph topology simplification are performed separately. Specifically, boundary vertex sampling for a partition includes: collecting boundary vertices at a preset sampling ratio to form a boundary vertex sampling set for the corresponding partition. The boundary vertices of this partition are the original graph vertices from other partitions that directly point to the vertices of this partition via original edges. Boundary edge sampling for a partition includes: selecting boundary edges whose source vertices belong to the boundary vertex sampling set of this partition to form a boundary edge sampling set for the corresponding partition. The boundary edges of this partition are the original edges connecting this partition to other partitions. Internal edge sampling for a partition includes: collecting the original edges within the partition at a preset sampling ratio to form an internal edge sampling set for the corresponding partition. Graph topology simplification for a partition includes: merging its boundary edge sampling set with its internal edge sampling set to obtain the graph topology sampling edge set corresponding to this partition. Obtain the key-value data of the vertices in the original graph. For each fused vertex, obtain the key-value data of its neighboring vertices based on its fused edges and perform attention calculation to obtain the updated embedding of the corresponding fused vertex. Using associated edges and graph topology sampling edges as information acquisition edges, for each original graph vertex, the embeddings of its neighboring vertices are obtained based on its corresponding information acquisition edges, and attention is calculated to obtain the updated embeddings of the corresponding original graph vertex. When all original graph vertices have obtained their updated embeddings, the graph attention mechanism calculation for the current attention calculation round ends.

[0009] According to a second aspect of the present invention, a reasoning computation system based on a graph attention mechanism is provided, comprising: The local tree construction module is used to divide the original graph data into multiple partitions. For each partition, the original graph vertices inside it are used as the bottom-level leaf nodes to generate parent nodes layer by layer until the top-level root node is generated, thus obtaining the local tree structure of the partition. All the parent nodes generated in the local tree structure are fused vertices. For each fused vertex, each original graph vertex it depends on is used as the source vertex to add a fused edge pointing to the fused vertex. The associated edge construction module is used to select the corresponding associated vertex set for each original graph vertex and construct the associated edges from each associated vertex to the original graph vertex. The associated vertex set includes other original graph vertices and some merged vertices within the partition to which the original graph vertex belongs, as well as some merged vertices within other partitions. The original graph vertices that each merged vertex in the associated vertex set depends on do not overlap, and the union of the original graph vertices that each merged vertex depends on and other original graph vertices in its associated vertex set is all other original graph vertices. The original graph topology sampling module performs boundary vertex sampling, boundary edge sampling, internal edge sampling, and graph topology simplification for each partition. Boundary vertex sampling for a partition includes: collecting boundary vertices at a preset sampling ratio to form a boundary vertex sampling set for the corresponding partition. The boundary vertices of this partition are the original graph vertices in other partitions that are directly connected to the vertices of this partition by original edges. Boundary edge sampling for a partition includes: filtering out boundary edges whose source vertices belong to the boundary vertex sampling set of this partition to form a boundary edge sampling set for the corresponding partition. The boundary edges of this partition are the original edges connecting this partition to other partitions. Internal edge sampling for a partition includes: collecting the original edges inside the partition at a preset sampling ratio to form an internal edge sampling set for the corresponding partition. Graph topology simplification for a partition includes: merging its boundary edge sampling set with its internal edge sampling set to obtain the graph topology sampling edge set corresponding to this partition. The fused vertex embedding update module is used to obtain the key-value data of the vertices in the original graph. For each fused vertex, it obtains the key-value data of its neighboring vertices based on its fused edges and performs attention calculation to obtain the updated embedding of the corresponding fused vertex. The original graph vertex embedding update module uses associated edges and graph topology sampling edges as information acquisition edges. For each original graph vertex, it obtains the embeddings of its neighboring vertices based on its corresponding information acquisition edges and performs attention calculations to obtain the updated embedding of the corresponding original graph vertex. When all original graph vertices have obtained their updated embeddings, the graph attention mechanism calculation for the current attention calculation round ends.

[0010] In summary, compared with the prior art, the technical solutions conceived in this invention have the following main advantages: This invention partitions the original graph and constructs a local tree structure to generate fused vertices. Subsequently, sparse edges are constructed based on associated and fused edges to learn global information. The original vertices and edges are also collected, reducing the number of cross-partition edges from the source. Based on the above tree structure and graph topology sampling, a sparse graph constructed by the graph topology-induced attention computation mechanism is built. This reduces the computational complexity of standard fully connected attention from O(N²) to a near-linear level, greatly reducing the number of attention pairs that need to be computed, thereby reducing computational overhead and memory consumption. While achieving high computational sparsity, it avoids a significant loss of accuracy in the graph attention computation model. Attached Figure Description

[0011] Figure 1 This is a flowchart of the reasoning computation method of the graph attention mechanism in one embodiment of the present invention; Figure 2 This is a schematic diagram of a local tree structure in one embodiment of the present invention; Figure 3 This is a schematic diagram of the processing of a local tree structure in one embodiment of the present invention; Figure 4 This is a schematic diagram of topological sampling of the original graph structure in one embodiment of the present invention; Figure 5 This is a schematic diagram of embedding neighboring vertices of a target vertex in one embodiment of the present invention; Figure 6 This is a structural block diagram of a graph attention mechanism inference computation system according to an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0013] In a first aspect, the present invention claims a reasoning computation method based on a graph attention mechanism.

[0014] like Figure 1 The diagram shown is a flowchart of the inference computation method of the graph attention mechanism in one embodiment of the present invention. The following is a summary of the steps. Figure 1 The method is described in detail.

[0015] S1. Divide the original graph data into multiple partitions. For each partition, use the original graph vertices within it as the bottom-level leaf nodes to generate parent nodes layer by layer until the top-level root node is generated, thus obtaining the local tree structure of the partition. All generated parent nodes in the local tree structure are fused vertices. For each fused vertex, add fused edges pointing to the fused vertex as source vertices for each original graph vertex it depends on.

[0016] First, construct a local tree structure: divide the original graph data into multiple partitions. For each partition, use the original graph vertices within it as the bottom-level leaf nodes to generate parent nodes layer by layer until the top-level root node is generated, thus obtaining the local tree structure of that partition. All generated parent nodes in the local tree structure are fused vertices.

[0017] Specifically, the system loads raw graph data through a high-speed interface. The raw graph data can be social network graphs, traffic network graphs, etc. Taking the OGBn-products dataset as an example, OGBn-products is a large-scale, real-world product association graph dataset in the Open Graph Benchmark (OGB), which is often used to evaluate the performance of graph neural networks on node classification tasks. Its raw graph vertex and raw edge data are transferred to the on-chip cache through the FPGA's DMA (Direct Memory Access), and then the raw graph data is partitioned.

[0018] In one embodiment, the classic METIS graph partitioning algorithm can be used for partitioning. The objective function is to minimize edge cuts and to make the number of vertices in each partition as balanced as possible. After METIS partitioning, each partition i obtains an internal vertex set V. i Then, based on the internal vertex set V i Construct the local tree structure for partition i.

[0019] The leaf nodes of the local tree structure of partition i are the set of internal vertices V of that partition. i The leaf nodes, representing level 0 of the local tree structure, are grouped, with each group's nodes becoming child nodes. A corresponding parent node is then created for each group, generating level 1 nodes. This process is repeated, grouping level 1 nodes and creating a parent node for each group, generating level 2 nodes. This process continues until every pair of vertices in each group is merged into a new level 1 vertex. Then, on the FPGA, parallel reduction logic merges every pair of vertices in the second level into a new level 3 vertex, and so on, until a root node is reached. This root node represents the highest level of the tree, providing paths to all vertices within the partition. The nodes in the local tree structure, excluding the leaf nodes, are called merged vertices.

[0020] like Figure 2The diagram shown illustrates a local tree structure according to an embodiment of the present invention. In this embodiment, every two child nodes in the local tree structure correspond to one parent node. Taking partition i as an example, let partition i have n... i n vertices of the original graph i =30,000). The system sequentially feeds these vertices into the merging unit as level 0 (leaf nodes). Then, a recursive merging process similar to merge sort is executed: the vertices of level 0 are paired in order (e.g., vertices 0 and 1, vertices 2 and 3, ...), each pair generating a level 1 merged vertex. Next, the level 1 merged vertices are paired in order again to generate level 2 merged vertices. This process can be implemented using hardware shift registers and address generators for recursive reduction. For a given set of n... i A full binary tree with n leaves will eventually generate n i - One fused vertex is used to form a tree structure. The root node at the highest level represents the aggregation of all original graph vertices within that partition, and the tree structure is output via a register.

[0021] Secondly, for each fused vertex, a fused edge is added pointing to that fused vertex, using each original graph vertex it depends on as the source vertex.

[0022] like Figure 3 The diagram shown is a schematic representation of the processing of a local tree structure in one embodiment of the present invention.

[0023] Specifically, for each fused vertex (i.e. the parent node generated in the tree), starting from each of its corresponding original graph vertices (i.e. each leaf node associated with the fused vertex), a directed edge pointing to the fused vertex itself is added. The directed edge from the original graph vertex to its fused vertex is the fused edge. This process can be completed on-chip through address generation logic.

[0024] Taking partition i as an example, after its local tree structure is constructed, the system immediately creates fusion edges for all its fused vertices. Assuming that parent nodes are generated by pairwise combinations according to the index numbers of child nodes, for the k-th fused vertex at level x... It is associated with the index in level 0 from arrive of total The system provides these original graph vertices. For each vertex in the original graph, a pointer to it is created in the corresponding memory area through the multiplexer. Directed edges do not require off-chip memory access. For example, for any fused vertex located in layer 2 (x=2), it represents a region containing 2... 2 =Grouping the four original graph vertices, the system will create four merged edges.

[0025] For each partition, the merged vertices and internal merged edges of that partition are generated in the manner described above, resulting in the local optimized tree structure for that partition. This tree contains the original graph vertices, merged vertices, and merged edges.

[0026] S2. For each original graph vertex, select its corresponding set of associated vertices and construct the associated edges from each associated vertex to the original graph vertex. The set of associated vertices includes other original graph vertices and some merged vertices within the partition to which the original graph vertex belongs, as well as some merged vertices within other partitions. The original graph vertices that each merged vertex in its set of associated vertices depend on do not overlap. The union of the original graph vertices that each merged vertex depends on and the other original graph vertices in its set of associated vertices is all other original graph vertices.

[0027] Specifically, for each original graph vertex, an associated edge is constructed. Based on this associated edge, information about any other original graph vertex can be retrieved. When selecting associated vertices for a given original graph vertex, for the partition to which the vertex belongs, some original graph vertices and some merged vertices are selected. For other partitions, some merged vertices from those partitions are directly selected. Since merged vertices are themselves associated with multiple original graph vertices, selecting merged vertices to construct associated edges establishes connections with all other original graph vertices while reducing the number of associated edges, thus simplifying computation.

[0028] In one specific embodiment, the number of associated edges in each layer can be controlled. For example, the local structure trees of all partitions constitute a global structure tree. When constructing associated edges for any original graph vertex, the vertex's index position is used as a reference position. Starting from the bottom layer of the global structure tree, associated vertices on the left and right sides adjacent to the reference position are searched layer by layer, and associated edges pointing to the original graph vertex are added. The number of associated vertices located on the same side of the reference position in the same layer is limited to m, where m is a preset hyperparameter, greater than or equal to 2, for example, 2 or 4. (Reference) Figure 3 As shown, the original graph vertex with position number 1 For example, when constructing related edges, start from layer 0, and... Using the given position as a reference, find the two nearest vertices on each of its left and right sides (assuming m=2). Since there is only one vertex on the left, only the two nearest vertices on the left are found. Find it on the right. The associated vertices of layer 0 are Then, associated vertices are searched starting from layer 1. Vertices from layer 1 onwards are all merged vertices. If a vertex in the original graph has already been selected as an associated vertex in layer 0, the merged vertex selected in layer 1 cannot have a dependency on it (to avoid duplication). Therefore, there are no selectable vertices to the left of the reference position in layer 1, but merged vertices can be selected to the right of the reference position. and fusion apex Similarly, in the second layer, there are no selectable vertices to the left of the reference position, but selectable merging vertices to the right of the reference position. and fusion apex This process continues until the associated vertices cover all other original graph vertices and there is no overlap between the selected associated vertices.

[0029] Taking a vertex v in an original graph in partition i as an example, its associated vertices may include: several adjacent original graph vertices in the same partition, several fused vertices in the lower layers of this partition, and several fused vertices in the higher layers of other partitions.

[0030] S3. Perform boundary vertex sampling, boundary edge sampling, internal edge sampling, and graph topology simplification for each partition to obtain the graph topology sampled edge set corresponding to that partition.

[0031] Specifically, performing boundary vertex sampling for a partition includes: collecting boundary vertices according to a preset sampling ratio to form a boundary vertex sampling set for the corresponding partition, where the boundary vertices of the partition are the original graph vertices in other partitions that directly point to the vertices of the partition through the original edges; performing boundary edge sampling for a partition includes: filtering out boundary edges whose source vertices belong to the boundary vertex sampling set of the partition to form a boundary edge sampling set for the corresponding partition, where the boundary edges of the partition are the original edges that connect the partition to other partitions; performing internal edge sampling for a partition includes: collecting the original edges inside the partition according to a preset sampling ratio to form an internal edge sampling set for the corresponding partition; and performing graph topology simplification for a partition includes: merging its boundary edge sampling set with its internal edge sampling set to obtain the graph topology sampling edge set corresponding to the partition.

[0032] 1) Boundary vertex sampling.

[0033] For each partition, its boundary vertices are sampled according to a preset sampling ratio to form the boundary vertex sampling set of the corresponding partition. The boundary vertices of the partition are the original graph vertices in other partitions that are directly pointed to by the original edges of the partition.

[0034] like Figure 4 The diagram shown is a schematic diagram of topological sampling of the original graph structure in one embodiment of the present invention.

[0035] Taking partition i as an example, first determine its boundary vertex set BV based on the original graph structure. i The boundary vertices of partition i are located in other partitions, but they point directly to the original graph vertices in partition i through the original edges.

[0036] The preset sampling ratio ranges from 1% to 10%. Assuming a sampling ratio p = 0.01, in the FPGA sampling control unit, the hardware random number generator generates a random sequence according to the sampling ratio p, and the boundary vertex selector selects from its complete set of boundary vertices BV. i In this process, simple uniform random sampling is performed. The resulting boundary vertex sample set RBV is... i The size is approximately |BV i | *0.01. This step directly and significantly reduces the number of boundary vertices that need to be processed.

[0037] After determining the boundary vertex sampling set RBV i Afterwards, the cross-regional communication requirements can be clearly defined, and for each other partition j, the RBV is calculated. i With V j The intersection of (the set of internal vertices of partition j) yields the set S. i,j = RBV i ∩ V j This set specifies which vertex embeddings partition i needs to obtain from partition j in this round. Because |RBV i | is already very small, the calculated S i,j The scale is also correspondingly small.

[0038] 2) Boundary edge sampling.

[0039] For each partition, the boundary edges whose source vertices belong to the boundary vertex sampling set of that partition are selected to form the boundary edge sampling set of the corresponding partition. The boundary edges of that partition are the original edges that connect that partition to other partitions.

[0040] Taking partition i as an example, for each boundary edge (u, v) of partition i (where u ∈ BV) i (where v is the boundary vertex of partition i and v is the vertex of other partitions), check if its source vertex u is in the boundary vertex sample set RBV. i In, if u ∈ RBV i If the edge is positive, keep it; otherwise, discard it. This results in the boundary edge sampling set RBE. i It is only related to those activated sampled boundary vertices (vertices in the boundary vertex sample set).

[0041] 3) Internal edge sampling.

[0042] For each partition, the original edges inside it are collected according to a preset sampling ratio to form the internal edge sampling set of the corresponding partition.

[0043] Specifically, the system also downsamples the original edges within the partition. A parallel sampling architecture can be used, where each sampling unit independently processes a portion of the internal edge set, starting from the internal original edge set IE of partition i.i Similarly, a subset of edges is randomly sampled with probability p=0.01 to obtain the internal edge sampling set RIE. i This step further reduces the amount of computation within the partition.

[0044] 4) Graph topology simplification.

[0045] For each partition, combine its boundary edge sampling set with its internal edge sampling set to obtain the graph topology sampling edge set corresponding to that partition.

[0046] Taking partition i as an example, the system can use a multiplexer and a pipelined distribution network to sample the boundary edge set RBE. i and internal edge sampling set RIE i Merged into the final set of sampled edges RE i = RBE i ∪RIE i These edges represent the graph topology-induced information introduced from the original graph structure G0 in this iteration.

[0047] S4. Obtain the key-value data of the original graph vertices. For each fused vertex, perform attention calculation based on the original graph vertices it depends on to obtain the updated embedding of the corresponding fused vertex.

[0048] The goal of this step is to perform "fusion vertex embedding update": for each fusion vertex, which depends on multiple original graph vertices, the key-value data of the original graph vertices it depends on can be obtained and attention calculation can be performed to obtain the updated embedding of each fusion vertex.

[0049] In one embodiment, key-value data blocks can be acquired in batches. Each key-value data block contains key-value data of multiple original graph vertices. All fused vertices that depend on their original graph vertices are determined based on the fused edges. Attention calculation is performed on each fused vertex that depends on its original graph vertices using the acquired key-value data blocks to obtain the updated embedding of the corresponding fused vertex. If a fused vertex completely depends on the original graph vertex corresponding to the current key-value data block, the updated embedding of the fused vertex is directly calculated. If a fused vertex does not completely depend on the original graph vertex corresponding to the current key-value data block, the corresponding key-value data blocks are acquired in batches based on all the original graph vertices it depends on, and attention calculation is performed in batches before being accumulated to obtain the updated embedding of the fused vertex.

[0050] Specifically, based on memory capacity, the key-value data matrix corresponding to all original graph vertices can be divided into fixed-size data blocks (K / V blocks), and read in batches for attention calculation. For the currently loaded K / V block, the original graph vertices corresponding to the K / V block can be clearly identified. The system determines which fused vertices' calculations depend on the data in this block based on the local tree structure. These fused vertices are divided into two categories: the first category is completely dependent on this block (mainly low-level fused vertices); the second category is partially dependent on this block (high-level fused vertices, which depend on multiple blocks).

[0051] For example, for the j-th block K loaded j V j This corresponds to B original graph vertices, which can be quickly located based on the local tree structure: all ancestor vertices of these B original graph vertices require this block of data. The calculation of lower-level ancestors can be completed immediately, while the calculation of higher-level ancestors requires accumulation.

[0052] Attention calculation for the first type of fused vertices: Load the query vectors of the first type of fused vertices. Then, using the data from the current K / V block, directly calculate the complete attention output for these fused vertices. After calculation, write the results directly back to memory. The calculation of these fused vertices ends at this step.

[0053] Specifically, in the hardware system, for fused vertices whose computation is entirely dependent on this block (e.g., all their subordinate vertices are within these B vertices), the system loads their query vector into the register file. Using the current K... j V j The final embeddings of these fused vertices are calculated directly using standard attention computation and stored directly in the target memory area via a write-back bus. Their computational task is thus completed without waiting for other blocks.

[0054] Attention calculation for the second type of fused vertices: Load the query vectors of the second type of fused vertices. Using the data from the current K / V block, calculate the attention contribution of this block to these fused vertices. Then, using an incremental softmax algorithm, merge the current contribution with their existing partial results, updating their attention output and statistics. Write the updated partial results back to memory.

[0055] Repeat the above steps until all key-value blocks have been processed. At this point, all merged vertices have received the updated embedding.

[0056] After the updated embedding of the merged vertices is obtained, a linear transformation can be used to convert it into a key vector and a value vector, thus obtaining the key-value data of the merged vertices for subsequent use by the vertices of the original graph.

[0057] S5. Using associated edges and graph topology sampling edges as information acquisition edges, for each original graph vertex, obtain the embeddings of its neighboring vertices based on its corresponding information acquisition edges and perform attention calculation to obtain the updated embeddings of the corresponding original graph vertex; when all original graph vertices have obtained their updated embeddings, the graph attention mechanism calculation for the current attention calculation round ends.

[0058] like Figure 5 The diagram shown is a schematic diagram of embedding neighboring vertices of a target vertex in one embodiment of the present invention.

[0059] The goal of this step is to perform "original graph vertex embedding update": for each original graph vertex, its neighboring vertices based on information-acquisition edges include other original graph vertices that point to that original graph vertex via information-acquisition edges, as well as merged vertices.

[0060] Specifically, for each vertex of the original graph, based on the previously established associated edges and the filtered graph topology sampling edges, all the input data required for attention calculation are obtained and attention calculation is performed to obtain the updated embedding of the corresponding original graph vertex. The algorithm formula for attention calculation can adopt a conventional attention calculation formula. The embedded data obtained using associated edges and graph topology sampling edges as information acquisition edges includes: The previous round embedding of the vertices of the original graph; Within this partition, the previous round embedding of other original graph vertices pointing to this original graph vertex is obtained through information acquisition edges (including associated edges and graph topology sampling edges); Within this partition, the updated embedding of the fused vertex is obtained by using information to obtain the edge (associated edge) pointing to the vertex of the original graph; The previous round embedding of other original graph vertices that point to the original graph vertex through information-acquiring edges (graph topology sampling edges) within other partitions; other original graph vertices that point to the original graph vertex through graph topology sampling edges within other partitions are called "sampled boundary original graph vertices"; The previous round embedding of the fused vertex that points to the original graph vertex through information acquisition edges (associated edges) within other partitions; the fused vertex that points to the original graph vertex through associated edges within other partitions is called the "boundary fused vertex".

[0061] Taking partition i as an example, for each original graph vertex v in partition i, the system collects its neighbor embeddings from different sources: the previous round embedding of vertex v itself, the embeddings of other original graphs and fused vertices connected to it in this partition through associated edges, the embeddings of the original graph neighbor vertices in this partition sampled in this round, the embeddings of the boundary original graph neighbor vertices obtained from communication with other partitions, and the embeddings of other fused vertices read from the historical cache. Each data source (communication data, historical cache) provides data simultaneously through independent reading channels. After collection, the data is uniformly sent to the computing array to maximize the data supply bandwidth.

[0062] For each vertex in the original graph, attention can be calculated using existing attention calculation algorithms. The original graph vertex to be subject to attention is called the target vertex. The general process of attention calculation is as follows: The initial attention score is calculated by performing a dot product between the query vector of the target vertex and the key vector of each of its neighbors. Then, the initial scores of all neighboring vertices are normalized using a softmax function to obtain the normalized attention weight for each neighboring vertex. For example, for the target vertex v, its query vector is multiplied by the K vector of each neighbor to obtain the initial attention score; then, a parallel comparison tree is used to apply the softmax function to the initial attention scores of all neighbors to efficiently obtain the final normalized attention weight. Collect the value vector of each neighbor vertex, and for each neighbor vertex, use its normalized attention weight to weight its value vector to obtain a weighted value vector; for example, in a hardware system, a hardware weighting unit can be designed, which multiplies the V vector of each neighbor by the corresponding attention weight through a multiplier array to obtain a weighted value vector. The weighted value vectors of each neighbor vertex are summed to obtain the aggregated information, i.e., the aggregated vector. This aggregated information is then subjected to a linear transformation or combined with the target vertex's own information (generally, it is added to or concatenated with the target vertex's own embedding from the previous round before undergoing a linear transformation) to finally generate the updated embedding of the target vertex in the current round. For example, in a hardware system, an accumulator can be used to sum all the weighted neighbor value vectors to obtain the aggregated information.

[0063] For each original graph vertex, attention is calculated through the above process until all original graph vertices have completed the update calculation of their embedding vectors, thus ending the graph attention mechanism calculation for the current attention calculation round.

[0064] Secondly, the present invention also claims a reasoning computation system based on a graph attention mechanism.

[0065] like Figure 6The diagram shown is a structural block diagram of a graph attention mechanism inference computation system according to an embodiment of the present invention.

[0066] The inference computation system for this graph attention mechanism includes: The local tree construction module is used to divide the original graph data into multiple partitions. For each partition, the original graph vertices inside it are used as the bottom-level leaf nodes to generate parent nodes layer by layer until the top-level root node is generated, thus obtaining the local tree structure of the partition. All the parent nodes generated in the local tree structure are fused vertices. For each fused vertex, each original graph vertex it depends on is used as the source vertex to add a fused edge pointing to the fused vertex. The associated edge construction module is used to select the corresponding associated vertex set for each original graph vertex and construct the associated edges from each associated vertex to the original graph vertex. The associated vertex set includes other original graph vertices and some merged vertices within the partition to which the original graph vertex belongs, as well as some merged vertices within other partitions. The original graph vertices that each merged vertex in the associated vertex set depends on do not overlap, and the union of the original graph vertices that each merged vertex depends on and other original graph vertices in its associated vertex set is all other original graph vertices. The original graph topology sampling module performs boundary vertex sampling, boundary edge sampling, internal edge sampling, and graph topology simplification for each partition. Boundary vertex sampling for a partition includes: collecting boundary vertices at a preset sampling ratio to form a boundary vertex sampling set for the corresponding partition. The boundary vertices of this partition are the original graph vertices in other partitions that are directly connected to the vertices of this partition by original edges. Boundary edge sampling for a partition includes: filtering out boundary edges whose source vertices belong to the boundary vertex sampling set of this partition to form a boundary edge sampling set for the corresponding partition. The boundary edges of this partition are the original edges connecting this partition to other partitions. Internal edge sampling for a partition includes: collecting the original edges inside the partition at a preset sampling ratio to form an internal edge sampling set for the corresponding partition. Graph topology simplification for a partition includes: merging its boundary edge sampling set with its internal edge sampling set to obtain the graph topology sampling edge set corresponding to this partition. The fused vertex embedding update module is used to obtain the key-value data of the vertices in the original graph. For each fused vertex, it obtains the key-value data of its neighboring vertices based on its fused edges and performs attention calculation to obtain the updated embedding of the corresponding fused vertex. The original graph vertex embedding update module uses associated edges and graph topology sampling edges as information acquisition edges. For each original graph vertex, it obtains the embeddings of its neighboring vertices based on its corresponding information acquisition edges and performs attention calculations to obtain the updated embedding of the corresponding original graph vertex. When all original graph vertices have obtained their updated embeddings, the graph attention mechanism calculation for the current attention calculation round ends.

[0067] It can be understood that the inference computation system of this graph attention mechanism can be used to implement the inference computation method of the graph attention mechanism introduced above. For specific details, please refer to the above introduction, which will not be repeated here.

[0068] In one embodiment, the original graph topology sampling module described above can be further subdivided into: The boundary vertex sampling submodule is used to perform boundary vertex sampling for each partition; The boundary edge sampling submodule is used to perform boundary edge sampling for each partition; The internal edge sampling submodule is used to perform internal edge sampling for each partition; The graph topology simplification submodule is used to perform graph topology simplification for each partition.

[0069] In one embodiment, considering the advantages of the hardware system in terms of parallelism, high throughput and low latency, an FPGA can be used to build the hardware system, and a pipelined mode can be used to execute the graph attention mechanism computation.

[0070] Specifically, based on the FPGA's storage hierarchy and bandwidth characteristics, the original graph vertex sequence is divided into multiple partitions. This involves executing a lightweight graph partitioning algorithm on the FPGA's parallel-processable graph structure, with the partitioning results adapted to the on-chip Block RAM distribution. In sequence, every two vertices are merged into a group to form a new vertex in the second layer. Then, on the FPGA, parallel reduction logic merges the vertices of the second layer into groups of two in sequence to form a new vertex in the third layer, and so on, until a root node is formed, which becomes the highest level of the tree. This root node contains all vertices from the sequence, thus generating merged vertices. The entire process leverages the FPGA pipeline to improve construction efficiency. For each merged vertex (i.e., an internal node in the tree), a directed edge is added from each of its constituent original graph vertices (i.e., each leaf node in the vertex group represented by the merged vertex). The directed edge from an original graph vertex to its constituent merged vertex is the merged edge. This process is completed on-chip using address generation logic. For each partition's local vertex sequence, following the steps described above, the merged vertices and internal merged edges for that partition are generated in the same way within the FPGA's local memory, constructing a global tree on the FPGA chip. Associative edges connecting the vertices between partitions are constructed, and each associated edge is assigned to the partition containing its destination vertex.

[0071] For each FPGA partition, a hardware random number generator randomly selects a subset of its boundary vertices from its set of boundary vertices according to a preset sampling ratio p, as the sampling boundary vertices for this round. On-chip, a comparison and filtering circuit retains only those edges whose source vertices are located within the subset of sampling boundary vertices from the original boundary edge set, forming the sampling boundary edge set. For the internal edge set of each partition, a parallel sampling circuit is used to randomly sample it according to the sampling ratio p, obtaining the sampling internal edge set. On the FPGA chip, a set merging operation is performed to merge the sampling boundary edges and the sampling internal edges, resulting in the final graph topology sampling edge set used by the partition in this iteration. Based on the sampling boundary vertex set, an address mapping circuit calculates in real time which vertex embeddings this partition needs to pull from each of the other partitions, providing a basis for subsequent communication steps.

[0072] Between the FPGA's off-chip memory and Block RAM, the system divides the key and value matrices of all original graph vertices in this partition into multiple data blocks of fixed size according to the order of the vertices in the sequence. The system loads each K / V data block sequentially into the FPGA's on-chip cache via DMA. For the currently loaded K / V block (containing a set of original graph vertices), the system quickly determines which fused vertices' computation depends on the data in this block based on the global tree structure and a pre-compiled dependency lookup table. These fused vertices are divided into two categories: the first category is computationally completely dependent on this block (mainly low-level fused vertices); the second category is computationally partially dependent on this block (high-level fused vertices, whose computation depends on multiple blocks). The system loads the query vectors of the first category of fused vertices. Subsequently, using the data of the current K / V block, the complete attention outputs of these fused vertices are directly computed. After computation, the results are directly written back to memory. The computation of these fused vertices ends at this step; the system loads the query vectors of the second category (high-level) fused vertices into the computation unit, their current partial attention outputs, and the current softmax statistics. Using the data of the current K / V block, the attention contribution of this block to these fused vertices is computed. Then, using an incremental softmax algorithm, the current contribution is merged with their existing partial results, updating their attention output and statistics. The updated partial results are written back to memory. This process is repeated until all key-value blocks have been processed. At this point, all fused vertices have obtained updated embedding vectors.

[0073] The system acquires all the input data required for computation in parallel for each original graph vertex in the current partition, including: the vertex's own embedding from the previous round, the embeddings of other vertices (including original graph and fused vertices) connected to it via associated edges within the current partition, the embeddings of the original graph neighbor vertices sampled in the current round, the embeddings of the boundary original graph neighbor vertices obtained from communication with other partitions, and the embeddings of the boundary fused vertex neighbors read from the history cache. For each original graph vertex, the system performs a dot product calculation on its own query vector with the key vector of each collected neighbor vertex within the FPGA attention computation core to obtain an initial attention score. Then, the initial scores of all neighbors are input into a softmax function for normalization to obtain the final attention weights. The system uses a hardware weighting circuit to weight the value vectors of each collected neighbor vertex with the corresponding calculated attention weights. The system uses an accumulator to sum all the weighted neighbor value vectors to obtain aggregated information. Typically, this aggregated information undergoes a linear transformation or is combined with the vertex's own information to finally generate a new embedding vector for the original graph vertex in this round. Repeat the above steps until all original graph vertices in this partition have completed the embedding vector update calculation.

[0074] In one embodiment, the general flow of graph attention mechanism computation at the current level in a graph attention mechanism inference computation system is as follows: (1) At the start of each layer of attention computation, obtain the list of boundary neighbor vertices (including original graph vertices and fused vertices) and the historical embedding cache of vertices for each partition from the configuration memory; (2) Perform the "Merge Vertex Embedding Update" calculation. At the same time, retrieve the previous round embeddings of the "Sample Boundary Original Graph Vertices" required for the "Original Graph Vertex Embedding Update" of this partition from other partitions through asynchronous communication operations; specifically, two tasks are triggered here: start the "Merge Vertex Embedding Update" calculation and initiate an asynchronous communication operation to request other partitions to send the embeddings of the "Sample Boundary Original Graph Vertices" required for this partition (these embeddings are the outputs of the previous round); (3) Wait for the "fusion vertex embedding update" calculation to be completed, and wait for the "boundary original graph vertex" embedding communication to be completed, and perform state coordination through the synchronization circuit; (4) After step (3) is completed, the “original graph vertex embedding update” calculation is started. This calculation uses the fused vertex embedding generated in step (3), the previously received “boundary original graph vertex” embedding, and the “boundary fused vertex” embedding in the historical cache; (5) Wait for the "original graph vertex embedding update" calculation to be completed; (6) Write the original graph vertex embedding update and the fused vertex embedding update to the corresponding positions in the historical embedding cache, overwrite the old values, and provide data for the calculation of the next iteration; (7) The attention calculation and scheduling of the current layer is completed. The updated original graph vertex embedding and fused vertex embedding are passed to the next layer of the model to enter the next round of attention calculation. Steps (1) to (6) are repeated until all layers of the model are calculated.

[0075] In general, the technical solution of the present invention and its optional embodiments can achieve the following beneficial effects: 1. Improved computational efficiency: By constructing a sparse graph based on a graph topology-induced attention computation mechanism using a global tree, the computational complexity of standard fully connected attention is reduced from O(N²) to a near-linear level, significantly improving the maximum sequence length supported by the system; 2. Systematic optimization of communication overhead: The original tree structure perception graph partitioning scheme reduces the number of cross-partition edges from the source and explicitly reduces the number of boundary vertices that need to be processed in each round based on the dynamic sampling strategy. In the optional scheme, the communication and computation are further overlapped through the pipeline scheme to form a multi-level and systematic communication optimization, which significantly improves the scalability of the system in the distributed environment. 3. Balance between accuracy and efficiency: The tree structure is used to construct fused vertices to learn global information and perform graph topology sampling. While achieving high computational sparsity, the model accuracy is avoided from significant loss. This makes the convergence accuracy of the model using this scheme comparable to that of the standard fully connected attention model. 4. Complete and systematic solution: This invention provides a complete and collaborative optimization scheme, which achieves a systematic breakthrough in overcoming the bottleneck of large-scale graph Transformer inference computation and brings about a comprehensive improvement in inference computation performance.

[0076] The technical features of the embodiments described above can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. It should be noted that the terms "in one embodiment," "for example," and "again" in this invention are intended to illustrate the invention and are not intended to limit the invention.

[0077] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A reasoning computation method based on graph attention mechanism, characterized in that, include: The original graph data is divided into multiple partitions. For each partition, the original graph vertices within it are used as the bottom-level leaf nodes to generate parent nodes layer by layer until the top-level root node is generated, thus obtaining the local tree structure of the partition. All the parent nodes generated in the local tree structure are fused vertices. For each fused vertex, each original graph vertex it depends on is used as the source vertex to add a fused edge pointing to the fused vertex. For each original graph vertex, select its corresponding set of associated vertices and construct the associated edges from each associated vertex to the original graph vertex. The set of associated vertices includes other original graph vertices and some merged vertices within the partition to which the original graph vertex belongs, as well as some merged vertices within other partitions. The original graph vertices on which each fused vertex in its associated vertex set depends do not overlap, and the union of the original graph vertices on which each fused vertex depends with other original graph vertices in its associated vertex set is all other original graph vertices. For each partition, boundary vertex sampling, boundary edge sampling, internal edge sampling, and graph topology simplification are performed separately. Specifically, boundary vertex sampling for a partition includes: collecting boundary vertices at a preset sampling ratio to form a boundary vertex sampling set for the corresponding partition. The boundary vertices of this partition are the original graph vertices from other partitions that directly point to the vertices of this partition via original edges. Boundary edge sampling for a partition includes: selecting boundary edges whose source vertices belong to the boundary vertex sampling set of this partition to form a boundary edge sampling set for the corresponding partition. The boundary edges of this partition are the original edges connecting this partition to other partitions. Internal edge sampling for a partition includes: collecting the original edges within the partition at a preset sampling ratio to form an internal edge sampling set for the corresponding partition. Graph topology simplification for a partition includes: merging its boundary edge sampling set with its internal edge sampling set to obtain the graph topology sampling edge set corresponding to this partition. Obtain the key-value data of the vertices in the original graph. For each fused vertex, obtain the key-value data of its neighboring vertices based on its fused edges and perform attention calculation to obtain the updated embedding of the corresponding fused vertex. Using associated edges and graph topology sampling edges as information acquisition edges, for each original graph vertex, the embeddings of its neighboring vertices are obtained based on its corresponding information acquisition edges, and attention is calculated to obtain the updated embeddings of the corresponding original graph vertex. When all original graph vertices have obtained their updated embeddings, the graph attention mechanism calculation for the current attention calculation round ends.

2. The reasoning computation method for the graph attention mechanism as described in claim 1, characterized in that, The original graph data is divided into multiple partitions, including: the classic METIS graph partitioning algorithm is used for partitioning, and the objective function is to minimize edge cuts and maximize the balance of the number of vertices between partitions.

3. The reasoning computation method for the graph attention mechanism as described in claim 1, characterized in that, Each parent node in the local tree structure corresponds to two child nodes.

4. The reasoning computation method of the graph attention mechanism as described in claim 1, characterized in that, The local structure trees of all partitions constitute the global structure tree. When constructing associated edges for any original graph vertex, the position number of the original graph vertex is used as the reference position. Starting from the bottom layer of the global structure tree, associated vertices on the left and right sides adjacent to the reference position are searched layer by layer and associated edges pointing to the original graph vertex are added. The number of associated vertices located on the same side of the reference position in the same layer is limited to m, where m is a preset hyperparameter, and m is 2 or 4.

5. The reasoning computation method for the graph attention mechanism as described in claim 1, characterized in that, When performing boundary vertex sampling and internal edge sampling, the preset sampling ratio ranges from 1% to 10%.

6. The reasoning computation method for the graph attention mechanism as described in claim 1, characterized in that, The process of calculating the updated embedding of a fused vertex includes: acquiring key-value data blocks in batches, each key-value data block containing key-value data of multiple original graph vertices; determining all fused vertices that depend on their original graph vertices based on the fused edges; and performing attention calculations on each fused vertex that depends on its original graph vertices using the acquired key-value data blocks to obtain the updated embedding of the corresponding fused vertex. Specifically, if a fused vertex is completely dependent on the original graph vertex corresponding to the current key-value data block, the updated embedding of the fused vertex is directly calculated. If a fused vertex is not completely dependent on the original graph vertex corresponding to the current key-value data block, the corresponding key-value data blocks are acquired in batches based on all the original graph vertices it depends on, attention calculations are performed in batches, and then incremental calculations are performed to obtain the updated embedding of the fused vertex.

7. A reasoning computation system based on graph attention mechanism, characterized in that, include: The local tree construction module is used to divide the original graph data into multiple partitions. For each partition, the original graph vertices inside it are used as the bottom-level leaf nodes to generate parent nodes layer by layer until the top-level root node is generated, thus obtaining the local tree structure of the partition. All the parent nodes generated in the local tree structure are fused vertices. For each fused vertex, each original graph vertex it depends on is used as the source vertex to add a fused edge pointing to the fused vertex. The associated edge construction module is used to select the corresponding associated vertex set for each original graph vertex and construct the associated edges from each associated vertex to the original graph vertex. The associated vertex set includes other original graph vertices and some merged vertices within the partition to which the original graph vertex belongs, as well as some merged vertices within other partitions. The original graph vertices on which each fused vertex in its associated vertex set depends do not overlap, and the union of the original graph vertices on which each fused vertex depends with other original graph vertices in its associated vertex set is all other original graph vertices. The original graph topology sampling module performs boundary vertex sampling, boundary edge sampling, internal edge sampling, and graph topology simplification for each partition. Boundary vertex sampling for a partition includes: collecting boundary vertices at a preset sampling ratio to form a boundary vertex sampling set for the corresponding partition. The boundary vertices of this partition are the original graph vertices in other partitions that are directly connected to the vertices of this partition by original edges. Boundary edge sampling for a partition includes: filtering out boundary edges whose source vertices belong to the boundary vertex sampling set of this partition to form a boundary edge sampling set for the corresponding partition. The boundary edges of this partition are the original edges connecting this partition to other partitions. Internal edge sampling for a partition includes: collecting the original edges inside the partition at a preset sampling ratio to form an internal edge sampling set for the corresponding partition. Graph topology simplification for a partition includes: merging its boundary edge sampling set with its internal edge sampling set to obtain the graph topology sampling edge set corresponding to this partition. The fused vertex embedding update module is used to obtain the key-value data of the vertices in the original graph. For each fused vertex, it obtains the key-value data of its neighboring vertices based on its fused edges and performs attention calculation to obtain the updated embedding of the corresponding fused vertex. The original graph vertex embedding update module uses associated edges and graph topology sampling edges as information acquisition edges. For each original graph vertex, it obtains the embeddings of its neighboring vertices based on its corresponding information acquisition edges and performs attention calculations to obtain the updated embedding of the corresponding original graph vertex. When all original graph vertices have obtained their updated embeddings, the graph attention mechanism calculation for the current attention calculation round ends.

8. The inference computation system based on the graph attention mechanism as described in claim 7, characterized in that, The original graph topology sampling module specifically includes: The boundary vertex sampling submodule is used to perform boundary vertex sampling for each partition; The boundary edge sampling submodule is used to perform boundary edge sampling for each partition; The internal edge sampling submodule is used to perform internal edge sampling for each partition; The graph topology simplification submodule is used to perform graph topology simplification for each partition.

9. The inference computation system based on the graph attention mechanism as described in claim 7, characterized in that, The inference computing system is a hardware system built on FPGA.

10. The inference computation system based on the graph attention mechanism as described in claim 9, characterized in that, The FPGA performs graph attention mechanism calculations in a pipelined manner.

Images