Dynamic heterogeneous graph neural network reasoning device and method based on FPGA

Abstract

The invention relates to a dynamic heterogeneous graph neural network reasoning device and method based on an FPGA. The device comprises a hierarchical data loading module and a collaborative feature aggregation module. The hierarchical data loading module senses and captures changes of a vertex topological structure between continuous snapshots in the dynamic heterogeneous graph in real time; meanwhile, neighbors of different target vertexes under each timestamp are combined into neighbor set information, and corresponding feature vectors are sequentially loaded according to the occurrence frequency from high to low; the collaborative feature aggregation module receives neighbor set information, reads feature vectors corresponding to the neighbor set information, decouples an incremental calculation process into two parts of matrix vector multiplication and element calculation of corresponding positions, and completes an operation process of feature vector multiplication between a target vertex and a neighbor based on a local aggregation unit formed by a systolic array; and organizing the calculation units into a single instruction stream and multiple data streams based on the feature collection unit so as to complete normalization of calculation results of the same vertex.

Description

Technical Field

[0001] This invention relates to the field of graph computing technology, and in particular to a dynamic heterogeneous graph neural network inference device and method based on FPGA. Background Technology

[0002] With the advent of the big data era, graph processing systems are becoming increasingly important. Real-world relational networks (such as social interaction networks, traffic flow networks, and e-commerce transaction networks) not only contain multiple types of nodes and relationships but are also constantly changing. To effectively capture and analyze this complex and continuously evolving relational data, Dynamic Heterogeneous Graph Neural Networks (DHGNNs) have emerged and demonstrated strong application potential. This DHGNN can integrate heterogeneous information from multiple types of nodes and edges, while effectively modeling the dynamic characteristics of graph structures evolving over time. Therefore, many current key graph applications (such as financial risk prediction, cybersecurity analysis, and e-commerce platform recommendations) employ DHGNNs for graph data analysis.

[0003] Existing dynamic heterogeneous graph neural network inference systems require generating computation results for each snapshot corresponding to a specific timestamp. Since the graph structure changes only slightly between adjacent snapshots, the system often needs to execute a computation process nearly identical to the previous snapshot. This leads to the repeated execution of a large number of reusable computation results, resulting in the repeated reading of feature vectors of the same vertices between snapshots, ultimately causing inefficient inference. Furthermore, the inherent irregularity of graph data connections leads to irregular memory access patterns for graph vertices and their neighboring nodes during inference. These irregular memory access patterns are difficult to predict and capture effectively by traditional cache hierarchies, resulting in low on-chip cache hit rates on traditional processors (such as CPUs and GPUs), frequently triggering high-latency off-chip data access operations, further exacerbating the performance bottleneck.

[0004] Therefore, when traditional processors execute dynamic heterogeneous graph neural network inference, there is a lot of redundant computation and frequent off-chip access, which severely restricts the efficiency of the inference process.

[0005] To address the inefficiency of traditional processor architectures, graph neural network acceleration technology based on Field-Programmable Gate Arrays (FPGAs) has become a mainstream research direction. However, existing FPGA-based dynamic graph neural network acceleration devices can only reuse computation results in scenarios where neighboring nodes remain completely unchanged, and such scenarios with fully reusable computation results account for a very small percentage of dynamic heterogeneous graphs. Because these FPGA-based graph neural network acceleration methods lack effective analysis mechanisms for intermediate computation results and ignore the large number of partially reusable intermediate computation results between snapshots of dynamic heterogeneous graph neural networks, existing solutions cannot fully utilize hardware resources to maximize inference efficiency, ultimately resulting in slow inference speeds.

[0006] CN113849599A discloses a joint fake news detection method based on pattern information and factual information. The technical solution employed in this invention is as follows: extracting pattern information trigger words, factual information trigger words, and ordinary words from the news to be tested; constructing a heterogeneous graph; applying dynamic heterogeneous graph convolution operations to the heterogeneous graph to dynamically update the edge weights between pairs of nodes, generating pattern preference distributions and factual preference distributions; obtaining a pattern information-aware aggregated vector based on the news to be tested and the pattern preference distribution; obtaining a factual information-aware aggregated vector based on the news to be tested and the factual preference distribution, and obtaining a factual information-aware inference vector based on the factual information-aware aggregated vector and relevant evidence corresponding to the news to be tested; determining the authenticity of the news to be tested based on the pattern information-aware aggregated vector and the factual information-aware inference vector. This technical solution addresses the semantic modeling problem in fake news detection, belonging to the content understanding method at the algorithmic model level. It completely fails to address any technical means related to redundant computation elimination, off-chip access optimization, or hardware resource scheduling in dynamic graph inference, and therefore cannot solve the technical problem proposed in this invention.

[0007] This invention aims to provide an FPGA-based dynamic heterogeneous graph neural network inference device and method, and to solve the problems of computational redundancy, frequent off-chip access, and low storage efficiency in the inference process of graph neural networks in dynamic graph scenarios.

[0008] Furthermore, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making this invention, but due to space limitations, not all details and contents were listed in detail. However, this does not mean that the present invention does not possess the features of these prior art. On the contrary, the present invention already possesses all the features of the prior art, and the applicant reserves the right to add relevant prior art to the background art. Summary of the Invention

[0009] The fundamental reason for the low inference efficiency of existing FPGA-based dynamic graph neural network accelerators lies in their overly idealistic design of computation reuse mechanisms in dynamic heterogeneous graphs. These methods typically assume that computation results from the previous time step can only be reused if neighboring nodes remain completely unchanged. However, in reality, the topology and node attributes of dynamic heterogeneous graphs frequently undergo local changes, making completely unchanged neighboring nodes extremely rare, resulting in a very low percentage of scenarios where full reuse is possible. Consequently, the accelerator fails to trigger result reuse most of the time, requiring the complete computation to be repeated, leading to a significant waste of computational resources.

[0010] A deeper problem lies in the lack of fine-grained analysis and management mechanisms for intermediate computation results. During dynamic graph reasoning, although a node's neighbor set may change, there are often many unchanged substructures or shared intermediate features (such as neighbor embeddings before aggregation), which possess partial reusability. However, existing FPGA acceleration solutions lack effective comparison and incremental update mechanisms, failing to identify and utilize these partially overlapping computation results. Furthermore, the diversity of node and edge types in heterogeneous graphs further increases the complexity of intermediate data; without structure-aware caching strategies and dependency analysis capabilities, hardware struggles to automatically extract reuse opportunities.

[0011] As mentioned above, existing FPGA accelerators cannot achieve efficient incremental inference in dynamic heterogeneous graph scenarios because they rely solely on fully matched reuse conditions and lack the ability to mine partially reusable local aggregation results. This leads to frequent repetitive computations, and the hardware parallelism and storage bandwidth are not fully utilized, ultimately limiting the overall inference speed.

[0012] To address the shortcomings of existing technologies, this invention provides, from a first aspect, an FPGA-based dynamic heterogeneous graph neural network inference device. This device includes a hierarchical data loading module and a collaborative feature aggregation module. The hierarchical data loading module senses and captures changes in vertex topology between consecutive snapshots in the dynamic heterogeneous graph in real time; simultaneously, it merges the neighbors of different target vertices at each time stamp into a neighbor set information, and loads the corresponding feature vectors sequentially according to their frequency of occurrence from high to low. The collaborative feature aggregation module receives the neighbor set information and reads its corresponding feature vectors, decoupling the incremental calculation process into two parts: matrix-vector multiplication and element-wise calculation at corresponding positions. It completes the feature vector multiplication operation between the target vertex and its neighbors based on a local aggregation unit composed of a systolic array. Based on the feature collection unit, the computation unit is organized into a single-instruction stream with multiple data streams to normalize the calculation results for the same vertex.

[0013] This invention uses a hierarchical data loading module to perceive graph changes in real time and load data according to the frequency of feature vector occurrences, effectively identifying reusable computations. Incremental computation is decoupled into matrix-vector multiplication and element-wise computation, achieving local aggregation through a pulsating array. This allows FPGAs to fully utilize unchanged substructures in dynamic graphs, significantly improving computational reuse, reducing redundant computations, fully leveraging the hardware parallelism and memory bandwidth advantages of FPGAs, and significantly enhancing the inference efficiency of dynamic heterogeneous graphs.

[0014] According to a preferred embodiment, the hierarchical data loading module includes a graph structure loading unit, a cross-snapshot vertex filtering unit, and a feature loading unit.

[0015] The graph structure loading unit analyzes the graph structure at different timestamps during loading, uncovering the changes in vertices between snapshots at adjacent timestamps; it merges the neighbor information of the target vertex at each timestamp into a neighbor set information; the cross-snapshot vertex filtering unit analyzes the topological changes between snapshots at adjacent timestamps in the dynamic heterogeneous graph in real time, identifies the addition, deletion, and changes in connection relationships of target vertices, and constructs complete neighbor set information and an incremental information array describing the changes in the neighbor set between adjacent snapshots; the feature loading unit divides the target vertices within a preset neighborhood into the same loading batch based on the spatial proximity principle of graph topology; it constructs a vertex-frequency statistical mapping table based on the neighbor set information of the target vertices in the same batch, sorts them from high to low based on the number of references in the vertex-frequency statistical mapping table, and loads the feature vectors of vertices in order of priority from high to low.

[0016] Constructing an incremental information array allows for precise identification of neighbor changes (additions / deletions / connections), enabling partial reuse. The feature loading unit loads feature vectors based on a vertex-frequency statistics map, prioritizing them by reference, and provides fine-grained management of unchanged substructures and shared intermediate features. This allows the FPGA to automatically identify and utilize frequently occurring unchanged parts in the dynamic graph, avoiding redundant calculations, significantly improving computational reuse, fully leveraging hardware parallelism and storage bandwidth, and solving the resource waste problem caused by low reuse rates in existing solutions.

[0017] According to a preferred embodiment, the collaborative feature aggregation module includes a local aggregation unit and a feature collection unit. The local aggregation unit includes several systolic arrays. The systolic arrays perform parallel multiplication operations between sparse matrices and vectors to calculate the local aggregation result after aggregating the target vertex with all its neighbors; the feature collection unit calculates the feature vectors of the elements corresponding to the changed vertex positions and standardizes the feature vectors, and applies incremental changes to the basic aggregation result based on the vertex change information in the incremental information array.

[0018] The local aggregation unit computes the local aggregation results in parallel using a systolic array, while the feature collection unit applies incremental changes to the basic aggregation results based on an incremental information array, accurately identifying and utilizing unchanged substructures. This allows the FPGA to automatically handle partial changes, avoiding full recalculation and significantly improving computational reuse. Simultaneously, the parallel processing of the systolic array fully utilizes hardware resources, effectively solving the problems of redundant computation and hardware resource waste caused by low reuse rates, and significantly improving the inference efficiency of dynamic heterogeneous graphs.

[0019] According to a preferred embodiment, the local aggregation unit in the collaborative feature aggregation module includes a vertex reading unit, a demultiplexer, a feature grabbing unit, and an aggregation unit. The vertex reading unit reads complete neighbor set information and an incremental information array related to the current target vertex from the cross-snapshot vertex filtering unit. The demultiplexer sends the neighbor set information to the feature grabbing unit and the incremental information array to the aggregation unit. The feature grabbing unit receives the neighbor set information, parses the vertex ID, and reads the feature vectors corresponding to neighbor vertices in batches from the on-chip cache. The aggregation unit performs matrix-vector multiplication of the feature vectors with the target vertex in sequence, and after completing the calculation, sends the vertex ID and the local aggregation result to the local aggregation buffer.

[0020] By accurately describing neighbor changes through an incremental information array, the feature extraction unit reads unchanged feature vectors in batches, and the aggregation unit applies incremental updates only to the changed parts. This enables the FPGA to automatically identify and reuse unchanged substructures, avoiding full recalculation and significantly improving computational reuse. Simultaneously, the incremental mechanism, in conjunction with the systolic array, fully utilizes hardware parallelism and storage bandwidth to solve the problem of redundant computation, significantly improving the inference efficiency of dynamic heterogeneous graphs.

[0021] According to a preferred embodiment, the feature collection unit in the collaborative feature aggregation module includes a snapshot update reader, a signal receiver, and a collection unit. The snapshot update reader is used to receive an incremental information array from the demultiplexer and update the incremental information array in real time; the signal receiver is used to receive an incremental calculation ready signal from the local aggregation unit; the collection unit selectively reads the local aggregation results corresponding to the changed vertices based on the incremental information array; calculates the feature vectors of the elements corresponding to the positions of the changed vertices, calculates the standardization factor and standardizes the feature vectors, and applies incremental changes to the basic aggregation results according to the vertex change information in the incremental information array.

[0022] The feature collection unit accurately locates changing vertices through incremental information arrays, selectively reads and updates local aggregation results, and achieves automatic reuse of unchanged substructures. This fine-grained incremental update mechanism enables the FPGA to effectively utilize frequently occurring unchanged parts in the dynamic graph, significantly improving computational reuse and avoiding full computation. Simultaneously, it works in conjunction with the parallel computing architecture of the systolic array to maximize hardware resource utilization, solve the resource waste problem caused by low reuse rate, and significantly improve inference efficiency.

[0023] According to a preferred embodiment, the cross-snapshot vertex filtering unit in the hierarchical data loading module includes a snapshot buffer, a vertex comparator, and a filtering result buffer. The snapshot buffer is used to store the snapshots loaded from the graph structure loading unit; the vertex comparator is used to read the vertices in neighboring snapshots in parallel, wherein the vertex comparators at odd and even positions traverse the vertices in neighboring snapshots in an alternating manner, filtering out the incremental information array of additions and subtractions between adjacent snapshots and the neighbor set information of all vertices that have appeared, and forming a filtering queue; the filtering result buffer is used to temporarily store the filtering results.

[0024] This invention addresses the core issue of low efficiency in existing FPGA dynamic graph accelerators by employing a cross-snapshot vertex filtering unit within the hierarchical data loading module. The vertex comparator traverses neighbor snapshot vertices in parallel, filtering out incremental information arrays and neighbor sets, thus achieving a shift from "complete matching" to "incremental matching." The technical advantages are: 1) It breaks through the idealized assumption of "neighbor nodes remaining completely unchanged," identifying unchanged substructures and shared intermediate features, significantly increasing the proportion of reusable scenarios; 2) It effectively utilizes some reusable computation results by forming a structure-aware caching strategy through the incremental information array; 3) The parallel processing mechanism optimizes hardware resource utilization, avoids redundant computation, and significantly improves inference efficiency and storage bandwidth utilization.

[0025] According to a preferred embodiment, the apparatus also includes high-bandwidth storage for aggregated feature vectors and infrequently accessed graph data, supporting batch write-back of aggregated results for multiple vertices.

[0026] This invention stores aggregated feature vectors and low-frequency graph data using a high-bandwidth storage mechanism, supporting batch write-back. This directly addresses the shortcomings of the "perfect match" reuse assumption in existing technologies, enabling accelerators to identify and utilize some reusable intermediate features (such as unchanged neighbor embeddings), significantly increasing the proportion of reuse scenarios. High-bandwidth storage optimizes data access efficiency, avoids redundant computation, and fully releases hardware parallelism and storage bandwidth through the batch write-back mechanism, enabling efficient incremental updates for dynamic graph inference and significantly improving overall inference speed.

[0027] The present invention provides, from a second aspect, a method for inference of a dynamic heterogeneous graph neural network based on FPGA. The method includes: real-time sensing and capturing changes in vertex topology between consecutive snapshots in a dynamic heterogeneous graph; simultaneously, merging the neighbors of different target vertices at each time stamp into a neighbor set information, and loading the corresponding feature vectors in descending order of frequency of occurrence; receiving the neighbor set information and reading its corresponding feature vectors, decoupling the incremental calculation process into two parts: matrix-vector multiplication and element-wise calculation at corresponding positions, and completing the multiplication of feature vectors between the target vertex and its neighbors based on a local aggregation unit composed of a systolic array; and organizing the calculation unit into a single instruction stream and multiple data stream based on the feature collection unit to normalize the calculation results for the same vertex.

[0028] This invention overcomes the shortcomings of existing methods that rely solely on full-match reuse by introducing a fine-grained incremental computation mechanism. Its technical advantages are: 1) Real-time perception of topological changes and identification of locally unchanged substructures, enabling the reuse of some intermediate results; 2) Decoupling aggregation computation into matrix-vector multiplication and element-level operations, adapting to systolic arrays for efficient execution; 3) Scheduling feature loading based on neighbor occurrence frequency, improving cache hit rate. This significantly reduces redundant computation, fully utilizes FPGA parallelism and bandwidth, and improves the inference efficiency of dynamic heterogeneous graphs.

[0029] According to a preferred embodiment, the steps of loading corresponding feature vectors sequentially according to their frequency of occurrence from high to low include: analyzing the graph structure under different loading timestamps to discover the changes in vertices between snapshots under adjacent timestamps; merging the neighbor information of the target vertex under each timestamp into a neighbor set information; analyzing the topological changes between snapshots under adjacent timestamps in the dynamic heterogeneous graph in real time, identifying the addition, deletion, and changes in connection relationships of the target vertex, and constructing complete neighbor set information and an incremental information array describing the changes in the neighbor set between adjacent snapshots; dividing the target vertices within a preset neighborhood into the same loading batch based on the spatial proximity principle of graph topology; constructing a vertex-frequency statistical mapping table based on the neighbor set information of the target vertices in the same batch, sorting the reference counts in the vertex-frequency statistical mapping table from high to low, and loading the feature vectors of the vertices sequentially according to their priority from high to low.

[0030] This invention accurately identifies local topological changes by constructing a neighbor set and an incremental information array, overcoming the limitation of existing methods that only support full reuse. It prioritizes loading high-frequency feature vectors based on frequency statistics to improve caching efficiency and enhances data locality by batch processing vertices using spatial proximity. This enables effective mining and scheduling of partially reusable intermediate results, significantly reducing redundant computations and improving FPGA storage bandwidth utilization and inference throughput.

[0031] According to a preferred embodiment, the steps for normalizing the calculation results for the same vertex include: performing multiplication operations between sparse matrices and vectors in parallel to calculate the local aggregation result after aggregating the target vertex with all its neighbors; calculating the eigenvectors of the elements corresponding to the changed vertex positions and standardizing the eigenvectors; and applying incremental changes to the basic aggregation result based on the vertex change information in the incremental information array.

[0032] This invention precisely locates changed vertices using an incremental information array, performing standardization and aggregation updates only on the affected parts, avoiding full recalculation and overcoming the shortcomings of existing methods that rely on complete matching reuse. Simultaneously, parallel sparse matrix-vector multiplication combined with an incremental correction mechanism effectively mines the reusability of locally unchanged substructures. This significantly reduces redundant computation, improves the utilization efficiency of FPGA computing resources and storage bandwidth, and enables efficient incremental inference. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the COSH architecture of the FPGA-based dynamic heterogeneous graph neural network inference device provided by the present invention; Figure 2 This is a schematic diagram of the cross-snapshot vertex filtering unit architecture provided by the present invention; Figure 3 This is a diagram illustrating the architecture and workflow of the collaborative feature aggregation module provided by this invention. Figure 4 The target vertex provided by this invention A schematic diagram of the polymerization process; Figure 5 This is a schematic diagram of the collaborative feature aggregation process provided by the present invention; Figure 6 This is a flowchart of the hierarchical graph data loading process provided by the present invention.

[0034] List of reference numerals 100: High-bandwidth storage; 110: Graph structure data; 120: Initial vertex feature vectors; 130: Vertex feature aggregation results; 200: Hierarchical data loading module; 210: Graph structure loading unit; 220: Feature loading unit; 230: Cross-snapshot vertex filtering unit; 231: Snapshot buffer; 232: Vertex comparator; 233: Filtering queue; 234: Filtering result buffer; 240: Task scheduler; 300: Cooperative feature aggregation module; 310: Local aggregation unit; 320: Pulsating array; 321: Vertex reading unit 322: Demultiplexer; 323: Feature extraction unit; 324: Aggregation unit; 325: Local register; 330: Feature collection unit; 331: Snapshot update reader; 332: Signal receiver; 333: Collection unit; 334: Collection register; 340: Local aggregation buffer; 410: Snapshot; 420: Filtering result; 430: Target vertex; 440: Feature vector; 450: Output result; 500: Dynamic heterogeneous graph; 510: Aggregation result within a relation; 520: Aggregation result between relations; 530: Temporal aggregation result. Detailed Implementation

[0035] The following is a detailed explanation with reference to the accompanying drawings.

[0036] Existing FPGA accelerator designs lack adaptability to dynamic graph changes. In dynamic heterogeneous graphs, node connections and attributes are continuously updated over time, but current FPGA acceleration methods only reuse the computation results of the previous round if the set of neighboring nodes remains completely unchanged. However, completely unchanging neighbor structures are extremely rare in real-world scenarios, resulting in very few reuse opportunities. The root cause is that these methods lack fine-grained analysis and management mechanisms for intermediate computation results (such as node features and aggregate intermediate values). Graph neural network aggregation operations exhibit locality; even if some neighbors change, the local aggregation results contributed by unchanged neighbors can still be reused. However, because FPGA designs lack effective caching and comparison logic for local aggregation results, all neighbors must be recalculated with each update, resulting in a large amount of redundant computation.

[0037] Furthermore, heterogeneous graphs contain multiple types of nodes and edges, and their feature spaces and aggregation rules are more complex, further amplifying the overhead of repeated computations. The hardware parallelism advantages of FPGAs cannot be fully realized due to the lack of task scheduling optimization and data reuse mechanisms, resulting in low resource utilization and limited inference throughput.

[0038] The main idea of ​​the dynamic heterogeneous graph neural network inference device and method proposed in this invention is as follows: During the inference process, the feature vector 440, which has a large reading cost, is first optimized by utilizing graph structure information with low reading cost. At the same time, the graph structure between adjacent snapshots is analyzed to obtain the relationship between the neighboring vertices of the same vertex in all snapshots. The existing calculation results are maximized by incremental calculation, which reduces the large amount of redundant calculation and irregular off-chip access overhead caused by sequential calculation of each snapshot in the traditional method.

[0039] The FPGA-based dynamic heterogeneous graph neural network inference device of the present invention can also be called an FPGA-based dynamic heterogeneous graph neural network inference accelerator. The present invention can provide a hardware accelerator for executing the dynamic heterogeneous graph neural network inference method of the present invention.

[0040] This invention provides an FPGA-based dynamic heterogeneous graph neural network inference device, such as... Figure 1 As shown, the device includes a hierarchical data loading module 200 (HDL) and a collaborative feature aggregation module 300 (CFA). Figure 1 As shown, the FPGA-based dynamic heterogeneous graph neural network inference device of the present invention also includes high-bandwidth memory 100 for aggregated feature vectors 440 and infrequently accessed graph data, supporting batch write-back of aggregation results for multiple vertices. Figure 1 As shown, the high-bandwidth storage 100 is used to store graph structure data 110, initial vertex feature vectors 120, vertex feature aggregation results 130, and other data.

[0041] like Figure 1 As shown, the FPGA-based dynamic heterogeneous graph neural network inference device adopts the COSH architecture. High-bandwidth storage 100 is connected to the hierarchical data loading module 200 for data transmission. The hierarchical data loading module 200 is connected to the collaborative feature aggregation module 300 for data transmission. Preferably, the task scheduler 240 is connected to both the hierarchical data loading module 200 and the collaborative feature aggregation module 300 for task scheduling.

[0042] To more rationally store the feature vectors 440 and local aggregation results that are dynamically generated during the computation process and have different lifecycles and access frequencies, the storage subsystem in the device of the present invention adopts a three-level storage architecture design to match the access requirements and storage characteristics of different data.

[0043] Block Random Access Memory (BRAM): Used to store graph structure data 110 with high random access and relatively dispersed data distribution, as well as the initial feature vectors 440 corresponding to the vertices. This type of data needs to support low-latency reading to quickly supply the subsequent cross-snapshot vertex filtering unit 230 and collaborative feature aggregation unit 324, ensuring the real-time performance of front-end computation.

[0044] Universal Random Access Memory (URAM): Used to store the local aggregation results generated by the collaborative feature aggregation unit 324. The core reasons are twofold: first, for the same vertex, the scale of the generated local aggregation results is typically 2 to 5 times that of the initial feature vector 440, requiring adaptation to a larger single-vertex data volume; second, these local aggregation results exhibit significant sequential access characteristics during the subsequent feature collection unit 330's computation, eliminating the need for high random access performance. Furthermore, these local aggregation results are typically read by subsequent units only after several cycles, making them less sensitive to storage latency.

[0045] High Bandwidth Storage 100 (HBM): Used to store the results of aggregations completed during computation, as well as some infrequently accessed data. This storage module uses the HBM's AXI (Advanced Extensible Interface) to read and write the aggregation results of multiple vertices in batches using a burst transfer mechanism. By increasing the amount of data transferred in a single transaction, it reduces the impact of latency on overall computational efficiency.

[0046] This invention designs a three-level storage architecture to store feature vectors 440 and local aggregation results that are dynamically generated during computation and have different lifecycles and access frequencies. This architecture rationally utilizes three heterogeneous storage resources—BRAM, URAM, and HBM—to store the vertex feature vectors 440 and the local aggregation results generated during computation, aiming to achieve a balance between access latency, storage capacity, bandwidth requirements, and on-chip resources.

[0047] The hierarchical data loading module 200 senses and captures changes in vertex topology between consecutive snapshots in the dynamic heterogeneous graph 500 in real time. The hierarchical data loading module 200 merges the neighbors of different target vertices 430 at each time stamp into neighbor set information, and loads the corresponding feature vectors 440 in descending order of frequency of occurrence.

[0048] The collaborative feature aggregation module 300 receives neighbor set information and reads the corresponding feature vector 440. The collaborative feature aggregation module 300 decouples the incremental calculation process into two parts: matrix-vector multiplication and element-wise calculation at corresponding positions. It completes the multiplication of feature vectors 440 between the target vertex 430 and its neighbors based on a local aggregation unit 310 composed of multiple systolic arrays 320. The collaborative feature aggregation module 300 organizes the calculation units into a single-instruction, multi-data stream based on the feature collection unit 330 to normalize the calculation results for the same vertex.

[0049] According to a preferred embodiment, such as Figure 1 As shown, the hierarchical data loading module 200 includes a graph structure loading unit 210, a cross-snapshot vertex filtering unit 230, and a feature loading unit 220. The graph structure loading unit 210, the cross-snapshot vertex filtering unit 230, and the feature loading unit 220 are connected sequentially. Preferably, the graph structure loading unit 210 is connected to the high-bandwidth storage 100. The feature loading unit 220 is connected to the collaborative feature aggregation module 300 through the task scheduler 240.

[0050] like Figure 1 As shown, the graph structure loading unit 210 reads graph structure data 110 and initial vertex feature vectors 120 from the high-bandwidth storage 100. The graph structure loading unit 210 analyzes the graph structures loaded at different timestamps to uncover the changes in vertices between snapshots at adjacent timestamps. The graph structure loading unit 210 merges the neighbor information of the target vertex 430 at each timestamp into a neighbor set information. The graph structure loading unit 210 sends the neighbor set information and the incremental information array to the cross-snapshot vertex filtering unit 230.

[0051] The cross-snapshot vertex filtering unit 230 receives neighbor set information from the graph structure loading unit 210. Based on the neighbor set information, the cross-snapshot vertex filtering unit 230 analyzes in real time the topological changes between snapshots 410 at adjacent timestamps in the dynamic heterogeneous graph 500, identifies the addition, deletion, and changes in connection relationships of target vertices 430, and constructs complete neighbor set information and an incremental information array describing the changes in the neighbor set between adjacent snapshots. The cross-snapshot vertex filtering unit 230 sends the complete neighbor set information and the incremental information array to the feature loading unit 220.

[0052] Preferably, the complete neighbor set information includes the IDs of all neighbor vertices of the target vertex in the first snapshot, used to guide the batch reading of subsequent feature vectors 440. The incremental information array records neighbor vertices added or deleted in subsequent snapshots relative to the previous snapshot, used to support incremental aggregation calculations across snapshots.

[0053] The COSH architecture of this invention can perceive and capture changes in the vertex topology between consecutive snapshots in a dynamic heterogeneous graph 500 in real time, including the addition and disappearance of vertices and changes in connecting edges. By using the cross-snapshot vertex filtering unit 230 to analyze graph structure data in parallel with low overhead, it proactively guides the execution of two subsequent key stages: first, it guides the priority-aware feature loading unit to optimize its working strategy, prioritizing the prefetching or caching of feature vectors 440 identified as high-priority; second, it accelerates the computation process of the collaborative feature aggregation module. Because the cross-snapshot vertex filtering unit 230 identifies changed regions in advance, the aggregation module can concentrate computational resources on processing these regions, avoiding redundant computation on static or minimally changing parts, thereby significantly improving overall processing efficiency.

[0054] like Figure 2 As shown, the cross-snapshot vertex filtering unit 230 in the hierarchical data loading module 200 includes a snapshot buffer 231, a vertex comparator 232, and a filtering result buffer 234.

[0055] Preferably, the snapshot buffer 231 is used to store the snapshot 410 loaded from the graph structure loading unit 210. The vertex comparator 232 reads the vertices in the neighboring snapshots in parallel, wherein the vertex comparators 232 at odd and even positions traverse the vertices in the neighboring snapshots in an alternating manner, filtering out the incremental information array of additions and subtractions between adjacent snapshots and the neighbor set information of all vertices that have appeared, forming a filtering queue 233. The filtering queue 233 is fed back to the vertex comparator 232 to form a cyclic filtering. The filtering result buffer 234 is used to temporarily store the filtering result 420.

[0056] like Figure 2 As shown, the snapshot buffer 231 receives and stores the loaded snapshot 410 from the graph structure loading unit 210. The input ports of several parallel vertex comparators 232 are connected to the snapshot buffer 231 to read vertices from neighboring snapshots. The output ports of each vertex comparator 232 output filtering results 420 to the filtering result buffer 234 so that the vertex reading unit 321 can read the target vertex 430.

[0057] This invention employs a cross-snapshot incremental computation approach. By analyzing the structural change information of the dynamic heterogeneous graph 500, the algorithm incrementally updates only the target vertex 430 affected by the change, based on the calculation results of its previous snapshot. Specifically, in the dynamic heterogeneous graph 500, a target vertex 430 typically connects to multiple types of neighboring vertices. This strategy fully leverages this characteristic: for each type of neighboring vertex associated with the target vertex 430, the accelerator independently determines whether that type of neighbor and its associated edges have changed within the current time window. If the structure of a certain type of neighboring vertex and its edges has not changed, the intermediate calculation results or aggregated information related to that type of neighbor do not need to be recalculated; instead, the corresponding results from the previous snapshot are directly and efficiently reused. This fine-grained, neighbor-type-based cross-snapshot incremental processing mechanism significantly reduces the overhead of repeated calculations on unchanged subgraph structures, while also greatly reducing the frequent data access demands to off-chip storage caused by full recalculation.

[0058] After receiving complete neighbor set information and incremental information array from the cross-snapshot vertex filtering unit 230, the feature loading unit 220 divides the target vertices 430 within a preset neighborhood into the same loading batch based on the spatial proximity principle of graph topology. The feature loading unit 220 constructs a vertex-frequency statistics mapping table based on the neighbor set information of the target vertices 430 in the same batch, then sorts them from high to low based on the reference count in the vertex-frequency statistics mapping table, and loads the feature vectors 440 of the vertices sequentially according to their priority from high to low. The feature loading unit 220 stores the vertex feature vectors 440 in the on-chip cache (BRAM) for use by the subsequent collaborative feature aggregation module 300.

[0059] The priority-aware feature loading unit 220 effectively alleviates the bottleneck problem of feature access in the dynamic heterogeneous graph 500. During neighbor aggregation in the dynamic heterogeneous graph 500, access to the feature vectors 440 of neighbor vertices often exhibits fragmentation (discontinuous access addresses) and potential redundancy (the same feature vector 440 may be repeatedly requested by multiple target vertices 430). The core innovation of this feature loading unit 220 lies in its ability to intelligently integrate these scattered access requests, reducing memory access overhead. More importantly, the feature loading unit 220 can prioritize the loading requests of vertex feature vectors 440 based on computational dependencies. By evaluating the importance of each vertex's feature vector 440 to the current computational task, the feature loading unit 220 strictly loads the vertex feature vectors 440 in descending order of priority, ensuring that critical data is prioritized, thereby improving overall data loading efficiency and system performance.

[0060] According to a preferred embodiment, such as Figure 1As shown, the collaborative feature aggregation module 300 includes a local aggregation unit 310 and a feature collection unit 330. The local aggregation unit 310 and the feature collection unit 330 are bidirectionally connected.

[0061] The local aggregation unit 310 includes several systolic arrays 320. Neighbor feature vectors and the feature vector 440 of the target vertex 430 are sequentially input into the systolic arrays 320. Under clock control, the systolic arrays 320 can perform parallel multiplication operations between sparse matrices and vectors, calculating the local aggregation result after aggregating the target vertex 430 with all its neighbors. The local aggregation unit 310 sends an "incremental computation ready" signal and the local aggregation result of all neighbor vertices to the feature collection unit 330.

[0062] Upon receiving the "incremental computation ready" signal and the local aggregation results of all neighboring vertices, the feature collection unit 330 calculates the feature vector 440 corresponding to the elements of the changed vertex position and standardizes the feature vector 440. The feature collection unit 330 applies incremental changes to the basic aggregation results based on the vertex change information in the incremental information array.

[0063] like Figure 3 As shown, the local aggregation unit 310 in the collaborative feature aggregation module 300 includes a vertex reading unit 321, a demultiplexer 322, a feature grasping unit 323, and an aggregation unit 324. The vertex reading unit 321 is connected to the input port of the demultiplexer 322. The first output port of the demultiplexer 322 is connected to the first input port of the feature grasping unit 323, and the second output port of the demultiplexer 322 is connected to the snapshot update reader 331 in the feature collection unit 330. The second input port of the feature grasping unit 323 is externally connected and receives the feature vector 440. The output port of the feature grasping unit 323 is connected to the input port of the aggregation unit 324. The output port of the aggregation unit 324 is connected to the signal receiver 332 in the feature collection unit 330 and also to the on-chip local aggregation buffer 340.

[0064] Vertex reading unit 321 receives target vertex 430 from task scheduler 240. Task scheduler 240 is responsible for managing the queue of target vertices 430 to be processed, and distributing the target vertices 430 to vertex reading unit 321 in sequence according to a certain scheduling strategy to start the subsequent neighbor reading and feature aggregation process.

[0065] Vertex reading unit 321 reads the complete neighbor set information and incremental information array related to the target vertex 430 from the cross-snapshot vertex filtering unit 230, based on the target vertex 430 to be processed. Vertex reading unit 321 then sends the complete neighbor set information and incremental information array to demultiplexer 322.

[0066] After receiving the neighbor set information and the incremental information array, the demultiplexer 322 sends the neighbor set information to the feature capture unit 323 and the incremental information array to the aggregation unit 324.

[0067] After receiving neighbor set information from demultiplexer 322, feature extraction unit 323 parses vertex IDs and reads feature vectors 440 corresponding to neighbor vertices in batches from on-chip cache.

[0068] After receiving the incremental information array from the demultiplexer 322, the aggregation unit 324 performs matrix-vector multiplication of the feature vector 440 with the target vertex 430 in sequence. After completing the calculation, the vertex ID and the local aggregation result are sent to the local aggregation buffer 340.

[0069] like Figure 3 As shown, the feature collection unit 330 in the collaborative feature aggregation module 300 includes a snapshot update reader 331, a signal receiver 332, and a collection unit 333. The first output port of the snapshot update reader 331 is connected to the input port of the signal receiver 332. The second output port of the snapshot update reader 331 is connected to the collection unit 333. The output port of the signal receiver 332 is connected to the input port of the collection unit 333. A collection register 334 is also provided within the collection unit 333. The collection unit 333 is also connected to a local aggregation buffer 340 to read vertex IDs and local aggregation results. The output port of the collection unit 333 is used to send the output result 450 to the off-chip high-bandwidth storage 100. Figure 3 As shown, the output result 450 is the vertex feature aggregation result 130.

[0070] The snapshot update reader 331 receives the incremental information array from the demultiplexer 322 and updates the incremental information array in real time. The signal receiver 332 receives the incremental calculation ready signal from the local aggregation unit 310. The collection unit selectively reads the local aggregation results corresponding to the changed vertices based on the incremental information array; calculates the feature vector 440 of the elements corresponding to the changed vertex positions, calculates the standardization factor and standardizes the feature vector 440, and applies incremental changes to the basic aggregation results according to the vertex change information in the incremental information array.

[0071] The incremental computation process is decoupled into two parts by the collaborative feature aggregation module 300: matrix-vector multiplication and element-wise computation. For these two distinctly different computational modes—the former being computationally intensive and featuring a regular data access pattern, and the latter relatively simpler but requiring flexible handling of inter-element operations—highly optimized dedicated hardware computing architectures, a local aggregation unit 310 and a feature collection unit 330, are designed to maximize hardware efficiency and processing throughput. The local aggregation unit 310 mainly consists of a systolic array 320 to perform the multiplication of feature vectors 440 between the target vertex 430 and its neighbors. The feature collection unit 330 mainly organizes the computation units into a single-instruction multiple-data (SIMD) model to normalize the computation results for the same vertex.

[0072] This invention utilizes a collaborative feature aggregation module 300 to achieve fine-grained scheduling and storage optimization for incremental computation. By tracking and recording the computational state of each vertex during the feature aggregation process in real time, the collaborative feature aggregation module 300 can proactively coordinate the workflows of the feature collection unit 330 and the local aggregation unit 310. Specifically, once the local aggregation unit 310 completes the local aggregation of some neighbor types for a target vertex 430, the collaborative feature aggregation module 300 immediately detects this state change. It then triggers the feature collection unit 330, allowing it to begin incremental computation promptly without waiting for all neighbor types to be fully computed, and releases storage space for some local aggregation results. This reduces the on-chip storage occupied by local aggregation results and improves overall computational efficiency.

[0073] This invention provides a dynamic heterogeneous graph neural network inference method based on FPGA, the method steps of which are as follows: Figure 3 As shown.

[0074] S100: Graph structure loading unit 210 acquires information about neighboring vertices of the target vertex 430 in each snapshot. The information about neighboring vertices includes graph structure data 110 and vertex initial feature vectors 120.

[0075] Preferably, the graph structure loading unit 210 uses the FPGA's multi-channel AXI to read the graph structure data 110 and the vertex initial feature vector 120 in parallel from the high-bandwidth storage 100, maps each snapshot neighbor data to different channels, and implements this in a pipelined manner, thereby efficiently accessing the neighbors of a specified node in parallel.

[0076] Step S100 specifically includes the following sub-steps, such as Figure 6 As shown. In the following text, the device will be referred to as an accelerator.

[0077] S110: Accelerator initialization.

[0078] The device is initialized by sending graph data information from high-bandwidth storage 100 to memory-mapped registers accessible to the accelerator through the accelerator's configuration interface.

[0079] Preferably, the graph data information includes: (a) the size and starting address of the offset array and edge array included in the graph data of each timestamp in CSR format; and (b) the starting address of the initialization feature vector corresponding to the vertex.

[0080] The offset array records the starting position of each vertex in the edge array. The edge array stores all neighboring vertices pointed to by the edges in order. The initialized feature vector is the original feature vector of each vertex at the initial time.

[0081] Since the aforementioned data is stored in high-bandwidth storage 100, its starting position and size are required to read it into the accelerator for processing. Because each feature vector 440 has the same dimension (e.g., 128 dimensions), there is no need to obtain its size information separately. However, the structure of the dynamic heterogeneous graph 500 differs in different snapshots, therefore, it is necessary to obtain additional information about the size of the offset array and the edge array.

[0082] S120: The graph structure loading unit 210 maintains a queue to store vertices that will be processed later. Meanwhile, the vertex reading unit 321 maintains a vertex state array using a bitmap. Each vertex uses 1 bit to record whether it has been visited (0 represents not visited, 1 represents visited). Then, the vertex reading unit 321 randomly selects a vertex to add to the queue.

[0083] S130: The graph structure loading unit 210 regards the first dequeued vertex in the queue as the target vertex, and sets the corresponding position of the target vertex in the vertex state array to 1. It also obtains the edge start / end offset of the target vertex under all snapshots from the offset array.

[0084] S140: The graph structure loading unit 210 reads the corresponding neighboring vertices in the edge array according to the offset value, that is, the neighboring vertices under all snapshots of the target vertex, and transmits the neighboring vertices to the cross-snapshot vertex filtering unit 230.

[0085] S150: The cross-snapshot vertex filtering unit 230 obtains the states related to neighboring vertices from the vertex state array. If the corresponding state record is visited, the neighboring vertex is not added to the queue, output, and the process ends. If it is not visited, the neighboring vertex is added to the queue, and step S120 continues until all neighboring vertices have been visited.

[0086] S200: The cross-snapshot vertex filtering unit 230 analyzes the topological changes between snapshots 410 at adjacent timestamps in the dynamic heterogeneous graph 500 in real time, identifies the addition, deletion and connection changes of the target vertex 430, and constructs complete neighbor set information and an incremental information array describing the changes of the neighbor set between adjacent snapshots.

[0087] Step S200 includes the following sub-steps, such as Figure 6 As shown.

[0088] S210: Vertex comparator 232 compares neighbor snapshots.

[0089] Snapshots of the target vertex at each timestamp are sent by the graph structure loading unit 210 to the cross-snapshot vertex filtering unit 230. Preferably, the received neighbor vertices are organized in the form of (target_vertex_id, [neighbor_snapshot_1,neighbor_snapshot_2,…, neighbor_snapshot_k]).

[0090] Preferably, the neighbor snapshots are transmitted in parallel to the eight vertex comparators 232 in ascending order of timestamp. For example... Figure 2 As shown, the vertex comparator 232 is arranged linearly in the cross-snapshot vertex filtering unit 230. Its reading function is controlled by two buses, which can read the vertices in the neighbor_snapshot in parallel.

[0091] Preferably, the two buses control the vertex comparators 232 at odd and even positions respectively, and operate alternately. For example, The vertex comparator 232 at points 1, 3, 5, and 7 runs continuously. Vertex comparator 232 runs continuously at points 2, 4, 6, and 8.

[0092] S220: The adjacent vertex comparator 232 uses the FPGA on-chip memory (BRAM) to implement a filtering queue 233, which records the number of times a vertex appears.

[0093] Each item in the filtering queue 233 is stored in the format (vertex_id, flag), where vertex_id records vertex information and flag records the cumulative value of the corresponding vertex, which is used for subsequent analysis of vertex occurrence.

[0094] Preferably, the vertex comparator 232 at odd-numbered positions and the vertex comparator 232 at even-numbered positions traverse the vertices in the neighbor_snapshot in an alternating manner. Since adjacent vertex comparators 232 access the same queue, alternating operation avoids conflicts in read and write operations.

[0095] Preferably, when the vertex comparator 232 at odd and even positions is running, it increments the flag value of the corresponding vertex by 1 or 2 respectively to record the snapshot order in which it appears, thereby distinguishing whether the vertex appears in the previous or the next snapshot.

[0096] S230: Filter across snapshot vertex filter unit 230 to obtain filter result 420.

[0097] By traversing the information of the vertex flags recorded in the filtering queue 233, the incremental information array of additions and subtractions between adjacent snapshots is filtered out. and the set of neighboring vertices that have appeared ,like Figure 2 As shown. Incremental information array include The filtered result 420 is an incremental information array. and neighboring vertex set .

[0098] Incremental information array Each element in the array consists of two parts, which record the vertices that have been added and deleted.

[0099] The specific filtering rules for cross-snapshot vertex filtering unit 230 are as follows: (1) Corresponding to the vertex A value of 3 indicates that the image appeared in both snapshots and remained unchanged. (2) Corresponding to the vertex A value of 2 indicates that the vertex did not exist in the previous snapshot but appeared in the next snapshot, meaning it is a newly added vertex. (3) Corresponding to the vertex A value of 1 indicates that the vertex existed in the previous snapshot but not in the next snapshot, and is therefore a deleted vertex. The cross-snapshot vertex filtering unit 230 transmits the filtering result 420 to the filtering result buffer 234 of the connection feature loading unit 220.

[0100] Repeat the above steps across snapshot vertex filtering unit 230 until all target vertices 430 in the graph have been visited.

[0101] like Figure 4 As shown, Figure 4(a) is a dynamic heterogeneous graph 500, where circles, squares, and hexagons represent vertices of different types. The timestamps on the vertex edges represent the times when the edges were added or deleted. Figure 4 (b) illustrates the aggregation process of the dynamic heterogeneous graph neural network, including intra-relation aggregation result 510, inter-relation aggregation result 520, and temporal aggregation result 530. In the intra-relation aggregation stage, vertices of the same type in each snapshot are aggregated to obtain a feature vector 440 representing that type of vertex. For example, vertices in snapshot 410... and vertex ,vertex and vertex By aggregating the data separately, we obtain the first feature vector representing the vertices of the corresponding categories. Second eigenvector The relationship aggregation stage involves aggregating the feature vectors 440 representing vertices of different classes. For example, the first feature vector in snapshot 410... Second eigenvector Aggregation is performed to obtain the features of the target vertex in the current snapshot. Finally, the features of vertices from different snapshots are aggregated to obtain the temporal aggregation result. For example, the target vertex in snapshot 410. Features in each snapshot Perform aggregation to obtain the final time-series aggregation result. Figure 4 The arrow in (b) represents the aggregation calculation of the eigenvector 440 of the vertex.

[0102] like Figure 4 As shown, assuming t=1 and T=3, the target vertex... After processing, the set of neighboring vertices in all snapshots is obtained as follows: And the incremental information array representing the vertex changes between adjacent snapshots is... .

[0103] Steps S210 to S230 analyze the changes in the neighbors of vertices between snapshots of the dynamic heterogeneous graph through high parallelism, providing a basis for the subsequent loading and calculation of feature vector 440.

[0104] S300: Priority-aware feature loading unit 220 receives the output result 450 of cross-snapshot vertex filtering unit 230 and completes it in the form of a hardware-implemented three-stage pipeline.

[0105] This step S300 specifically includes the following sub-steps, such as Figure 6 As shown: S310: Feature loading unit 220 reads the set of neighboring vertices of multiple target vertices 430 from the filtering result buffer 234. Based on the spatial proximity principle of graph topology, target vertices 430 within a relatively short 2- or 3-hop range (reachable via 2 or 3 edges) are considered as a batch. This means that target vertices 430 within the same batch may share more neighboring vertices. This is achieved by traversing the set of neighboring vertices within the same batch. This allows them to be merged into an on-chip stored array, common_vertex, while also recording the number of times each vertex appears.

[0106] S320: Feature loading unit 220 traverses the above common_vertex array, constructs a vertex-frequency statistics mapping table, accurately records the number of times each neighbor vertex is referenced by the target vertex 430 in the current batch, and then performs unstable fast sorting according to the vertex occurrence frequency from high to low.

[0107] S330: The feature loading unit 220 reads the corresponding feature vector 440 from the high-bandwidth storage 100 in sequence according to the arrangement order of step S320, using the vertex ID as the index, and then outputs it to the on-chip cache (BRAM) to end.

[0108] like Figure 4 As shown in (a), if the target vertex 430 currently being processed only includes the target vertex and vertex ,based on The corresponding vertex-frequency statistics mapping table is as follows: Then, with the vertex and vertex The corresponding feature vector 440 has the highest loading priority.

[0109] Loading other vertices may cause problems with subsequent target vertices. The computing unit is in an idle state. At this time, computing resources have already been allocated to the current target vertex. However, the lack of loaded, computable feature vectors 440 leads to wasted computational resources, thus prolonging inference time. The priority-aware feature loading unit 220 ensures that the prioritized loaded feature vectors 440 can integrate the fragmented redundant accesses of different target vertices 430 to off-chip storage, and enables the loaded feature vectors 440 to be used by more computing units, improving the utilization of high-bandwidth storage 100 while reducing the idle waiting time of computing units.

[0110] S400: The collaborative feature aggregation module 300 uses the loaded feature vector 440 and graph structure data 110 to perform aggregation during the inference phase.

[0111] This step is completed collaboratively by the local aggregation unit 310 and the feature collection unit 330. These two units organize the FPGA's computing resources into a systolic array 320 and a single instruction multiple data (SIMD) array, respectively. The former is used for computationally intensive matrix multiplication; the latter is used for calculating element-wise operations (such as summation, averaging, maximization, etc.) and nonlinear activation functions (such as ReLU, Sigmoid, etc.).

[0112] In this embodiment, the dynamic heterogeneous graph neural network is taken as an example, HTGNN.

[0113] Step S400 specifically includes the following sub-steps (corresponding to S410~S470) Figure 3 (①~⑦ in the text) S410 (Step ①): Vertex reading unit 321 reads the complete set of neighbor information related to the current target vertex 430 from cross-snapshot vertex filtering unit 230 based on the target vertex 430 to be processed. and an array of incremental information describing the changes in the neighbor set between adjacent snapshots. .

[0114] S420 (Step ②): The high-speed demultiplexer 322 reads the complete neighbor set information. and incremental information array Perform route distribution.

[0115] Neighbor aggregation information The incremental information array is transmitted to the feature-grabbing unit 323. It is directed to the snapshot update reader 331.

[0116] S430 (Step ③): When receiving neighbor set information Then, the feature extraction unit 323 parses the vertex ID information and accesses the on-chip cache BRAM accordingly to read the feature vectors 440 corresponding to these neighboring vertices in batches.

[0117] In aggregation unit 324, a state array partial_agg is initialized for precise tracking of neighbor set information. The feature collection unit 330 checks whether the features of each neighboring vertex in the first snapshot have been aggregated with the target vertex 430. Simultaneously, in the feature collection unit 330, the snapshot update reader 331 updates the feature collection unit based on the incremental information array. Another state array is initialized to track whether neighboring vertices that have changed (added or deleted) between adjacent snapshots have completed their incremental aggregation calculations. Snapshot update reader 331 updates the incremental information array in real time; the updated incremental information array is then sent to signal receiver 332.

[0118] S440 (Step ④): The feature vector 440 loaded in the aggregation unit 324 is sequentially calculated with the target vertex to obtain the local aggregation result, and stored in the local aggregation buffer 340 (on-chip URAM).

[0119] The neighbors of the target vertex in all snapshots are recorded as a set of neighbor vertices. During the computation, nearly half of the same target vertex 430 can have their computation results reused and stored in local register 325, thus reducing redundant computation. Whenever the neighboring vertex set... When a vertex in the first snapshot completes its calculation with the target vertex, the aggregation unit 324 marks the vertex as having completed its calculation in its array and transmits the vertex's ID and local aggregation result to the local aggregation buffer 340 implemented by URAM. Simultaneously, the array maintained by the feature collection unit 330 is updated according to the vertex's aggregation completion status. When all neighboring vertices in the first snapshot have completed their calculations, the aggregation unit 324 sends a signal to the collection unit 333 via the signal receiver 332 indicating that subsequent incremental calculations can begin. For example, with the target vertex... The corresponding feature vector 440 is .

[0120] Local aggregation results The calculation formula is as follows: .

[0121] In the above formula, Through the target vertex With neighboring vertices The calculated weight vector. Some of the results that can be reused are as follows: Stored in collection register 334. Whenever the neighboring vertex set... One of the neighboring vertices corresponding feature vector With the target vertex The calculation is completed, and the local aggregation result is obtained. The partial_agg array maintained by aggregation unit 324 will store the neighboring vertices. The corresponding bit is set from "0" to "1", and the vertex ID and local aggregation result are also set. The signal is transmitted to the collection unit 333 via the signal receiver 332. When all the neighboring vertices in the first snapshot have completed their calculations, i.e., when the partial_agg array is all "1", the aggregation unit 324 sends a signal to the collection unit 333 via the signal receiver 332 that subsequent incremental calculations can begin.

[0122] Figure 5 The computation process of the collaborative feature aggregation unit 324 is demonstrated. Figure 5In this diagram, circles, squares, and hexagons represent vertices of different types. Squares represent type 1 vertices, and hexagons represent type 2 vertices. The lines connecting the circles represent calculations performed between corresponding vertices, and the values ​​above the lines represent the calculated weights. First characteristic quantity Second characteristic represent The aggregation result within the time-relationship is 510.

[0123] by Figure 5 As shown in the example, in At any given time, the vertex is represented by a square. And vertices represented by hexagons After the feature vector 440 is loaded, it can be immediately compared with the target vertex. Local aggregation calculations were performed to obtain and Similarly, in At any given time, the vertex is represented by a square. And vertices represented by hexagons After loading feature vector 440, it matches the target vertex. Local aggregation calculations were performed to obtain and No need to wait until All target vertices at any given time This method allows subsequent calculations to proceed only after all features of similar neighboring vertices have been loaded, reducing idle time for computational resources. Existing methods require waiting for all similar neighboring nodes to be loaded before initiating further calculations because they treat the aggregation of similar vertices as a single, continuous operation; the first feature vector cannot be computed until all similar neighbors are loaded. Second eigenvector The denominator. In this case, even after loading the feature vectors 440 of some vertices, computing resources cannot be used for computation, resulting in wasted resources and low inference efficiency.

[0124] S450 (Step ⑤): After receiving the "incremental calculation ready" signal from the aggregation unit 324, the signal receiver 332 inside the collection unit 333 immediately starts the incremental aggregation calculation process and begins to collect and utilize the previously stored local aggregation results for efficient incremental updates.

[0125] S460 (Step 6): Collection unit 333 reads in the local aggregation results of the vertices in the first snapshot.

[0126] Based on the incremental information array provided by snapshot update reader 331 The collection unit 333 selectively reads the local aggregation results corresponding to these changed vertices from the local aggregation buffer 340, and then passes them through the SIMD 8 module in the collection unit 333 (see...). Figure 3 ) Perform calculations on the corresponding elements to obtain unstandardized feature vectors. ( Represents the type of neighboring nodes. (Represents a timestamp). Figure 4 The target vertex in the first snapshot in (b) For example, suppose a square represents a vertex of type 1, and a hexagon represents a vertex of type 2. Target vertex The neighboring vertices of the same type are vertices and vertex and vertices and vertex .

[0127] Preferably, the SIMD 8 module is used in the collection unit 333 for parallel computation of unnormalized feature vectors. The core processing unit adopts a single instruction stream multiple data stream (SIMD) parallel architecture, which supports performing the same operation on 8 sets of data at the same time to improve the computational efficiency of local aggregation results.

[0128] First, the local aggregation results are preliminarily calculated as follows: ; .

[0129] Then, the weight vectors of the corresponding vertices are summed to obtain the normalization factor. ( Represents the type of neighboring nodes. (representing timestamps), processed by the SIMD 8 module for unstandardized feature vectors. Standardize it. Figure 4 The target vertex in the first snapshot in (b) For example, its neighboring vertices of the same type are vertices. and vertex and vertices and vertex .like Figure 5 As shown, the standardization factor is and Then, the first eigenvector is calculated. Second eigenvector .

[0130] like Figure 5 As shown, in time, . .

[0131] Finally, the SIMD 8 module uses the incremental information array... Mid-vertex change information is efficiently applied to the underlying aggregation results using incremental changes: For newly added neighbors, their local aggregation results are added to the base result; For deleted neighbors, subtract them from the previous local aggregation results.

[0132] like Figure 4 As shown, target vertex exist New vertices have been added under timestamps Then, the results can be aggregated locally. and standardization factor Based on the local aggregation results and standardization factor Finally, the standardized feature vector is obtained as follows: .

[0133] The method of this invention maximizes the use of reusable local aggregation results between adjacent snapshots, performing additional computation only on added and deleted vertices, whereas conventional methods require full computation of each snapshot individually. Therefore, step S460 reduces redundant computation by nearly 95% and also reduces off-chip access to the same vertices between adjacent snapshots, improving overall inference speed and reducing energy consumption.

[0134] S470 (Step ⑦): After aggregation is complete, the latest normalized feature vector 440 of the target vertex 430 under the specified snapshot is efficiently written back to the specified location of the high-bandwidth storage 100 using burst write operations supported by the AXI protocol, for use by subsequent layers or tasks. This burst write method can fully unleash the high bandwidth potential of the high-bandwidth storage 100 and improve data writing efficiency.

[0135] It should be noted that the specific embodiments described above are exemplary. Those skilled in the art can devise various solutions inspired by the disclosure of this invention, and these solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents. This specification contains multiple inventive concepts; phrases such as "preferredly" or "according to a preferred embodiment" indicate that the corresponding paragraph discloses an independent concept. The applicant reserves the right to file divisional applications based on each inventive concept.

Claims

1. A dynamic heterogeneous graph neural network inference device based on FPGA, characterized in that, The device includes: The hierarchical data loading module (200) senses and captures the changes in vertex topology between consecutive snapshots (410) in the dynamic heterogeneous graph (500) in real time; at the same time, it merges the neighbors of different target vertices (430) under each time stamp into neighbor set information, and loads the corresponding feature vectors (440) in order of frequency of occurrence from high to low. The collaborative feature aggregation module (300) receives the neighbor set information and reads the corresponding feature vector (440). It decouples the incremental calculation process into two parts: matrix-vector multiplication and element calculation at corresponding positions. Based on the local aggregation unit (310) composed of pulsating array (320), it completes the operation process of multiplying the feature vectors (440) between the target vertex (430) and its neighbors. Based on the feature collection unit (330), the calculation unit is organized into a single instruction stream and multiple data stream to complete the normalization of the calculation results for the same vertex.

2. The apparatus according to claim 1, characterized in that, The hierarchical data loading module (200) includes: The graph structure loading unit (210) analyzes the graph structure under different timestamps and discovers the changes in vertices between snapshots (410) under adjacent timestamps; it merges the neighbor information of the target vertex (430) under each timestamp into the neighbor set information. The cross-snapshot vertex filtering unit (230) analyzes the topological changes between snapshots (410) at adjacent timestamps in the dynamic heterogeneous graph (500) in real time, identifies the addition, deletion and connection changes of the target vertex (430), and constructs complete neighbor set information and an incremental information array describing the changes of the neighbor set between adjacent snapshots (410). The feature loading unit (220) divides the target vertices (430) within a preset neighborhood into the same loading batch based on the spatial proximity principle of graph topology; constructs a vertex-frequency statistical mapping table based on the neighbor set information of the target vertices (430) in the same batch; sorts the reference counts in the vertex-frequency statistical mapping table from high to low; and loads the feature vectors (440) of the vertices in order of priority from high to low.

3. The apparatus according to claim 1 or 2, characterized in that, The collaborative feature aggregation module (300) includes: The local aggregation unit (310) and the pulsating array (320) perform multiplication operations between sparse matrices and vectors in parallel to calculate the local aggregation result after the target vertex (430) and all its neighbors are aggregated; The feature collection unit (330) calculates the feature vector (440) of the element corresponding to the changed vertex position and standardizes the feature vector (440). It then applies incremental changes to the basic aggregation result based on the vertex change information in the incremental information array.

4. The apparatus according to any one of claims 1 to 3, characterized in that, The local aggregation unit (310) in the collaborative feature aggregation module (300) includes: The vertex reading unit (321) reads the complete neighbor set information and incremental information array related to the vertex from the cross-snapshot vertex filtering unit (230) according to the target vertex to be processed (430); The demultiplexer (322) sends the neighbor set information to the feature capture unit (323) and the incremental information array to the aggregation unit (324). The feature extraction unit (323) receives neighbor set information, parses vertex IDs, and reads feature vectors (440) corresponding to neighbor vertices in batches from the on-chip cache. The aggregation unit (324) performs matrix-vector multiplication of the feature vector (440) with the target vertex (430) in sequence. After completing the calculation, it sends the vertex ID and the local aggregation result to the local aggregation buffer (340).

5. The apparatus according to any one of claims 1 to 4, characterized in that, The feature collection unit (330) in the collaborative feature aggregation module (300) includes: The snapshot update reader (331) receives the incremental information array from the demultiplexer (322) and performs real-time updates to the incremental information array; A signal receiver (332) is used to receive an incremental calculation ready signal from the local aggregation unit (310); The collection unit (333) selectively reads the local aggregation results corresponding to the changed vertices based on the incremental information array; calculates the feature vector (440) of the element corresponding to the position of the changed vertex; calculates the standardization factor and standardizes the feature vector (440); and applies incremental changes to the basic aggregation results based on the vertex change information in the incremental information array.

6. The apparatus according to any one of claims 1 to 5, characterized in that, The cross-snapshot vertex filtering unit (230) in the hierarchical data loading module (200) includes: Snapshot buffer (231) stores snapshots (410) received from graph structure loading unit (210). Vertex Comparator (232) reads vertices in neighboring snapshots in parallel. The vertex comparators (232) at odd and even positions traverse the vertices in neighboring snapshots in an alternating manner, filtering out the incremental information arrays of additions and subtractions between adjacent snapshots (410) and the neighbor set information of all vertices that have appeared, and forming a filtering queue (233). The filter result buffer (234) temporarily stores the filter results (420).

7. The apparatus according to any one of claims 1 to 6, characterized in that, The device also includes high-bandwidth storage (100) for storing aggregated feature vectors (440) and infrequently accessed graph data, supporting batch write-back of aggregated results for multiple vertices.

8. A dynamic heterogeneous graph neural network inference method based on FPGA, characterized in that, The method includes: Real-time sensing and capture of the changes in vertex topology between consecutive snapshots (410) in the dynamic heterogeneous graph (500); at the same time, the neighbors of different target vertices (430) under each time stamp are merged into neighbor set information, and the corresponding feature vectors (440) are loaded in order of frequency of occurrence from high to low. The neighbor set information is received and the corresponding feature vector (440) is read. The incremental calculation process is decoupled into two parts: matrix-vector multiplication and element calculation at the corresponding position. The operation process of multiplying the feature vectors (440) between the target vertex (430) and its neighbors is completed based on the local aggregation unit (310) composed of the pulsating array (320). Based on the feature collection unit (330), the calculation unit is organized into a single instruction stream and multiple data stream to complete the normalization of the calculation result of the same vertex.

9. The method according to claim 8, characterized in that, The steps to load the corresponding feature vectors (440) in descending order of frequency of occurrence include: The graph structure under different loading timestamps is analyzed to discover the changes in vertices between snapshots (410) under adjacent timestamps; the neighbor information of the target vertex (430) under each timestamp is merged into the neighbor set information; Real-time analysis of the topological changes between snapshots (410) at adjacent timestamps in the dynamic heterogeneous graph (500) is performed to identify the addition, deletion and connection changes of the target vertex (430), and to construct complete neighbor set information and an incremental information array describing the changes of the neighbor set between adjacent snapshots (410). Based on the principle of spatial proximity in graph topology, target vertices (430) within a preset neighborhood are divided into the same loading batch; based on the neighbor set information of the target vertices (430) in the same batch, a vertex-frequency statistical mapping table is constructed, and the reference counts in the vertex-frequency statistical mapping table are sorted from high to low, and the feature vectors (440) of the vertices are loaded in order of priority from high to low.

10. The method according to claim 8 or 9, characterized in that, The steps for normalizing the calculation results for the same vertex include: Parallel multiplication operations between sparse matrices and vectors are performed to calculate the local aggregation result of the target vertex (430) and all its neighbors; Calculate the feature vector (440) of the element corresponding to the changed vertex position and standardize the feature vector (440). Apply incremental change to the basic aggregation result based on the vertex change information in the incremental information array.

Images