An FPGA-based dynamic graph random walk acceleration system and method

Abstract

A dynamic graph random walk acceleration system and method based on FPGA is disclosed. The system includes a main processor and an FPGA accelerator. The main processor is used for information preprocessing, compressing graph structure fragments and walker grouping information into a shared-packet slice. The FPGA accelerator is used for scheduling and decompression. In response to the control of the main processor, the FPGA accelerator reads the shared-packet slice, performs parsing verification and decompression operations, and provides the decompressed data to the main processor. Upon receiving the decompressed data, the main processor processes walkers with the same vertex to perform random walks, updates the walker states, writes the updated walker states to the walker buffer, and notifies the FPGA accelerator to initiate the compression operation. In response to the notification, the FPGA accelerator reads the walker states in the walker buffer, compresses them, writes the compression result back as a walk state slice, and updates the corresponding directory information.

Description

Technical Field

[0001] This invention relates to the field of graph data processing technology, and in particular to an FPGA-based dynamic graph random walk acceleration system and method. Background Technology

[0002] With the continuous expansion of applications in social networks, recommendation systems, risk control, knowledge graphs, and cybersecurity, graph data is exhibiting characteristics of high scale, strong sparsity, and high dynamism. Random walks and their variants (such as representation learning / retrieval methods based on random sampling) are widely used in computational scenarios such as node representation, similarity search, and structure sampling due to their strong versatility for models and tasks. Such computations share three common characteristics: first, small access units and frequent cross-address jumps, exhibiting strong irregular memory access; second, lightweight computation but intensive data movement, often constrained by memory bandwidth and access latency; and third, in dynamic graph scenarios, the addition or deletion of edges / vertices causes already loaded or computed local structures to quickly become invalid, necessitating efficient incremental updates and consistency maintenance.

[0003] Existing technologies primarily employ three approaches to implement random walks: the first is a pure software framework (for CPUs or GPUs), which alleviates memory access bottlenecks through prefetching, batch processing, and parallelization; the second is a general-purpose accelerator + DMA model, where graph fragments are stored in main memory, moved to host memory or on-chip by the accelerator, and then decoded and computed by the host or device; the third is a graph database / offline system, which improves locality by rebuilding indexes or repartitioning. However, these technologies are effective for static graphs or a small number of incremental updates, but they still have significant shortcomings when facing a combination of high-concurrency random walks and high-frequency graph updates. High bandwidth consumption and increased decompression overhead: To reduce the pressure on main memory, graph structures are often stored in compressed form (adjacency list compression, incremental / bit plane, etc.). However, the host or device needs to decompress the graph before it can be traversed, resulting in multiple transfers of compression → decompression → write-back to memory / cache. This repeatedly consumes bandwidth and cache, accumulating significant decompression latency and write-back overhead.

[0004] The data plane and control plane are loosely coupled: the conventional DMA process relies on writing to main memory → CPU reading to complete the data handover, lacking direct control over the host cache level (such as cache injection / consistent read and write); the control plane (MMIO doorbell / interrupt) and the data plane (DMA) lack throughput-oriented window / backpressure coordination, making it difficult to form a stable overlapping pipeline between the three stages of decompression, computation and write-back.

[0005] The existing solution stores the "compressed fragments of graph structure" and the "wander state / path" separately. The host side needs to perform secondary aggregation, which causes additional round trips and format conversions. It cannot guarantee the locality and synchronous visibility of "structure + state" within the same graph block.

[0006] dynamic Figure 1 Coarse-grained consistency handling: After a graph update (adding or deleting edges / vertices) is completed, repartitioning or batch reloading is usually used. For walkers that are already in progress or have completed some steps, there is a lack of precise backtracking of the earliest inconsistent step and control over the minimum repair range, which can easily lead to large-scale repeated calculations or inconsistent results.

[0007] Storage organization lacks self-description and atomicity guarantees: Most implementations rely on external metadata to locate compressed fragments, lacking slice-level version (VER) / integrity (CRC) headers and atomic header-swapping strategies that write the load first and then swap the header. This makes it easy for unfinished or old version data to appear during concurrent read and write operations, affecting stability and recoverability.

[0008] Lack of pipelining scheduling for roaming throughput: Existing work mostly performs batch processing and thread parallelism at the software layer, but lacks hardware-side three-stage overlapping pipeline (prefetch decompression / CPU computation / compression write-back) and adjustable prefetch window (PREFETCH_W), making it difficult to maintain steady throughput under complex memory levels and unstable bandwidth.

[0009] The control / register interface is not systematic: Industry solutions often use scattered MMIO registers and interrupt events to drive DMA, lacking standardized conventions for doorbell semantics (fragment retrieval / compression framing), status / backpressure observability, and error codes (CRC / timeout / format) for random walk workloads, which restricts systematic collaboration.

[0010] For example, CN119808857A discloses a hardware acceleration device for processing dynamic graph convolutional neural networks, including: an FPGA parallel acceleration system containing multiple acceleration modules for processing different types of computational tasks; a dynamic data flow control system for calling acceleration modules to perform data computation according to the data processing task of the dynamic graph convolutional neural network, and managing the data transmission and status between the called acceleration modules; and a memory management and data scheduling system for allocating memory space for the acceleration modules, and storing the input data required to execute the data processing task, the intermediate computation results obtained during the execution of the data processing task, and the output data obtained after the execution of the data processing task. This technical solution cannot solve the seven key problems raised in the background art. This technical solution mainly targets the hardware acceleration of graph convolutional neural networks, rather than the optimization of random walks. Although the dynamic data flow control system and memory management and data scheduling system mentioned in this technical solution may have some value in graph data processing, they do not provide specific solutions to the challenges specific to random walks described in detail in the background art.

[0011] In summary, for scenarios combining large-scale dynamic graphs and high-concurrency random walks, there is an urgent need to construct an end-to-end hardware and software collaborative system: maintaining compressed data in main memory for extended periods, decompressing data briefly resident at the near end and injecting it directly into the host cache via a consistency channel; ensuring location and verification through slice self-description; implementing a controllable three-stage overlapping pipeline using doorbell / interrupt + window / backpressure; and limiting update costs to the affected range by incremental updates and the earliest inconsistency backtracking. Currently available technologies in this area focus on the compression / decompression and consistency data paths of FPGA accelerators, low-overhead control interfaces of CPUs, and dynamic... Figure 1 There are still significant technological gaps in the area of ​​integrity maintenance, which urgently need to be addressed.

[0012] Furthermore, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making this invention, but due to space limitations, not all details and contents were listed in detail. However, this does not mean that the present invention does not possess the features of these prior art. On the contrary, the present invention already possesses all the features of the prior art, and the applicant reserves the right to add relevant prior art to the background art. Summary of the Invention

[0013] This invention addresses the random walk problem on large-scale dynamic graphs, solving five major problems inherent in existing solutions: (1) bandwidth and latency overhead caused by multiple data transfers and decompressions back to main memory; (2) insufficient coordination between the data plane and control plane on the device and host sides, making it difficult to form a steady-state pipeline; (3) the graph structure and walker states are stored separately, resulting in poor locality; (4) the lack of precise backtracking and minimum repair range control for the earliest inconsistency after incremental updates; and (5) the lack of slice-level self-description, versioning, and integrity guarantees, leading to unstable concurrent read / write operations. To address these issues, this invention proposes an FPGA-based acceleration method and system that combines compression direct access, cache injection, and hardware / software collaboration, aiming to improve the throughput of random walks in dynamic graph scenarios, reduce memory access latency, and ensure result consistency.

[0014] To address the shortcomings of existing technologies, this invention provides, from a first aspect, an FPGA-based dynamic graph random walk acceleration system. The system includes a main processor and an FPGA accelerator. The main processor is used for information preprocessing; wherein, the original graph is divided into blocks, and directory information for locating data slices is constructed; walkers are activated at the starting vertex, and the graph structure fragments and walker grouping information are compressed into shared slices.

[0015] The FPGA accelerator is used for scheduling and decompression. In response to the main processor's control, it reads the shared packet slice, performs parsing, verification, and decompression operations, and provides the decompressed data to the main processor. The main processor receives the decompressed data, processes walkers at the same vertex to perform random walks, updates the walker states, writes the updated walker states to the walker buffer, and notifies the FPGA accelerator to start the compression operation. In response to the notification, the FPGA accelerator reads the walker states from the walker buffer, performs compression operations on them, writes the compression result back as a walker state slice, and updates the corresponding directory information.

[0016] The core advantages of the acceleration system of this invention are: first, it adopts a co-packet slicing mechanism to jointly compress and store the graph structure and the walker state, reducing the overhead of data transportation and secondary aggregation; second, it uses an FPGA accelerator to directly inject decompressed data into the main processor's cache, avoiding writing decompressed data back to the main memory and significantly reducing memory access latency; and third, it improves the overall throughput and hides latency through a windowed doorbell mechanism and a three-stage overlapping pipeline (prefetch / compute / writeback).

[0017] According to a preferred embodiment, the FPGA accelerator supports dynamic graph updates and path backtracking; wherein, in response to graph structure updates, affected co-package slices are located and repaired, and affected walkers are located and their states are rolled back. This invention supports incremental repair and precise walker backtracking under dynamic graph updates, ensuring computational consistency.

[0018] According to a preferred embodiment, the FPGA accelerator is configured to: receive a descriptor from the main processor and trigger the FETCH doorbell; perform consistent DMA fetching, parsing verification, and decompression operations under window constraints to obtain decompressed data. Upon completion of the operations, a READY interrupt is immediately sent to the main processor; the main processor receives the decompressed data, processes the random walks according to the rule of grouping walkers with the same vertex, writes the processing results back to the walker buffer, and sends a COMPRESS doorbell to the FPGA accelerator to notify it to start the compression operation; the FPGA accelerator performs compression and write-back, performs a consistent read operation on the walker states in the walker buffer, then uses an adaptive encoding scheme to write the encoding results back to the walker state slice, updates the directory information, and sends a DONE interrupt to release a window slot in transit; then it performs graph updates and path backtracking; the process terminates when all walkers meet the termination criteria and there are no pending updates / backtrackings; and outputs plaintext results or archives them in compressed format as needed.

[0019] According to a preferred embodiment, the FPGA accelerator includes a consistency DMA engine unit, a slice parsing unit, a decompression unit, a compression unit, a write-back assembly unit, a control unit, a graph update unit, and a walker backtracking unit. The Consistency DMA engine unit performs consistent DMA slice retrieval operations; the slice parsing unit recalculates the integrity check value of the slice and determines whether the slice is valid; invalid slices are discarded; the decompression unit decodes valid slices, restoring them to aligned adjacent segments and wanderer group segments; the slices are classified and stored in the on-chip temporary buffer according to the type of index information in the slice header; the compression unit performs adaptive encoding on the wanderer state vector; the write-back assembly unit segments the encoding result according to the format constraints of the slice directory table and adds a self-description header, finally encapsulating it into a complete wanderer state slice; the control unit schedules the slice parsing unit, decompression unit, write-back assembly unit, graph update unit, and wanderer backtracking unit; the graph update unit locates the affected co-packed slices according to the update log, performs local repair and atomic header replacement, and maintains the affected index table and the affected list; the wanderer backtracking unit rolls back the relevant wanderers to the earliest inconsistency step according to the index / list, writes the rolled-back wanderer state back to the wanderer buffer 223, and notifies the main processor 101 to perform priority scheduling.

[0020] This FPGA accelerator employs a modular architecture, with each functional unit working collaboratively to achieve efficient random walk processing. The consistency DMA engine unit ensures data transmission consistency, while the slice parsing unit uses CRC checks to guarantee data integrity and filter invalid slices, improving processing efficiency. The decompression unit decodes valid slices back into aligned data segments and stores them in on-chip buffers according to their type, optimizing access performance. The write-back assembly unit uses adaptive encoding technology to repackage walker states into a standard slice format, increasing storage density. The control unit uniformly schedules all processing units, enabling pipelined operations. The graph update unit supports incremental patching of the dynamic graph, maintaining data consistency through atomic head swapping operations and avoiding the overhead of full graph reconstruction. The walker backtracking unit can accurately locate and roll back affected walkers, ensuring computational correctness. The advantages of this architecture lie in its hardware-level parallel processing, data compression, and error recovery, significantly improving the execution efficiency and system reliability of the random walk algorithm.

[0021] According to a preferred embodiment, the system further includes a main memory that stores graph data and walker data in a compressed state, and sets up a two-level directory for locating tiles; the two-level directory includes a block-level directory table and a tile directory table. The block-level directory table is used to map tile IDs to tile ranges; the tile directory table is used to map tile IDs to physical addresses, lengths, and format metadata.

[0022] The acceleration system of this invention employs a two-level directory structure to manage compressed and stored graph data and walker data, achieving efficient data location and access. The block-level directory table maps graph tile IDs to corresponding slice ranges, providing a coarse-grained indexing mechanism for quick location of the storage range of large-scale graph data. The slice directory table provides fine-grained mapping relationships, precisely mapping slice IDs to physical addresses, lengths, and format metadata, supporting direct access to specific data slices. The advantages of this hierarchical directory structure are: firstly, compressed storage significantly reduces memory usage and lowers storage costs; secondly, the two-level directory mechanism balances query efficiency and storage overhead, avoiding the performance bottleneck of full table scans; thirdly, the separation of the two directory tables from the compressed slices allows the system to flexibly support different types of graph data and walker states, improving the architecture's versatility; and finally, this invention provides a clear data access interface for FPGA accelerators, facilitating high-speed hardware-level data retrieval and processing, thereby improving the overall system execution efficiency.

[0023] According to a preferred embodiment, the main processor is configured to: organize the original graph into blocks, divide vertices and edges into blocks according to a preset or adaptive strategy, and package the data within the blocks into slices for batch transport; activate several walkers at each starting vertex as needed for the task, initialize their walker states, wherein local structural fragments of the same block and the grouping states of the walkers in that block are merged and packaged into the same slice, compressed, and written to the main memory.

[0024] This technical solution optimizes graph data storage and access efficiency through block organization and a joint packaging strategy. The main processor divides the original graph into blocks according to a preset or adaptive strategy, achieving local data management and facilitating subsequent batch processing. The key innovation lies in merging and packaging local structural fragments of the same block with the grouped states of traversers within that block into a single compressed slice. This joint storage method eliminates the overhead of multiple accesses caused by the separate storage of structural and state data in traditional solutions. Its advantages are: improved data locality through block partitioning, reducing cross-block access frequency; the joint packaging mechanism tightly couples related graph structures and traverser states, reducing data association complexity; compressed storage significantly reduces memory usage and bandwidth requirements; and the batch transport strategy fully utilizes memory bandwidth, improving data transmission efficiency. This method effectively solves the core problems of data dispersion, frequent access, and high storage overhead in large-scale graph processing, laying an efficient data foundation for subsequent FPGA-accelerated processing.

[0025] According to a preferred embodiment, the main processor writes a batch of descriptors to the descriptor queue of the FPGA accelerator via the MMIO channel and sends a FETCH doorbell; simultaneously, a transit window is set; the next batch of descriptors is received and processed only when the window is not full; when the FETCH doorbell arrives, the prefetch control unit of the FPGA accelerator triggers the coherence DMA engine unit to perform a DMA read operation on the co-packet slice via the coherence interconnect channel to obtain the target co-packet slice; the slice parsing unit performs fast verification on the read slice header fields FID / VER / CRC, and records the error in the error register and automatically re-fetches when the verification passes; when the verification passes, the decompression unit decodes the adjacency information and the vertex-grouped walker states in parallel in the on-chip temporary buffer, and rearranges them into a layout that the main processor can directly consume.

[0026] This technique achieves efficient slice scheduling and processing through a doorbell mechanism and window control. The main processor uses the MMIO channel to write descriptors in batches to the FPGA accelerator's descriptor queue and triggers the processing flow via the FETCH doorbell. A transit window is set to prevent overload and ensure stable system operation. The FPGA accelerator's prefetch control unit responds to the doorbell signal and efficiently reads the co-packet slices from main memory through the consistency DMA engine unit. The slice parsing unit performs rapid verification on the slice header, automatically recording and re-fetching any anomalies to ensure data integrity. After successful verification, the decompression unit decodes the adjacency information and walker states in parallel in the on-chip temporary buffer and rearranges them into a layout suitable for direct use by the main processor. The advantages of this technique are: first, the doorbell mechanism enables asynchronous triggering, improving processing efficiency; second, window control avoids system congestion; third, hardware-level verification ensures data reliability; and fourth, on-chip parallel decoding and rearrangement optimization reduce the burden on the main processor, thus improving the overall throughput and response speed of random walk processing.

[0027] According to a preferred embodiment, the main processor reads the adjacency information and the states of the walkers grouped by vertex; for each walker in the group, the main processor generates the next-hop vertex according to the adjacency list and a preset random strategy, and updates the walker state; after completing the current batch of walks, the main processor writes the updated walker state into the walker buffer and sends a compressible COMPRESS doorbell through the MMIO channel; then it performs compressed write-back.

[0028] This technical solution achieves efficient random walk computation through a processing and state update mechanism. The main processor receives decompressed adjacency information and vertex-grouped walker states from the FPGA accelerator. It performs batch computation on walkers within each group, determining the next-hop vertex based on the adjacency list and a pre-defined random strategy, and updating the walker states. After completing the batch walk, the updated walker states are written to the walker buffer, and the FPGA accelerator is notified via the COMPRESS doorbell to initiate compression write-back. The advantages of this implementation are: first, the vertex-grouped batch processing fully utilizes data locality, reducing redundant access overhead; second, it separates computation from data movement, avoiding frequent memory interactions; third, the doorbell mechanism enables pipelined collaboration between computation and compression, improving overall efficiency; and fourth, state updates are centrally processed on the main processor, ensuring algorithm flexibility and correctness. This method effectively balances the computational load, reduces the number of data moves, and significantly improves the execution efficiency of the random walk algorithm.

[0029] According to a preferred embodiment, when the FPGA accelerator sends a request to read the walker state to the walker buffer via the coherence interconnect channel, if the cache line corresponding to the target data to be read still resides in the main processor's cache, the FPGA accelerator directly obtains the target data from the cache via the coherence interconnect channel and determines the latest walker state; if the target data is not hit, it accesses the main memory; the compression unit in the FPGA accelerator adaptively encodes the walker state vector and outputs a compressed stream; the write-back assembly unit receives the compressed stream output by the compression unit, divides the walker state into fixed-length or variable-length segments according to the format constraints in the slice directory table, adds self-describing header information, and then encapsulates them into complete walker state slices and records them in the slice directory table; after compression is completed, the FPGA accelerator writes the newly generated walker state slice into the walker buffer in the main memory via the coherence interconnect channel and updates the corresponding entry in the slice directory table.

[0030] This technical solution achieves efficient walker state write-back through consistent interconnects and adaptive compression. The FPGA accelerator utilizes the consistency domain to prioritize accessing cached data, only accessing main memory when a cache miss occurs, thus optimizing data read efficiency. The compression unit adaptively encodes the walker state vector, and the write-back assembly unit segments and encapsulates the data according to the slice directory table format constraints, ensuring format standardization. Finally, the data is persistently written to main memory and the directory table is updated. Its advantages include: fully utilizing the cache hierarchy to reduce memory access latency; adaptive compression to improve storage efficiency; standardized encapsulation to facilitate data management; and optimized write strategies to avoid cache pollution, improving overall system performance.

[0031] According to a preferred implementation, when the update log records an operation of adding or deleting an edge, the graph update unit accurately locates the shared slices affected by this update based on the block-level directory table or slice directory table, and then performs read, local repair, recompression, atomic head swap and write-back operations only on these slices, while maintaining the directory synchronously; after the repair is completed, an impact index table and an affected list are generated / refreshed based on the update content. The impact index table records the mapping of the affected blocks / vertices / adjacent segments to their walker sets, and the affected list provides a list of specific walker IDs that need to be reviewed.

[0032] This technical solution achieves efficient dynamic maintenance of graph data through precise positioning and incremental update mechanisms. The graph update unit accurately locates affected shared-package slices based on the catalog table, performing local repair operations only on these slices to avoid full graph reconstruction. The repair process includes reading, local modification, recompression, atomic head swapping, and write-back, while simultaneously updating the catalog table. Upon completion, an impact index table and an affected list are generated. The former records the mapping relationship from blocks / vertices / adjacent segments to the walker set, while the latter provides the specific walker IDs that need to be verified. Its advantages include: precise positioning reduces unnecessary data access; incremental updates reduce computational overhead; atomic operations ensure data consistency; and the impact analysis mechanism ensures the correctness of walker states, thus improving the overall efficiency and reliability of dynamic graph processing.

[0033] According to a preferred embodiment, the walker backtracking unit in the FPGA accelerator retrieves the latest state of the relevant walker based on the influence index table and the affected list: it reads first from the walker buffer, and if archived, it obtains it from the walker state slice decompressed.

[0034] This technical solution achieves efficient recovery of wanderer states through an intelligent backtracking mechanism. The wanderer backtracking unit accurately locates the wanderers requiring review based on the impact index table and the affected list, prioritizing direct reading of the latest state from the wanderer buffer. If the state has been archived, it is decompressed and retrieved from the wanderer state slice. Its advantages include: precise location based on impact analysis avoids full state retrieval; a dual-layer storage strategy optimizes access efficiency, with fast read speeds from the wanderer buffer and archive decompression ensuring data integrity; on-demand backtracking reduces unnecessary state recovery overhead; and automated processing reduces system complexity, overall improving the response speed and resource utilization of wanderer state management in dynamic graph environments.

[0035] This invention provides a second aspect of an FPGA-based method for accelerating random walks in dynamic graphs. The method includes: a main processor performing information preprocessing; wherein the original graph is divided into blocks to construct directory information for locating data slices; a walker is activated at the starting vertex, and the graph structure fragments and walker grouping information are compressed into a shared-packet slice; and an FPGA accelerator performing scheduling and decompression; wherein, in response to the control of the main processor, the shared-packet slice is read, parsing, verification, and decompression operations are performed, and the decompressed data is provided to the main processor; the main processor receives the decompressed data, processes walkers at the same vertex to perform random walks, updates the walker state, writes the updated walker state into the walker buffer, and notifies the FPGA accelerator to start the compression operation; in response to the notification, the FPGA accelerator reads the walker state in the walker buffer, performs compression operations on it, writes the compression result back as a walker state slice, and updates the corresponding directory information.

[0036] The method of this invention brings significant performance improvements and ease of use to users. Users can achieve higher random walk processing throughput, FPGA accelerators significantly reduce computation time, and output results on demand to meet the needs of different application scenarios. This method also enables hardware and software co-optimization for producers: improving data organization efficiency through block-level partitioning and two types of directory table management; achieving pipelined parallel processing through doorbell mechanisms and window control; reducing data migration overhead through a consistent DMA engine unit and data cache injection; and optimizing storage efficiency through adaptive compression and incremental update mechanisms. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the overall structure and data path of the FPGA-based dynamic graph random walk acceleration system provided by the present invention; Figure 2 This is a schematic diagram of the two-level directory and slice data structure provided by the present invention; Figure 3 This invention provides a three-segment overlapping flow and throughput window control diagram for the runtime; Figure 4 This is the interrupt / doorbell timing diagram provided by the present invention; Figure 5 This is a schematic diagram of the dynamic graph update and backtracking link provided by the present invention; Figure 6 This is a flowchart illustrating the FPGA-based dynamic graph random walk acceleration method provided by the present invention.

[0038] List of reference numerals 101: Main Processor; 102: FPGA Accelerator; 103: MMIO Channel; 104: Coherence Interconnect Channel; 105: Interrupt Channel; 120: Coherence DMA Engine Unit; 122: Descriptor Queue; 124: Prefetch Control Unit; 130: Decompression Unit; 132: Slice Parsing Unit; 140: First On-Chip Temporary Buffer; 141: Second On-Chip Temporary Buffer; 160: Graph Update Unit; 165: Walker Backtracking Unit; 170: Compression Unit; 172: Write-back Assembly Unit; 180: Control Unit; 190: MMIO Register Set; 200: Main Memory; 205: Cache; 210: Block-Level Directory Table; 211: Slice Directory Table; 212: Impact Index Table; 223: Walker Buffer; 224: Walk State Slice; 225: Co-packet Slice; 226: Update Log; 227: Affected List. Detailed Implementation

[0039] The following is a detailed explanation with reference to the accompanying drawings.

[0040] This invention provides explanations for some terms and nouns.

[0041] Dynamic graph: refers to a graph data structure whose structure changes over time. Its vertex set and / or edge set can be dynamically updated during system operation, including but not limited to the addition, deletion and attribute changes of vertices and edges.

[0042] Random walk: A random traversal process performed on a graph structure. Starting from a given initial vertex, the process iteratively visits neighboring vertices according to a predefined transition probability distribution (such as randomly selecting neighboring nodes with equal probability, or selecting based on edge weights), thereby generating a series of vertex sequences or paths.

[0043] FETCH doorbell: A task trigger signal, specifically defined as follows: After the main processor 101 prepares the descriptor, it writes a trigger value to a specific register of the FPGA accelerator 102, thereby generating a signal to notify the hardware to initiate the instruction fetch (FETCH) operation. The FETCH doorbell is used to define the logical trigger point for the FPGA accelerator 102 to transition from the idle state to the instruction fetch state.

[0044] COMPRESS doorbell: A task trigger signal, specifically defined as: a start notification signal sent by the main processor 101 to the FPGA accelerator 102 via register write after completing the random walk processing and writing back to the walker buffer 223. The COMPRESS doorbell is used to notify the FPGA accelerator 102 to immediately start the subsequent compression operation.

[0045] Writing: refers to the operation of recording data information into a storage medium using specific technical means.

[0046] Cache Injection: refers to the operation in which the coherent DMA engine 120 initiates a cacheable write transaction directly to the cache level of the main processor 101 via the coherent interconnect channel 104 without the participation of the main processor 101, so that the target data resides directly in the cache 205.

[0047] Write-back: This refers to the process where the main processor 101 modifies the cache contents when a cache write hit occurs, without immediately writing them to the main memory 200, which serves as the main memory. The modified data is only written back to the main memory 200 when the cache contents are swapped out.

[0048] Water level: refers to the occupancy or filling status of the temporary buffer 140 on the first chip or the temporary buffer 141 on the second chip, used for flow control and system stability management.

[0049] A consistency domain refers to the range of data consistency jointly comprised of the main processor 101, main memory 200, and FPGA accelerator 102, and managed by the same cache consistency protocol 205. Within this consistency domain, accesses to the same cache line from different origins follow uniform visibility and consistency rules.

[0050] The following terms are names of structures within the main memory 200.

[0051] Block-level directory table (BlockDir) 210: Records tile IDs, realizes the mapping of the first tile / tile number, and is used to quickly locate the tile range corresponding to a tile.

[0052] SliceTable 211: Records the physical address, length, and format / verification information (such as KIND, FID, VER, CRC) of each slice for DMA, decompression, and update processes to look up and verify.

[0053] Impact Index Table 212: An index derived from update log 226, associated with affected tiles / vertices, and records related walkers / slices, used for quickly filtering objects that need to be backtracked.

[0054] Wanderer buffer 223: Used to store intermediate state data written by the main processor 101 during the computation phase, as a computation write-back area, which is located in the main memory 200; the FPGA accelerator 102 consistently reads data from the wanderer buffer 223 and performs compression; in a preferred embodiment, the hot data in the wanderer buffer 223 has a high cache residency probability during runtime, so that the FPGA accelerator 102 prioritizes reading the wanderer buffer 223 by hitting the cache 205 of the main processor 101 to improve the hit rate.

[0055] Walker Slices 224: Used for writing back to the archive. This refers to compressing the updated walker state into slices and writing it back, which facilitates subsequent recovery, backtracking, and incremental processing.

[0056] CoPack Slices 225: These are the read sources. They group and compress the adjacency structure of a tile and the walkers on that tile into slices, facilitating one-time reading and near-end decompression.

[0057] UpdateLog 226: Records the addition, deletion, and attribute changes of the dynamic graph in chronological order, serving as input for triggering graph structure updates and backtracking processes.

[0058] AffectedList 227: Lists the specific rovers that need to be rolled back (ID, block, rollback position / step number, etc.), for rovers to be processed one by one and rescheduled with priority by the rovers backtracking unit 165.

[0059] Except for the walker buffer 223, the above structure is stored in the main memory 200 by default. In a preferred embodiment, hot data in the walker buffer 223 has a high cache resident probability during runtime, so that data access to the walker buffer 223 is preferentially returned by the cache 205 of the main processor 101. KIND indicates the slice purpose (e.g., 0=CoPack / read source, 1=Walker / write back), FID indicates the encoding format type, VER is the version number, and CRC is the integrity check value.

[0060] The following terms are names of structures within the FPGA accelerator 102.

[0061] MMIO Register Block 190: A set of control / status registers located on FPGA accelerator 102. The main processor 101 reads and writes through MMIO channel 103 to start / stop tasks, configure window size, read walker status and error codes, and mask / enable interrupts.

[0062] Controller 180: As the scheduling and state machine core of FPGA accelerator 102, it is responsible for receiving doorbell signals, allocating bandwidth, advancing the S200 / S400 pipeline stage, handling back pressure and exceptions, and performing corresponding processing when READY / DONE / ERROR interrupts are triggered.

[0063] Descriptor Queue 122: The entry point for the main processor 101 to submit pending items (such as block_id / slice_id); the control unit 180 receives tasks from the descriptor queue 122 and triggers the subsequent DMA read and decompression process.

[0064] Prefetch Controller Unit 124: Maintains the number of slices in transit not exceeding PREFETCH_W; Adaptively adjusts the speed based on the level of the temporary buffer 140 on the first chip or the temporary buffer 141 on the second chip and the back pressure of the main processor 101 to ensure steady-state throughput.

[0065] Coherent DMA Engine unit 120: Initiates coherent memory access requests to the address spaces of main memory 200 and main processor 101 through coherent interconnect channel 104, processes the coherent memory access requests based on cache coherence protocol, so that data can be directly returned by cache 205 of main processor 101 or received by cache 205 and kept consistent; supports cacheable read / write, cache injection, non-temporal / no-allocate write-back and other modes.

[0066] Slice Parser Unit 132: Used to parse slice headers (such as KIND / FID / VER / CRC / off_). Complete version and integrity verification; when an error occurs, record ERROR_CODE and retrieve or report it according to the policy.

[0067] Decompression Unit 130: Decodes the adjacent segments and wanderer groups in the common slice 225 respectively, and rearranges them into a layout that can be directly consumed by the main processor 101; the decoding results are written to the cache 205 of the main processor 101 by cache injection.

[0068] On-chip temporary buffer (StageBuf): Includes the first on-chip temporary buffer 140 and the second on-chip temporary buffer 141; these are the turnaround buffers for the S200 / S400 stage pipeline, respectively. The on-chip temporary buffer is used to receive compressed slices read by the consistency DMA engine unit 120, intermediate decompression results from the decompression unit 130, and compressed outputs from the compression unit 170, providing buffering isolation and backpressure absorption for the parallel stage.

[0069] Compression Unit 170: Adaptively compresses the walker state of walker buffer 223 (such as intra-group differential Δ+ variable length coding, bit-plane coding, or pass-through backoff), dynamically balancing the equivalent compression ratio and latency.

[0070] Write-back Assembler Unit 172: Encapsulates the compression result of compression unit 170 into a wandering state slice 224, outputs it in the order of writing content first and then writing header, and updates the entries (VER / CRC / len, etc.) of slice directory table 211 when necessary.

[0071] Graph Update Unit 160: Reads added and deleted edges from update log 226, accurately locates the affected shared slice 225 according to block-level directory table 210 / slice directory table 211, performs local repair, recompression, and atomic head swap, and maintains the consistency of directory information. Directory information refers to the information recorded in block-level directory table 210 and slice directory table 211.

[0072] Walker Backtrack Unit 165: Locates the walker that needs to be backtracked according to the impact index table 212 / affected list 227; initiates a consistent read of the data in the walker buffer 223 via the consistency interconnect channel 104 to obtain its current intermediate state; when the walker does not have a valid historical state in the walker buffer 223 or needs to backtrack to a historical step not covered by the walker buffer 223, it reads the corresponding archive slice from the walker state slice 224, backtracks to the earliest inconsistent step, writes the backtracked walker state back to the walker buffer 223, and notifies the main processor 101 to perform priority scheduling.

[0073] This invention provides an FPGA-based dynamic graph random walk acceleration system and method. This invention may also provide a storage medium for storing the encoding program of the FPGA-based dynamic graph random walk acceleration method of this invention. This invention further provides an electronic device, including a main processor 101, an FPGA accelerator 102, and a main memory 200. This invention may also provide an electronic device, including a main processor 101, an FPGA accelerator 102, a main memory 200, and a terminal device.

[0074] Preferably, the physical hardware of the main processor 101 includes a central processing unit (CPU), a multi-core processor, and a dedicated integrated circuit. The physical hardware of the main memory 200 includes storage hardware such as DRAM and disk storage. The terminal devices include smart terminals such as computers, tablets, smartphones, smartwatches, and smart glasses.

[0075] Example 1

[0076] like Figure 1As shown, the system of the present invention includes a main processor 101, an FPGA accelerator 102, and a main memory 200. Control registers are located in the MMIO register group 190 of the FPGA accelerator 102. The MMIO register group 190 provides a control interface for the main processor 101 to configure and query the status of the FPGA accelerator 102. The main processor 101 performs read and write operations via the MMIO channel 103. The main processor 101 receives task completion / error events from the FPGA accelerator 102 via the interrupt channel 105. The FPGA accelerator 102 interacts with the cache 205 / memory level of the main processor 101 via a coherence interconnect channel 104 (e.g., CXL.cache, CCIX, or an equivalent implementation of PCIe+DDIO).

[0077] The main memory 200 stores the following structures: directory information (including block-level directory table 210 and slice directory table 211), compressed state data (involving wandering state slice 224 and co-packed slice 225), computation write-back data (involving wanderer buffer 223), and update-related data (involving update log 226, affected index table 212, and affected list 227).

[0078] Figure 1 The numbers in the diagram represent the critical paths, as detailed below.

[0079] ① indicates that the co-packet slice 225 (which can be non-temporal / no-snoop) is read from the main memory 200 through the consistency interconnect channel 104.

[0080] ② indicates (in) Figure 1 (Not shown) The slice parsing unit 132 in the FPGA accelerator 102 performs slice parsing, version and CRC verification.

[0081] ③ indicates that the decompression unit 130 writes the result to the cache 205 of the main processor 101 in the manner of cache injection (cacheable write, allocated to the cache 205 of the main processor 101 through the consistency field).

[0082] ④ indicates that the main processor 101 reads the decompressed data from its cache 205, performs random walk calculations, and writes the updated walker state into the walker buffer 223.

[0083] ⑤ indicates that the main processor 101 sends COMPRESS doorbell executable compression through MMIO channel 103.

[0084] ⑥ indicates that the FPGA accelerator 102 reads the walker state in the walker buffer 223 in a consistent read manner through the consistency interconnect channel 104; when the cache line corresponding to the walker state as the target data resides in the cache 205 of the main processor 101, it is directly returned by the cache 205; when the walker state as the target data is not hit, the memory controller accesses the main memory 200 and returns the data.

[0085] ⑦ indicates that the FPGA accelerator 102 writes the compressed walk state slice 224 back to the main memory 200 (which can be in non-temporal / no-allocate mode).

[0086] like Figure 1 As shown, the FPGA accelerator 102 internally includes: an MMIO register group 190, a control unit 180, a descriptor queue 122, a prefetch control unit 124, a coherent DMA engine unit 120, a slice parsing unit 132, a decompression unit 130, a first on-chip temporary buffer 140, a second on-chip temporary buffer 141, a compression unit 170, a write-back assembly unit 172, a graph update unit 160, and a walker backtracking unit 165.

[0087] The main processor 101 is used for information preprocessing, including: dividing the original graph into plots and constructing catalog information for locating data slices; activating walkers at the starting vertex and compressing the graph structure fragments and walker grouping information into co-packed slices 225.

[0088] The FPGA accelerator 102 is used for scheduling and decompression, including: in response to the control of the main processor 101, reading the co-packet slice 225, performing parsing verification and decompression operations, and providing the decompressed data to the main processor 101.

[0089] The main processor 101 receives the decompressed data, processes the walkers at the same vertex to perform random walks, updates the walker state, writes the updated walker state into the walker buffer 223, and notifies the FPGA accelerator 102 to start the compression operation.

[0090] In response to the notification, FPGA accelerator 102 reads the wander state in wander buffer 223, performs a compression operation on the wander state, writes the compression result back as a wander state slice 224, and updates the corresponding directory information.

[0091] The FPGA accelerator 102 also supports dynamic graph updates and path backtracking, including: locating and repairing affected co-pack slices 225 in response to graph structure updates, and locating affected walkers and reverting their states.

[0092] The aforementioned hardware framework can be implemented physically independently, or it can be combined or reused when resources are available, and neither of these constitutes a limitation on the claims.

[0093] Preferably, the main memory 200 of the present invention is configured with a two-level directory: a block-level directory table 210 and a slice directory table 211. The block-level directory table 210 maps the tile ID to the first slice and the slice number. The slice directory table 211 maps the slice index to the physical address and the format indicator FID / VER / CRC. Preferably, the data is stored in a compressed state for a long time; specifically, the co-package slice 225 (storing the graph structure and wanderer groups) is used as the read source, and the wandering state slice 224 generated after calculation is used as the write-back destination. The wanderer buffer 223 is the calculation write-back area used by the main processor 101 to calculate the write-back, and its data is stored in the main memory 200 by default; in a preferred embodiment, the hot data in the wanderer buffer 223 has a high cache resident probability during runtime, so that the data access to the wanderer buffer 223 is preferentially returned by the cache 205 of the main processor 101; the update log 226, the impact index table 212, and the affected list 227 related to the update are used for incremental updates and backtracking. Slice size is configurable (preferred 2–16 MB, default 8 MB), and a content-first, header-later write order is used to avoid consuming half-slices. To support... Figure 1 The compressed direct path shown in this invention employs a two-level directory and self-describing slice storage organization in the main memory 200. See [link / reference]. Figure 2 .

[0094] Preferably, the update log 226 is located in DRAM (main memory 200) and is used to store the update log of the dynamic graph. Internally, it employs a circular buffer or a segmented append-only sequential structure. Each log entry includes a source vertex identifier (src_id), a target vertex identifier (dst_id), an operation type (op, used to indicate adding / deleting edges), a timestamp (ts), and optional edge identifiers (opt_eid) and version information (opt_ver). The log is directly written to the main memory 200 by the main processor 101. After writing is complete, notification can be sent via a doorbell write, or the graph update unit 160 can detect the existence of a new log by polling the tail pointer.

[0095] Preferably, the impact index table 212 is also located in DRAM (main memory 200) to record the blocks or vertices affected by the update. Its implementation can be as follows: a block-level bitmap (bitmap[block_id]) marks the affected blocks; within the block, specific affected vertices are further marked using a vertex sub-bitmap or a sparse table; and an optional boundary summary is used to quickly filter adjacent segments that need repair. The graph update unit 160 can directly read and write the impact index table 212 locally through the consistency interconnect channel 104 without having to move the entire table to the on-chip temporary buffer.

[0096] Furthermore, the directory area and data area are also located in DRAM (main memory 200). The block-level directory table 210 and the slice directory table 211 are used to store metadata such as slice location, format, version, and CRC, enabling the map update unit 160 to perform precise addressing and head-swapping operations during the update process. The block-level directory table 210 and the slice directory table 211 can establish a two-level mapping relationship when the descriptor is in the FETCH doorbell triggered state, so that the main processor 101 fixes the access logic to a specific physical address in the main memory 200.

[0097] The shared packet slice 225 is used to store the adjacent segments of the graph structure and optional shared packet information, serving as the compressed read source; the wanderer buffer 223 and the wandering state slice 224 serve as the calculation write-back buffer and compressed archive area for the wanderer state, respectively.

[0098] To facilitate understanding of the following runtime data path, this invention provides a lightweight control plane outside the data plane (consistency interconnect channel 104): the main processor 101 communicates via MMIO channel 103 (see...). Figure 1 The system reads and writes to the MMIO register group 190 on the FPGA accelerator 102 and receives READY / DONE / ERROR events via interrupt channel 105. The control plane only transmits a small number of statuses and doorbell signals, while the data plane handles batch data transfer and compression / decompression. Key rules are as follows (see Table 1 and...). Figure 4 ).

[0099] Doorbell and Window: The main processor 101 triggers two types of actions by writing to the KICK_DOORBELL register: bit 0 = FETCH is used to initiate prefetching, decompression, and cache injection in the S200 stage; bit 1 = COMPRESS is used to initiate compression and write-back in the S400 stage. To prevent overload, the FPGA accelerator 102 maintains an in-transit window, the upper limit of which is specified by PREFETCH_W (recommended value 2–4). The next chip can only be received and processed when the window is not full, and the window slot is reclaimed when the DONE event arrives (see...). Figure 4 (T4 in the middle).

[0100] Ready and Completed: STATUS.READY is set and an interrupt is triggered upon completion of each cache injection; STATUS.DONE is set and an interrupt is triggered upon completion of a batch of compressed write-back. To suppress interrupt storms, READY / DONE supports threshold merging, which can be configured as a mixed interrupt and polling mode via INT_MASK.

[0101] Consistency and Visibility: In the CXL / CCIX variant, reads and writes of the FPGA accelerator 102 to the main processor 101's cache 205 are inherently consistent. In the PCIe+DDIO variant, the main processor 101 performs CLWB / SFENCE before allowing the FPGA accelerator 102 to read the wanderer buffer 223 to ensure visibility in the S400 stage. Decompressed data is written to the main processor 101's cache 205 via cache injection, thus preventing decompressed data from falling back to main memory 200.

[0102] Back pressure and tuning: When the level of the first on-chip temporary buffer 140 or the second on-chip temporary buffer 141 is too high, or when cache injection of the main processor 101's cache 205 is limited, the FPGA accelerator 102 raises STATUS.BACKPRESS to suspend new DMA requests; the main processor 101 can then temporarily lower PREFETCH_W or delay the doorbell accordingly. STAGE_WM provides a level reading for debugging.

[0103] Mapping with flowing water: Figure 4 The control timing sequence from T0 to T4 is shown. Initialization (T0) → Chip Retrieval Doorbell (T1) → READY / Enter S300 (T2) → Compress Doorbell (T3) → DONE / Window Reclamation (T4). The three segments S200 / S300 / S400 overlap in time, and the window is used to maintain steady-state throughput.

[0104] In this invention, the random walk process is mainly divided into three stages: a prefetch / decompression / injection stage (S200 stage), a main processor 101 calculation stage (S300 stage), and a compression and write-back stage (S400 stage). These three stages are executed in an overlapping pipeline manner, thereby maximizing throughput while ensuring consistency. The specific process is described below.

[0105] S200: Prefetch / Decompression / Injection Phase (executed by FPGA Accelerator 102).

[0106] First, the control unit 180 reads a batch of descriptors (block_id, slice_id) sequentially from the descriptor queue 122. When the number of slices in transit is less than the prefetch window limit PREFETCH_W (preferably 2-4, default 3), the next batch of descriptor requests is allowed to be sent; when the limit is reached, the process is suspended until a DONE interrupt is received from the interrupt channel 105, after which one in-transit window slot is released.

[0107] Subsequently, the coherent DMA engine unit 120 reads the corresponding compressed slice from the co-packet slice 225 (see ①) via the coherent interconnect channel 104 and writes it to the first on-chip temporary buffer 140 / the second on-chip temporary buffer 141. A non-temporal / no-snoop mode can be used here to reduce cache pollution.

[0108] Next, the slice parsing unit 132 performs an integrity check on the input slice. The method is as follows: first, it reads the FID, VER, CRC, and offset information from the slice header; then, it recalculates the integrity check value for the actual data and compares it with the value carried in the header. When the two match, the slice is deemed valid and proceeds to the subsequent indexing and decoding stages; if they do not match, the slice is discarded, and an error signal is reported to the wanderer backtracking unit 165 via the error signal interface to ensure system robustness.

[0109] After successful verification, the decompression unit 130 first decodes the compressed data, restoring it to aligned adjacent segments and walker segments. Based on the index information in the slice header, it distinguishes the type: if slice_type=walker, the walker segment is sent to the first on-slice temporary buffer 140; if slice_type=block, the adjacent segment is sent to the second on-slice temporary buffer 141. This reassembles the data layout within the first and second on-slice temporary buffers into a layout directly consumed by the main processor 101, providing input for subsequent parsing and verification. Subsequently, this decompressed data is written to the main processor 101's cache 205 (see ③) using either cacheable write or cache injection, thereby preventing data from falling back to main memory 200 and reducing access latency.

[0110] The FPGA accelerator 102 is in the FETCH state during computation prefetching, so that the slice parsing unit 132 and the decompression unit 130 have a pipelined serial processing relationship. The decompression unit 130 is connected to the output with alignment mapping function through an internal rearrangement layout circuit to realize physical address reorganization, thereby matching the cache line alignment rules of the cache 205 of the main processor 101.

[0111] After injection is complete, the FPGA accelerator 102 generates a READY interrupt through the interrupt channel 105, notifying the main processor 101 to enter the calculation stage (S300). If the level of the first on-chip temporary buffer 140 / the second on-chip temporary buffer 141 is too high or the injection of cache 205 is blocked, new DMA requests are suspended to maintain steady-state throughput.

[0112] In the PCIe+DDIO variant, cache injection relies on DDIO; if the platform does not have this feature, the alternative is to resident the decompressed data in a 200-sliding window buffer in main memory, while keeping the rest of the process unchanged.

[0113] The present invention achieves a synergistic optimization effect by using the structure / state pre-binding technology implemented by the co-packaging slice 225, combined with the near-end decompression injection of the consistent interconnect channel 104. This not only improves the effective bandwidth utilization of the bus by reducing the number of random memory accesses, but also eliminates the memory access latency caused by DRAM relay in the traditional scheme by decompressing data directly to the cache 205.

[0114] S300: Calculation phase (executed by main processor 101).

[0115] In phase S300, the main processor 101 directly reads the data injected in phase S200 from its cache 205 and performs random walk steps according to the strategy of grouping walkers at the same vertex (supporting first-order or second-order modes). This includes various processing methods such as batch processing and sequential processing. During this process, the main processor 101 updates the walker states such as curr_v, prev_v, and steps_left, and writes the results back to the walker buffer 223 (see ④).

[0116] After the write-back is complete, the main processor 101 sends a COMPRESS doorbell to the FPGA accelerator 102 via MMIO channel 103 (see ⑤) to trigger the subsequent compression and write-back phases. In the PCIe variant, the main processor 101 may execute the CLWB / SFENCE instruction before sending the doorbell to ensure that the results in the wanderer buffer 223 are visible to the FPGA accelerator 102.

[0117] In the case of state write-back, the FPGA accelerator 102 is in the COMPRESS state position, so that the compression unit 170 and the write-back assembly unit 172 have a format constraint-based encapsulation relationship, so that the latest walker state vector can have a non-contaminated write-back relationship with the walker state slice 224 in the main memory 200.

[0118] S400: Compression and Write-back (performed by FPGA Accelerator 102) Upon entering the S400 phase, the FPGA accelerator 102 sends a request to read the walker state to the walker buffer 223 via the coherence interconnect channel 104 (see ⑥). If the cache line corresponding to the walker state to be read is still residing in the cache 205 of the main processor 101, the cache 205 returns the walker state directly to the FPGA accelerator 102 via the coherence interconnect channel 104; if the walker state is not found, the main memory 200 is accessed.

[0119] Subsequently, compression unit 170 performs adaptive encoding on the wanderer state vector and sends the compressed stream to write-back assembly unit 172. Write-back assembly unit 172, according to the format constraints of slice directory table 211, segments the encoding result and appends a self-description header, ultimately encapsulating it into a complete wanderer state slice 224. The detailed generation steps and design rationale are described in the subsequent interrupt / doorbell timing interpretation section (adaptive compression and assembly).

[0120] The generated wandering state slice 224 is written back to main memory 200 by the coherent DMA engine unit 120 in non-temporal / no-allocate mode (see ⑦), and at the same time, the new physical address, length, version number and integrity check value information are registered in the slice directory table 211. After the write-back is completed, the FPGA accelerator 102 generates a DONE interrupt through the interrupt channel 105, returning a window slot, thereby allowing a new task to enter the S200 stage.

[0121] Through the above design, the three segments S200 / S300 / S400 overlap in time to form a steady-state flow: decompression (k+2) | calculation (k+1) | write-back (k) (see...) Figure 3 (Column 4). The prefetch window PREFETCH_W controls the parallelism of slices in transit, used to hide DMA round-trip and decompression delays; there is preheating and air bubble removal in the first and last rounds, but it does not affect the overall functionality and throughput.

[0122] The dynamic graph update and backtracking process of this invention is described below.

[0123] like Figure 1 As shown, the main processor 101 interacts with the FPGA accelerator 102 via MMIO channel 103 / interrupt channel 105. The FPGA accelerator 102 accesses the directory area (storing block-level directory table 210 and slice directory table 211), log and index area (storing update log 226, impact index table 212 and affected list 227), compressed state data area (storing co-packed slice 225 and wander state slice 224) and compute write-back area (as wanderer buffer 223) in the main memory 200 via coherence interconnect channel 104.

[0124] The update chain of the dynamic graph update and backtracking process includes stages U1 to U6.

[0125] like Figure 5As shown, in phase U1 (CPU / Application Write), the application adds or deletes edges and writes them to update log 226. In phase U2 (Read), graph update unit 160 reads update log 226. In phase U3 (Query), graph update unit 160 queries block-level directory table 210 / slice directory table 211 to pinpoint the affected slice. In phase U4 (DMA Read), graph update unit 160 reads the compressed slice from co-package slice 225. In phase U5 (DMA Write-back after Recompression), graph update unit 160 locally repairs and recompresses the co-package slice 225, and synchronously updates the {phys_addr / len / FID / VER / CRC} of the slice entry in slice directory table 211. In phase U6 (Write), graph update unit 160 generates / refreshes the impact index table 212 and the affected list 227.

[0126] The backtracking chain of the dynamic graph update and backtracking process includes stages B1 to B3.

[0127] like Figure 5 As shown, in phase B1 (read), the walker backtracking unit 165 reads the set of affected walkers according to the influence index table 212 or the affected list 227. In phase B2 (read), the walker backtracking unit 165 obtains the current intermediate walker state of the relevant walkers from the walker buffer 223 through a consistent read operation; if the required historical state is not resident in the walker buffer 223, or if it is necessary to backtrack to an earlier historical step, the corresponding archive slice is read from the walker state slice 224 to restore its historical walker state. In phase B3 (write), the walker backtracking unit 165 writes the backtracked walkers back to the walker buffer 223 and notifies the main processor 101 to perform priority scheduling through the interrupt channel 105.

[0128] Figure 5 All U / B Both read and write operations are performed by the consistency engine unit 120 via the consistency interconnect channel 104; it is naturally consistent in the CXL / CCIX variant, and uses write injection and CLWB / SFENCE to ensure visibility in the PCIe+DDIO variant.

[0129] The anomaly handling and consistency of the hardware accelerator of this invention are described below.

[0130] The system follows a content-first, header-later approach when writing slices; the decoding end verifies via VER / CRC, and if it fails, it reports via ERROR_CODE and automatically re-fetches the code. Native consistency of CXL / CCIX is preferred. In the PCIe variant, DDIO write injection is used for implementation ③, and CLWB / SFENCE is executed by the main processor 101 before reading the wanderer buffer 223 to ensure read consistency. Error code enumeration is provided by ERROR_CODE (see Table 1), and STATUS.ERROR is synchronized and pressed when triggered. Figure 4 The abnormal path shown is reported.

[0131] The present invention provides the following description of the two-level directory and slice self-description format.

[0132] Figure 2 This invention illustrates a two-level directory and slice self-describing format used in the main memory 200. This format enables fast and stable data location and verification in scenarios where data is stored and decompressed in the compression unit 170 directly reaches the cache 205. For example... Figure 2 As shown, block-level directory table 210 is used to record the mapping relationship between blocks and slice ranges. The fields include block_id, first_slice, and slice_count.

[0133] For example: block_id=17→first_slice=120, slice_count=3 means that block 17 covers slice_id=120, 121, 122; block_id=18→first_slice=300, slice_count=1 means that only slice_id=300 is covered.

[0134] like Figure 2 As shown, slice directory table 211 is used to record physical location and format information for each slice. Fields include slice_id, phys_addr, len, KIND (optional, used to distinguish read source / write back), FID (format identifier), VER (version), and CRC (integrity check). Slice directory table 211 may also include optional flags.

[0135] Example (with) Figure 2 (consistent with the above) slice 120: phys=0x1200_0000, len=0x0080_0000, KIND=0(CoPack), FID=0(Δ+VLC), VER=21, CRC=0xA1B2.

[0136] slice 121: phys=0x1280_0000, len=0x0080_0000, KIND=0, FID=1(BPC), VER=21, CRC=0xC3D4.

[0137] slice 122: phys=0x1300_0000, len=0x0080_0000, KIND=0, FID=0, VER=21, CRC=0xE5F6.

[0138] slice 300: phys=0x3000_0000, len=0x0040_0000, KIND=1(Walker), FID=2(pass-through), VER=11, CRC=0x1D2C.

[0139] In the above content, KIND=0 indicates that the common-pack slice 225 (read source, adjacent + wanderer common-pack), and KIND=1 indicates that the wandering state slice 224 (write back, wanderer state).

[0140] like Figure 2 As shown, the slice self-description header (SliceHdr) of a total of 225 slices includes at least: FID, VER, CRC, and segment offsets off_adj (start of adjacent segment) and off_walk (start of wanderer block segment). All offsets are byte offsets relative to the slice base address, preferably aligned to 4KiB. For example: Base address 0x1200_0000, off_adj=0x0000_1000→0x1200_1000, off_walk=0x0030_0000→0x1230_0000; the payload AdjSegment and WalkerGroupSegment use compressed encoding indicated by FID (such as Δ+VLC or BPC).

[0141] like Figure 2 As shown, the header of wander state slice 224 (write-back) includes: FID, VER, CRC, and off_groups (wanderer group load start). For example: Base address 0x3000_0000, off_groups=0x0000_1000→0x3000_1000.

[0142] The load consists of several wanderer groups (which can use Δ+VLC / BPC / straight-through rollback in groups).

[0143] Preferred constraints: a) Write order: The order follows the rule of writing the payload first, then the sliceHdr. Consumers use this to verify the VER / CRC beforehand to prevent half-slices from being consumed.

[0144] b) Consistency: The FID / VER / CRC in the slice catalog table 211 are completely consistent with the slice header; after the graph is updated, VER is incremented and CRC is recalculated.

[0145] c) Alignment and Length: off_ Alignment of ≥4KiB is recommended; len is the upper limit of the slice, and a repair margin is reserved at the end.

[0146] d) Heterogeneous support: Different FIDs can be used for the same tile (e.g., 120 / 122=Δ+VLC, 121=BPC) to adapt to different statistical characteristics.

[0147] e) Fault tolerance: When CRC fails or VER does not match, the parsing unit records the error code and retrieves it again; see the repair process. Figure 5 / Figure 6 .

[0148] Preferably, Figure 1 The process involves: ① Accurately locating phys_addr / len / FID through the slice directory table 211, then reading the co-packet slice 225 by the coherent DMA engine unit 120; ② Performing VER / CRC verification in the slice parsing unit 132 and selecting the decoder according to FID; ③ Writing the decoding result to the cache 205 of the main processor 101 via cache injection; ⑦ Writing back the compressed wandering state slice 224 and synchronously updating the VER / CRC of the slice_id in the slice directory table 211 (updating len if necessary).

[0149] Preferably, FID can be extended to other lightweight codes; the KIND field can be omitted and replaced by an address range or table entry attribute. If the platform does not have cache injection functionality, it can still maintain the following... Figure 2 The structure shown remains unchanged, except that the decoding result is placed into the sliding window buffer of the main memory 200, which is an equivalent implementation.

[0150] This self-describing structure guarantees: (i) random location, (ii) verifiable version and integrity, and (iii) evolvable format. Figure 1 The compression unit 170 shown is closely integrated with the storage and cache 205 direct path.

[0151] The present invention describes the control of the three-segment overlapping flow and the throughput window as follows. Figure 3 The diagram illustrates the control of the three-stage overlapping flow and throughput window during operation of this invention. Figure 3In the table, the first row represents the time, the second row represents the number of slices in stage S200, the third row represents the number of slices calculated in stage S300, and the fourth row represents the number of write-back slices in stage S400. The first column represents the category. The second column represents the number of slices in the three stages at time T. The third column represents the number of slices in the three stages at time T+1. The fourth column represents the number of slices in the three stages at time T+2. The fifth column represents the number of slices in the three stages at time T+3.

[0152] like Figure 3 As shown, the FPGA accelerator 102 performs overlapping pipelined processing on slices in three stages: S200 (prefetch / parse / decompress / cache injection via coherence interconnect channel 104), S300 (main processor 101 performs random walk computation, reads from cache 205 of main processor 101, and writes to walker buffer 223), and S400 (FPGA accelerator 102 compresses and writes back walker state slice 224 via coherence interconnect channel 104). In steady state (e.g., time T+2 and later), the FPGA accelerator 102 establishes a parallel processing relationship for adjacent ordinal slices, i.e., S200 processes the (k+2)th slice, S300 processes the (k+1)th slice, and S400 processes the (k)th slice. The window parameter PREFETCH_W=3 (example) limits the concurrency of slices in transit. When stage S200 ends, a READY value is generated (interrupt channel 105). After the write-back in stage S300, stage S400 is triggered via doorbell ⑤ (MMIO channel 103). Stage S400 completes the write-back, generates a DONE value (interrupt channel 105), and releases the window slot. If the platform does not have cache injection functionality, the decompressed data in stage S200 can be temporarily stored in the sliding window buffer of main memory 200, and the remaining processes remain the same, which is an equivalent implementation.

[0153] Specifically, the FPGA accelerator 102 reads the co-package slice 225 through the coherence interconnect channel 104. After verification by the slice parsing unit 132, it is decompressed in the on-chip temporary storage buffer and written to the cache 205 of the main processor 101 via cache injection. Upon completion, a READY event is generated through the interrupt channel 105. After receiving the READY event, the main processor 101 reads the random walk step from its cache 205, writes the update to the walker buffer 223, and sets the compressible doorbell via the MMIO channel 103. The FPGA accelerator 102 reads the walker buffer 223 accordingly, compresses it, assembles the walk state slice 224, and writes it back to the main memory 200 in non-temporal / no-allocate mode. Upon completion, a DONE event is generated, and an in-transit slot is released.

[0154] Three-stage formation to decompress (k+2) | calculate (k+1) | write-back (k) overlap. The window mechanism has adaptive suppression for the instantaneous congestion of the cache 205 of the DRAM / main processor 101. If a CRC / VER failure occurs, the slice parsing unit 132 records the error code and re-fetches; when necessary, it triggers the dynamic graph update and backtracking process (see Figure 5 / Figure 6 ). When terminating or switching the window, the control unit 180 gradually reduces PREFETCH_W to 0 and waits for the in-flight clear to achieve smooth emptying.

[0155] The present invention describes the control / register interface and the interrupt / doorbell timing as follows.

[0156] The control registers are located in the MMIO register group 190 of the FPGA accelerator 102, and the main processor 101 performs read and write operations through the MMIO channel 103. Event notifications are in the form of the interrupt channel 105 (preferably using the MSI-X method). All control registers require 32-bit aligned access. The doorbell register uses the Write-1-to-Set (W1S) semantics, and the STATUS register is a level semantics read-only, used to cooperate with the interrupt status and mask bits to achieve interrupt merging. Unless otherwise specified, the unlisted bits are reserved bits, returning 0 when read and being ignored when written.

[0157] Table 1: Control Interface and Registers (MMIO Channel 103, excerpt)

[0158] The register semantics in Table 1 are described as follows.

[0159] DESCQ_BASE / DESCQ_SIZE: Takes effect immediately after being written by T0; the queue elements are (block_id, slice_id) or its extended format.

[0160] PREFETCH_W: Adjustable during runtime; the FPGA accelerator 102 only receives new FETCH doorbells when the in-flight count < PREFETCH_W.

[0161] KICK_DOORBELL: W1S; can set bits 0 / 1 independently or in combination; if the in-flight is full, the entry is ready but the start is postponed until the window is released.

[0162] STATUS.READY / DONE: Can be merged with interrupts; the main processor 101 can also only poll the STATUS to reduce the interrupt frequency.

[0163] ERROR_CODE: Typical encodings include 1=CRC_FAIL, 2=DMA_TIMEOUT, 3=FORMAT_ERR, and 4=ADDR_FAULT. STATUS.ERROR (if implemented) is also set when an error occurs.

[0164] STAGE_WM: Provides the approximate depth of the on-chip temporary buffer queue, which facilitates on-site parameter tuning of PREFETCH_W.

[0165] Figure 4 The T0–T4 interaction process of the main processor 101, MMIO channel 103, FPGA accelerator 102 (control unit 180) and interrupt channel 105 is shown for interrupt / doorbell timing interpretation.

[0166] T0 Initialization: The main processor 101 writes DESCQ_BASE / SIZE and PREFETCH_W sequentially; the FPGA accelerator 102 updates its registers, and interrupt channel 105 is silenced. The main processor 101 writes PREFETCH_W, and the FPGA accelerator 102 loads the window parameters.

[0167] T1 Startup: The main processor 101 writes descriptors to the descriptor queue 122, and the FPGA accelerator 102 receives the descriptor queue 122 and is ready.

[0168] The main processor 101 then writes KICK_DOORBELL (bit0=FETCH); after receiving it, the FPGA accelerator 102 enters the S200 stage, that is, the FPGA accelerator 102 receives the FETCH doorbell (bit0=FETCH) and then starts the S200 prefetch / decompression process (consistent DMA → slice parsing verification → decompression → cache injection).

[0169] T2READY: After the first chip is injected into the cache 205 of the main processor 101, the FPGA accelerator 102 sets STATUS (READY) and triggers an interrupt; the main processor 101 reads STATUS, retrieves data from the cache 205 of the main processor 101, performs the calculation in stage S300, and writes the result to the walker buffer 223. If it is a PCIe variant, the main processor 101 executes CLWB / SFENCE before allowing the FPGA accelerator 102 to read the walker buffer 223 to ensure that the walker buffer 223 is visible.

[0170] T3 Compression License: The main processor 101 writes KICK_DOORBELL (bit1=COMPRESS), the FPGA accelerator 102 receives the COMPRESS doorbell (bit1=COMPRESS), and starts the S400 compression / write-back process (consistent read of walker buffer 223 → adaptive compression → write back walker state slice 224).

[0171] T4DONE: Write-back complete. FPGA accelerator 102 sets STATUS (DONE) and interrupts; main processor 101 reads STATUS (DONE), which means main processor 101 sends the next batch of descriptors or terminates. Specifically, interrupt channel 105 triggers DONE, FPGA accelerator 102 releases window slots and updates slice directory table 211. Subsequently, main processor 101 can send the next batch of descriptors or terminate.

[0172] Abnormal Path: If CRC / retake fails, the FPGA accelerator 102 sets CRC / retake failure and triggers an ERROR interrupt; after reading ERROR_CODE, the main processor 101 selects retry / skip / report according to the strategy. STATUS.BACKPRESS reflects temporary storage congestion, and the main processor 101 can temporarily reduce PREFETCH_W.

[0173] The above control timing corresponds one-to-one with the three stages of the pipeline: S200, S300, and S400. T1 initiates stage S200, T2 opens stage S300, T3 enables stage S400, and T4 reclaims the window. Combined with PREFETCH_W, this forms a decompression (k+2) | computation (k+1) | write-back (k) sequence (see [link]). Figure 3 The steady-state parallelism of the fourth column improves overall throughput and energy efficiency without increasing DRAM round trips.

[0174] This invention provides a detailed description of the update chain (U1–U6).

[0175] The main processor 101 only needs to write edge addition and deletion requests to the update log 226, while subsequent location, repair, and directory information updates are automatically executed by the graph update unit 160 inside the FPGA accelerator 102. The advantage of this design is that it can offload a large number of repetitive lookup and repair operations to the FPGA accelerator 102, and ensure that all data and directory information are updated synchronously through the consistency interconnect channel 104, thereby avoiding the latency and overhead caused by polling by the main processor 101.

[0176] First, in step U1, the application or main processor 101 writes the edge information that needs to be added or deleted in batches to the update log 226 located in DRAM. Log entries typically include source node ID, destination node ID, operation type (add or delete), timestamp, and optional edge ID or version number. This sequential append structure efficiently records update requests and ensures that the log can be read sequentially.

[0177] Next, in stage U2, the graph update unit 160 inside the FPGA accelerator 102 reads the update log 226 through the consistency interconnect channel 104 to obtain the latest update request. Subsequently, in stage U3, the graph update unit 160 queries the directory area, including the block-level directory table 210 and the slice directory table 211, and accurately locates the affected co-package slice 225, i.e., the compressed data including the relevant adjacent segments, through the block-level index and slice metadata.

[0178] After location is completed, the process enters stage U4. The graph update unit 160 retrieves the target co-package slice 225 from the main memory 200 via the consistency interconnect channel 104, decompresses it to the on-chip temporary buffer, and parses the corresponding adjacency list data. Then, in stage U5, the graph update unit 160 performs local patching (such as adding or deleting edges) on the adjacency segments that need modification and re-compresses them. To ensure consistency, the patched slice is written back following the order of writing content first, then updating the header. The graph update unit 160 writes the patched slice back to the co-package slice 225 via the consistency DMA engine unit 120, and simultaneously atomically updates the physical address, length, version number, and integrity checksum of the entry in the slice directory table 211. This atomic header-swapping strategy ensures that concurrent reads do not encounter incomplete or inconsistent slice data.

[0179] Finally, in phase U6, graph update unit 160 synchronously updates the impact index table 212 and the affected list 227 according to the repair scope. These data structures record the affected tile IDs, vertex sets, and associated walker information. In this way, subsequent backtracking pipelines can quickly hit the walker states that need adjustment, achieving random walk computation strongly consistent with dynamic updates.

[0180] In summary, the update chain U1–U6 in this embodiment decomposes the update process of the dynamic graph into pipelined hardware steps through log-driven, slice positioning, local repair, atomic head replacement, and index impact. This not only significantly reduces the involvement of the main processor 101, but also improves the update throughput and overall computational convergence speed while ensuring consistency.

[0181] This invention provides a detailed description of the backtracking chains (B1–B3).

[0182] After the update chain (U1–U6) completes the positioning, patching, and atomic head replacement of the graph structure, only a small portion of the walkers that are coherent with the modified edges or vertices may become invalid. To avoid the high cost of full recomputation, in this embodiment, a walker backtracking unit 165 is provided within the FPGA accelerator 102, and a minimum necessary rollback and priority scheduling are performed on this portion of the walkers through a three-stage backtracking pipeline (B1–B3), enabling the overall convergence to quickly resume without interrupting the main pipeline (S200 / S300 / S400).

[0183] In stage B1, affected walkers are selected for rollback target determination. The graph update unit 160 has refreshed the influence index table 212 and the affected list 227 in stage U6. The walker backtracking unit 165 reads the influence index table 212 or the affected list 227 through the consistency interconnect channel 104, receives the affected blocks / vertices, obtains the mapping of the walker set, and merges, de-duplicates, and groups by block to form a pending backtracking queue. To determine where to roll back, in this embodiment, the most recent consistent version marker (such as the last_seen_VER of the block or related edge where the walker is located) or the most recent consistent step number (last_consistent_step) of each walker is maintained in the entries of the affected list 227. The walker backtracking unit 165 compares the current VER in the slice directory table 211. If it is found that the last_seen_VER of the walker < current_VER, it is determined that the walker needs to roll back, and the target step number is taken as the last_consistent_step (if not recorded, it is obtained by the walker backtracking unit 165 using short history scanning or heuristic backtracking based on step metadata). The affected objects and rollback targets are both prefixed to the metadata layer of the influence index table 212, the affected list 227, or the slice directory table 211 to avoid reading the entire path history. The rollback range is limited to the earliest inconsistent step, significantly shortening the recomputation length; and grouping by block can improve the locality of subsequent reads and writes.

[0184] In phase B2, the wanderer state is acquired (primarily from cache 205 of main processor 101) and necessary parsing is performed. After determining the backoff target, the wanderer backtracking unit 165 sequentially acquires the latest wanderer state for each wanderer to be backtracked: it prioritizes reading the wanderer state from the wanderer buffer 223 (if the cache line corresponding to the wanderer state resides in cache 205 of main processor 101, it is returned directly by cache 205; otherwise, it is returned by main memory 200); if the wanderer buffer 223 has been evicted or the target is in the archive area, it reads from the wanderer state slice 224 and is quickly decoded by the decompression unit 130 to obtain the current {prev_v,curr_v,step_cnt,flags,rng_seed} fields. If it is necessary to confirm the backoff boundary and the state contains a short step trace / signature (optional lightweight history), the wanderer backtracking unit 165 only needs to parse the short segment and does not need to decode the complete path. The main processor 101 can obtain the state with zero copy if the cache 205 is hit; the wander state slice 224 is only accessed and decompressed when the wander state in the wander buffer 223 is not hit, avoiding additional pressure on the storage system. Read amplification is minimized; combined with short-range traces, only necessary parsing is performed, achieving both high latency and high bandwidth.

[0185] In phase B3, the earliest inconsistency is rolled back, written back, and rescheduled with priority. The walker backtracking unit 165, according to the goal determined in phase B1, rolls back the corresponding walker to the earliest inconsistency (e.g., the last consistent {prev_v,curr_v, step_cnt} before edge deletion takes effect), resetting relevant flags and random number advance states if necessary to ensure repeatability. Subsequently, the walker backtracking unit 165 writes the updated walker state back to the walker buffer 223 via the consistency interconnect channel 104, marking the entries with high priority. After the write-back is complete, the walker backtracking unit 165 triggers a backtracking completion / priority scheduling notification via the interrupt channel 105. The scheduler of the main processor 101 then prioritizes the computation of these blocks in the subsequent S300 phase, allowing them to rejoin the main pipeline as quickly as possible. Writing the backtracked walkers back to the walker buffer 223 and setting them with high priority allows them to directly enter the existing S200 / S300 / S400 pipeline, avoiding the need for a separate bypass. Consistency is restored quickly with minimal backoff; it does not impact the throughput window PREFETCH_W of the mainstream water; and the convergence time is significantly shortened.

[0186] Consistency and Robustness Description: Reads and writes in stages B1–B3 are all completed via the consistency interconnect channel 104; reads and writes are inherently consistent in the CXL / CCIX variants; in the PCIe + DDIO variant, cache injection of the main processor 101's cache 205 is implemented by DDIO, and visibility is guaranteed by the main processor 101 performing CLWB+SFENCE before reading the wanderer buffer 223. All write-backs to the wandering state slice 224 still follow the principle of content first, then header, and are accompanied by VER incrementing and CRC verification; if decompression or write-back verification fails, ERROR_CODE is set and the data is retrieved again or rolled back according to the policy, without affecting read-side availability.

[0187] Example 2 This embodiment combines Figure 6 The flowchart shown provides a six-stage implementation for the random walk task: S100 preprocessing stage, S200 scheduling and decompression stage, S300 calculation stage, S400 compression and write-back stage, S500 graph update and path backtracking stage, and S600 termination stage.

[0188] This invention uses the RandomWalk algorithm as an example for illustration, but is not limited to a specific walk order or a specific probability model.

[0189] S100: Preprocessing stage (Initialization).

[0190] The goal of this phase is to handle the most time-consuming aspects of the subsequent random walk—how to find and move data, and how to ensure concurrent consistency—all before execution. The result is that subsequent computations efficiently proceed only around a small number of locally available, frequently accessed data points, without back-and-forth between main memory 200 and the bus.

[0191] S110: Initialize tile and directory information.

[0192] First, the main processor 101 organizes the original graph into blocks, dividing vertices and edges into blocks according to a preset or adaptive strategy, and further packaging the data within each block into slices for batch transfer. The preferred setting is a block vertex / slice size range of 2–16MB (default 8MB). This is to match the granularity of DMA / bus transfers, replacing numerous small, scattered accesses with fewer large block reads and writes, thus significantly reducing the number of accesses and bus handshake overhead. To ensure that which slice to read, its location, and the decoding method can be determined at once, the system establishes a two-level directory: a block-level directory table 210 (block ID → first slice / slice number) and a slice directory table 211 (slice index → ​​physical address / format indicator FID, version VER, CRC). This directory and self-describing design integrates location and verification into one step: once a slice is found, the correct decoder can be selected immediately based on its FID / VER / CRC and the integrity can be verified at the moment of reading, avoiding reading unfinished or old version data, and laying the foundation for subsequent incremental patching and atomic head replacement of dynamic graphs (writing the payload first and then updating the header).

[0193] S120: Wanderer initialization and co-packet compression.

[0194] Subsequently, the system activates several walkers at each starting vertex as needed for the task, initializing their walker states (such as curr_v, prev_v, steps_left, seed, etc.). Existing technologies store graph structures and walker states separately. Unlike existing technologies, this invention merges and packages the local structural fragments of the same graph block plus the grouped states of the walkers in that block into a co-package slice 225, compresses it, and writes it to the main memory 200. The reason for co-packaging structure and state is that each step of the random walk depends on two types of information simultaneously: looking up the adjacency list and updating / reading the walker state. By placing these two types of information in the same slice, both can be obtained simultaneously with a single read, avoiding multiple data retrievals and secondary assembly. The main processor 101 can also batch process walkers of the same vertex in groups during the computation phase, maximizing cache hit rate. The write order strictly follows the content-first, header-later write order: the version and checksum fields (VER / CRC) in the header are written after the load is written. The reading end will only consume the complete and verified version, ensuring concurrency safety. Compression can be performed either by batch processing by the main processor 101 or by sending the descriptor to the compression unit 170 of the FPGA accelerator 102 via the MMIO channel 103 to generate slice write-back. Both implementations are equivalent to the technical effects of the present invention.

[0195] It is important to emphasize that main memory 200 only stores the compressed state in the long term. This means that decompression does not fall back to DRAM, but instead resides briefly in the near-end of the computation (the cache 205 of the main processor 101) and is directly consumed by the main processor 101. This strategy of persistent compression and near-end decompression eliminates write amplification and read loops between main memory 200 and the bus (preventing the secondary detour of decompressing to DRAM → then reading from DRAM by the main processor 101). Furthermore, in conjunction with the two-level directory structure, it results in fast location, minimal data transfer, and stable verification: reading one slice is sufficient to obtain the required structure and state for this step; version and integrity are uniformly guaranteed by the slice header; and during dynamic graph updates, only the affected slices are modified and the header is atomically replaced, avoiding large-scale rewrites and inconsistencies.

[0196] In summary, S100 solves the problems of location and concurrency consistency through block slicing, two-level directories, and self-describing headers. It also solves the problems of bandwidth and latency amplification by using structure / state co-packet compression and decompression to only dodge briefly at the near end of the cache. In this way, after entering the runtime, the system can form a stable overlapping pipeline between the three segments S200 / S300 / S400: after the FPGA picks up the chip and decompresses it, it is directly injected into the cache 205 of the main processor 101. The main processor 101 performs the walk on the spot, and then the FPGA accelerator 102 compresses and writes it back. End-to-end, there is almost no unnecessary DRAM round trip, which is the key to achieving low latency and high throughput in the dynamic graph random walk scenario of this invention.

[0197] S200: Dispatch & Decompression phase.

[0198] In this phase, the system's goal is to ensure that the main processor 101 always accesses ready data in cache 205, thereby hiding the time spent on data fetching and decompression in the background. To this end, the main processor 101 first writes a batch of descriptors (block_id, slice_id) to the descriptor queue 122 of the FPGA accelerator 102 via MMIO channel 103 and sends a FETCH doorbell to trigger chip fetching. This is done to explicitly define which blocks / slices should be loaded as a pipelined task queue, avoiding jitter caused by runtime ad-hoc queries. Simultaneously, the FPGA accelerator 102 maintains a configurable in-transit window PREFETCH_W (recommended 2–4): prefetching continues if the window is not full, and is suspended if the window is full. Window control benefits both ends: it ensures that there are always chips in the pipeline, preventing the main processor 101 from waiting for data, and it prevents excessive prefetching from consuming resources in cache 205 and on-chip temporary buffers, maintaining stable throughput.

[0199] S210: Descriptor sent with doorbell.

[0200] The main processor 101 writes a batch of descriptors (block_id, slice_id) to the descriptor queue 122 of the FPGA accelerator 102 via the MMIO channel 103 and sends a doorbell signal. This explicitly defines the requirement to load which slices as queued tasks, facilitating the FPGA accelerator 102 to advance the data stream sequentially and in batches. Simultaneously, a pre-fetch window (pre-window recommended 2–4) is set so that the next slice is received and processed only when the window is not full. The use of a queue + window ensures that slices are always in transit for background prefetching, preventing the main processor 101 from waiting for data during computation; it also prevents the fetching of too many slices at once from consuming the bandwidth of the on-chip temporary buffer and cache 205, thus avoiding jitter. The direct benefit is smoother throughput and controlled peak congestion, solving the problems of unpredictable data fetching and frequent waiting by the main processor 101 in the background technology.

[0201] S220: Slice acquisition and verification.

[0202] When the doorbell rings, the prefetch control unit 124 of the FPGA accelerator 102 triggers the coherent DMA engine unit 120. The coherent DMA engine unit 120 reads the target co-package slice (corresponding to) from the co-package slice 225 via the coherent interconnect channel 104. Figure 1 The slice parsing unit 132 performs fast verification (②) on the read slice header fields FID / VER / CRC, and records the error in the error register and automatically re-fetches the data when an error occurs. This approach of reading first and verifying on the edge achieves remote data retrieval with a smaller transmission volume, saving bus and DRAM bandwidth. Furthermore, it utilizes VER / CRC to ensure that during dynamic graph atomic head swapping (writing the payload first, then the header), the consumer only sees the complete and latest version, avoiding misreading of semi-finished or older versions. The effect is that, under frequent updates and concurrent reads, data correctness is verifiable, and system stability is higher, directly addressing the pain points of consistency and high rollback costs in the background.

[0203] S230: Decompression and cache injection.

[0204] After verification, the decompression unit 130 decodes the adjacency information and the vertex-grouped walker states in parallel in the first on-chip temporary buffer 140 / second on-chip temporary buffer 141, and rearranges them into a layout that the main processor 101 can directly consume. Unlike conventional decompression and subsequent write to DRAM, this invention writes the decompressed data directly to the cache 205 (③) of the main processor 101 using Cache Injection, and then generates a READY interrupt through the interrupt channel 105 to notify the main processor 101 to enter the calculation. The reason for directly sending the decompressed data to the cache 205 is to completely eliminate the two bypass paths of decompression → write back to main memory 200 → main processor 101 retrieves from main memory 200, which would cause the cache hit level to be delayed if the main processor 101 reads according to this path; this significantly shortens the end-to-end waiting time and makes S200 (chip fetching and decompression) and S300 (main processor 101 calculation) naturally overlap into a pipeline. If the platform does not have a cache injection function, the decompressed data can be temporarily stored in the sliding window buffer of the main memory 200 and then read by the main processor 101. Although this adds one more memory round trip, the queue + window + ready notification mechanism remains unchanged, and stable throughput can still be maintained. In summary, by placing the decompression process at the near end and placing the decompressed data in the nearest location accessible to the main processor 101, the S230 directly resolves the problems of bandwidth amplification caused by decompressed data fallback and the main processor 101 fetching data twice in the background technology.

[0205] S300: Random-Walk Step on CPU.

[0206] S310: Hit Read and Group Execution.

[0207] With the decompressed data already directly written to the cache 205 of the main processor 101 by S200, the main processor 101 can read adjacency information and vertex-grouped walker states locally when entering computation, without needing to return to DRAM. The strategy of grouping walkers on the same vertex for batch processing is to allow the same adjacency list to be reused by multiple walkers in a single batch: the main processor 101 only needs to load the adjacency vector of that vertex once to serve all walkers in that group, significantly reducing cache line jitter and memory accesses; simultaneously, the computational paths within the group are similar, resulting in less branch divergence, and making it easier for SIMD (Simultaneous Localization and Derivatives) and batch random number generation to take effect. This transforms the problems of frequent waiting and high bandwidth overhead of the main processor 101 mentioned in the background technology into cache hits + batch processing reuse, allowing time to be spent on computation rather than data retrieval. If the platform does not support cache injection, data is read from the sliding window buffer of the main memory 200, but the advantages of group batch processing remain unchanged.

[0208] S320: Random walk step.

[0209] For each walker within a group, the main processor 101 generates the next-hop vertex based on the adjacency list and a pre-defined random strategy (which can be either first-order or second-order, neither is limited), and updates the states such as prev_v / curr_v / steps_left. If steps_left reaches zero or a termination condition is met (such as a repeated access threshold or a coverage threshold), the process is marked as complete. To balance speed and accuracy, aliasing or prefix sum sampling can be used to achieve O(1) / O(log d) level selection of adjacent edges; in the second-order case, the transition rule <prev_v, curr_v> is used to avoid repeatedly loading irrelevant adjacencies. The reason for explicitly maintaining these states is to accurately determine whether a step is affected by a new edge / deleted edge in a dynamic graph update scenario: once a version inconsistency (VER / CRC mismatch) or the adjacency list is marked as invalid is detected, the main processor 101 immediately stops the current step and marks it as backtracked, adds the walker to the affected list 227, and sends it to the S500 for backtracking processing. This approach embeds correctness checks within the steps, avoiding error propagation and eliminating the need for a global pause, thus directly addressing the issues of difficulty in guaranteeing consistency in the background and high rollback costs.

[0210] S330: Write back and notify.

[0211] After completing this round of walks, the main processor 101 writes the updated walker state into the walker buffer 223 (④, in a preferred embodiment, the cache line corresponding to the write result has a high cache residency probability during runtime, so that subsequent consistency read requests initiated by the FPGA accelerator 102 can preferentially hit the cache 205 of the main processor 101 and return the latest walker state), and sends a compressible COMPRESS doorbell through the MMIO channel 103 so that the subsequent S400 stage can perform compressed write-back (⑤). Only the modified hot fields need to be written back to avoid unnecessary bandwidth overhead; on the other hand, the main processor 101 also summarizes the unfinished tile IDs and remaining steps to provide direct scheduling clues for the next round of S200 slice selection. In the PCIe+DDIO variant, in order to ensure that the FPGA accelerator 102 can read the latest state immediately, the main processor 101 performs CLWB+SFENCE before sending the doorbell to complete the cache line write-back and order guarantee. The write-back → doorbell → continue forward design allows the S300 and S200 / S400 stages to naturally overlap on the timeline into a pipeline: while calculating the current batch, the background executes the decompression of the next batch and the compression of the previous batch in parallel, eliminating waiting and jitter, and continuously outputting stable throughput.

[0212] S400: Compression & Write-back phase.

[0213] S410: Consistent read of wanderer buffer 223.

[0214] FPGA accelerator 102 initiates a consistent read request (⑥) to the data in the walker buffer 223 via the consistent interconnect channel 104 to read the latest walker state written by the main processor 101. This consistent read request is parsed by the cache consistency protocol: if the cache line corresponding to the target data (i.e., the walker state) resides in the cache 205 of the main processor 101, FPGA accelerator 102 directly obtains the walker state from cache 205; when the walker state to be read is not found in cache 205, the memory controller accesses the main memory 200. The reason for this design is to prioritize receiving the most recently generated and most likely to be accessed again data from the near end, avoiding unnecessary memory round trips, thereby reducing read latency and DRAM pressure. This step is complementary to the in-situ hit and write-back of the wanderer buffer 223 of the aforementioned S300: hot data that the main processor 101 has just written is usually still in the cache 205 of the main processor 101, enabling the S400 to start compression with cache-level latency, solving the problem of having to retrieve data remotely after calculation and having many round trips in the background.

[0215] S420: Adaptive Compression and Assembly.

[0216] Upon receiving the updated walker state, to avoid the imbalance of incompression / high cost in different data stages caused by a fixed compression algorithm, compression unit 170 adaptively encodes the walker state. The core approach is to split and vectorize a set of walker states on the same tile and vertex, and select the most suitable encoder based on the statistical characteristics of the fields. This strategy significantly reduces write-back bandwidth while ensuring an upper bound on the worst-case overhead. A typical field-to-encoder mapping is as follows: -curr_v[] / prev_v[]: When traversing within the same block, the vertex ID changes less, which is suitable for Δ+VLC; -step_cnt[]: mostly small values ​​or zero, suitable for bit-plane compression, or RLE-VLC when there are consecutive identical values; -flags[]: Boolean flags are sparse, and can be efficiently used for bit-plane encoding after being expanded by bits; -rng_seed[]: Approximately high entropy, difficult to compress, usually passed through storage.

[0217] The write-back assembly unit 172 receives the compressed stream output by the compression unit 170, and according to the format constraints in the slice directory table 211, divides the wanderer state into fixed-length or variable-length segments, adds self-description header information (such as offset, version, checksum), and then encapsulates them into complete wanderer state slices 224, which are then recorded in the slice directory table 211. The compressed stream output by the compression unit 170 is centrally encapsulated by the write-back assembly unit 172, ensuring that all written-back data units conform to a unified slice format. This facilitates subsequent indexing and version management in the slice directory table 211, meaning that each wanderer state slice 224 can be directly registered in the slice directory table 211, and avoids write-back fragmentation and directory information inconsistency problems caused by random-length data.

[0218] S430: Non-polluting write-back and directory information update.

[0219] After compression, the FPGA accelerator 102 writes the newly generated wanderer compressed slice into the wandering state slice 224 region (⑦) in the main memory 200 via the coherence interconnect channel 104 in a non-temporal / no-allocate manner, and updates the {phys_addr / len / FID / VER / CRC} of the corresponding entry in the slice directory table 211. The writing method here is persistent writing.

[0220] The use of non-polluting writes avoids occupying the capacity of the main processor 101's cache 205, preventing compressed data used only for archiving from pushing hot data from the main processor 101's cache 205 out of the cache. The atomic header-swapping process, which writes the payload first and then updates the header (VER / CRC), ensures that other readers always see the complete and verifiable new version. After the write-back and directory information update are complete, a DONE interrupt is generated via interrupt channel 105: this releases the in-transit window slots, allowing S200 to continue prefetching; it also indicates that the batch results have been safely persisted. This decouples the data path from the control path, solving the pain points of write-back polluting cache data, directory information asynchrony, and the easy reading of unfinished data in concurrent scenarios in the background technology.

[0221] The correlation between the flow relationship and the revenue (overlap of S200 / S300 / S400) of the present invention is as follows.

[0222] In steady state, the S400 stage overlaps with the previous two stages: decompression (k+2) | computation (k+1) | write-back (k). The window parameter PREFETCH_W ensures that there are always several slices in transit in the pipeline, thus hiding DMA round trips and decompression / compression delays in the background, and the main processor 101 is basically not waiting for data idly. If the cache injection of the main processor 101 is limited, or the level of the on-chip temporary buffer of the FPGA accelerator 102 (the first on-chip temporary buffer 140 or the second on-chip temporary buffer 141) is too high, the FPGA accelerator 102 will report the backpressure through the status bit, and the system can temporarily narrow PREFETCH_W to maintain steady-state throughput. The overall effect is: reduced write-back traffic, no contamination of cached data, strong directory consistency, and uninterrupted pipeline, directly addressing and alleviating the core problems in the background technology such as bandwidth amplification, cache thrashing, poor concurrency consistency, and throughput jitter.

[0223] S500: Graph Update & Backtrack phase.

[0224] In this step, the present invention clarifies the causal chain based on physical order logic. The graph update unit 160 has log reading logic, local repair logic and atomic update logic. The repaired Payload and SliceHdr can establish a sequential write relationship of writing content first and then writing header in the state of VER field increment, so that the co-packed slice 225 is fixed to the high consistency storage part in the main memory 200.

[0225] Specifically, the graph update unit 160 is connected to the directory area with update log 226 that is in a synchronous maintenance relationship with the slice directory table 211 in a streaming memory access manner. This enables the graph update unit 160 to have a local repair relationship with the co-package slice 225 based on the atomic head swap strategy, thereby avoiding the risk of reading semi-finished data that may occur in the prior art.

[0226] S510: Incremental updates and patching.

[0227] When update log 226 records a newly added or deleted edge, graph update unit 160 accurately locates the co-packed slices 225 affected by this update based on block-level directory table 210 / slice directory table 211. Then, it only performs the read → local patch → recompression → atomic header swap (VER increment, CRC recalculation) → write-back operation on these slices, while simultaneously maintaining directory information. Most updates to dynamic graphs are local and sparse. Reconstructing the entire graph would amplify bandwidth and latency, and also disrupt the pipeline. Limiting updates to a small number of affected slices allows for patching with minimal cost. Using atomic header swapping—writing the payload first and then updating the header (VER / CRC)—ensures that the reader never sees a half-finished product; even if S200 / S300 / S400 are concurrently reading, they will not read old format or incomplete data. The advantages of this design are: (i) Minimizes bandwidth and rewrite scope, resulting in faster updates; (ii) The version and verification are verifiable, and the consistency is verifiable; (iii) It runs in parallel with the existing pipeline without interrupting the main processor 101’s calculations and subsequent compression write-back process.

[0228] S520: Impact Assessment and Inventory Generation.

[0229] After the patch is completed, the system generates / refreshes the impact index table 212 and the affected list 227 based on the updated content: the former records which blocks / vertices / adjacent segments have been modified and their mapping to the rovers set, and the latter provides a list of specific rovers IDs that need to be reviewed (indexed by key such as the block, current vertex, previous vertex, etc.). Dynamic graph updates only affect a small portion of rovers already on the path. A full rerun would overwrite a large number of correct results, which is costly and has a large delay. Therefore, this invention first refines the scope of impact, only selecting rovers that may have traversed the modified edges / segments, and then only rollbacks and reschedulings are performed on these objects. The advantages of doing so are (i) limiting the workload of rollback and recalculation to a necessary subset; (ii) the location is verifiable, facilitating auditing and experiment reproduction; and (iii) the information structure is of the same origin as the directory information (block-level directory table 210 / slice directory table 211), making it easy to hit the affected data location in constant time on a hardware accelerator.

[0230] S530: Wanderer backtracking and rescheduling.

[0231] The wanderer backtracking unit 165 retrieves the latest wanderer state of the relevant wanderer based on the influence index table 212 and the affected list 227: it first reads the wanderer state from the wanderer buffer 223 (if the cache line corresponding to the wanderer state resides in the cache 205 of the main processor 101, it is returned directly from the cache 205; otherwise, it is returned from the main memory 200). If it has been archived, it is obtained by decompressing the wanderer state slice 224. Here, "archived" means that the required historical wanderer state does not reside in the wanderer buffer 223, or that it is necessary to backtrack to an earlier historical step not covered by the wanderer buffer 223.

[0232] For each hit walker, the system rolls back the walker's state to the earliest inconsistent step (e.g., before edge deletion, the last valid {prev_v, curr_v, step_cnt}), writes the rolled-back state back to the walker buffer 223, and simultaneously notifies the main processor 101 via interrupt channel 105 to prioritize these blocks / vertices for the next round S200 / S300. Rolling back to the earliest inconsistent step avoids a complete rerun and prevents path fragments from the new graph from being mixed with those from the old graph, ensuring the semantic correctness of statistics and path distribution. Through interruption and priority scheduling, affected objects can be recalculated as quickly as possible, achieving a balance between convergence and timeliness.

[0233] The advantages of doing this are (i) only rolling back the necessary steps, minimizing rework; (ii) not stopping the entire pipeline, maintaining overall throughput stability; and (iii) combining the aforementioned version / verification mechanism to avoid error propagation, ensuring that the results remain repeatable and consistent in dynamic graph scenarios.

[0234] S600: Finalize.

[0235] If there are no pending graph update or backtracking tasks in S500 (i.e., the decision is no), then proceed directly to S600. Before entering the termination phase, the system makes a decision based on a consistent snapshot: the closure is initiated only if two conditions are met simultaneously: (i) all walkers have completed (steps_left=0 or the termination criterion is met), and (ii) there are no pending graph update or backtracking tasks in the queue.

[0236] This technical solution aims to avoid mixing path segments generated from old graphs with path segments updated in new graphs during output, which could cause statistical distortion or unreproducible results. Before triggering termination, the control unit 180 also performs a version consistency check on the slice directory table 211 and briefly freezes the write header (VER / CRC) operation to ensure that the output corresponds to the same version set. The advantage of this approach is that the output results form a complete, verifiable, and reproducible final state, directly solving the problem of inconsistent results easily generated under dynamic graphs in the background technology.

[0237] Regarding the output method, the system provides two mutually exclusive options to adapt to different application scenarios: First, plaintext output: the wandering state slice 224 in the computation write-back area is decompressed offline piece by piece according to the directory information (block-level directory table 210 / slice directory table 211), and the complete path of each wanderer is concatenated and directly delivered to the upper layer. This method facilitates immediate consumption and analysis, and is suitable for online evaluation or debugging. Second, compressed archiving output: the wandering state slice 224 and directory information (block-level directory table 210 / slice directory table 211) are directly output, and the upper layer decodes and replays them when needed. The advantages of this approach are small I / O volume, fast disk write, and ease of long-term storage and distributed post-processing; since each slice has a self-describing header with FID / VER / CRC, subsequent decoding can be performed in parallel slices, ensuring integrity and reducing one-time memory overhead. The two methods only affect the timing and size of presentation, and do not affect the accuracy of the calculation.

[0238] During the cleanup process, the control unit 180 sequentially reclaims resources and states: it releases the descriptor queue 122 and the on-chip temporary buffer, resets the in-transit window quotas and doorbell / interrupt masks, and writes key statistics of this run (such as the number of slices processed, average decompression / compression latency, cache injection hit rate, ERROR_CODE count, and last error location) into registers for upper-layer reading. This approach avoids resource leaks, ensures a clean start for the next task, and accumulates observable data, facilitating parameter tuning (such as dynamically adjusting PREFETCH_W to suppress jitter) and rapid identification of bottlenecks or anomalies—a key challenge in the background technology.

[0239] Finally, to ensure consistency across platforms, an equivalent implementation is provided: If the platform lacks cache injection capabilities, the aforementioned judgment and termination logic remain unchanged. Only the decompressed data in stage S200 is temporarily resident in the sliding window buffer of main memory 200 instead of the cache 205 of the main processor 101. In stage S300, the data is read from the sliding window, and in stage S400, it is still written back in compressed state. Since the version and verification process of the directory information (block-level directory table 210 / slice directory table 211) remain unchanged, the consistency and reproducibility of the output in the termination stage will not be affected, thus balancing portability and correctness.

[0240] The field encoding strategy and judgment criteria of this invention are explained below.

[0241] In this embodiment, the compression unit 170 only compresses the walker state vector and does not compress the tile structure data. Tile data is directly stored in the on-chip temporary buffer and external storage to ensure controllable access latency. To balance compression efficiency and real-time performance, this embodiment introduces an adaptive encoding determination mechanism based on statistical features within the compression unit 170. This mechanism detects and selects the most suitable encoding method group by group according to field dimensions, avoiding the introduction of additional latency on incompressible data. The determination process and criteria are as follows: (1) Data grouping and field splitting.

[0242] Construct a field vector for each group of walkers at the same vertex (e.g., each group G=32), as shown in the table below.

[0243]

[0244] (2) Feature acquisition and threshold.

[0245] Lightweight statistics are computed for each field vector in the on-chip temporary buffer: The sparsity ρ_{0} = (#zeros / G), which is the percentage of 0 values. In the example, T_sparse is 0.7 by default. The percentage of a single value ρ_{mono} = (frequency of the most frequently occurring value / G), with the example defaulting to T_mono = 0.6; The span of the value range is calculated as max - min, with the example defaulting to T_span = 2^{ 12}; Incremental distribution: Δ[i] = 90th percentile of the absolute value of x[i] - x[i-1] P90(|Δ|), the example defaults to T_Δ = 2^{6}; Run percentage: ρ_{runlen} = (total number of consecutive repeating values ​​ / G), the example defaults to T_runlen = 0.3.

[0246] Estimate entropy / number of distinct values ​​(optional, O(G) approximation): used to detect high-entropy fields.

[0247] (3) Encoder selection rules.

[0248] A: Bit-Plane Coding (BPC). Bit-plane coding is used when ρ_{0}≥T_sparse or ρ_{mono}≥T_mono, representing a large number of 0s in the vector. This coding method has a high compression ratio for sparse vectors, is simple to encode, and allows for parallel decoding.

[0249] B: Δ+ Variable Length Coding (Δ+VLC). Δ+ variable length coding is used when span ≤ T_span or P90(|Δ|) ≤ T_Δ, indicating strong locality / small span characteristics of the data. This coding method is highly efficient for small variations / near-neighbor data, resulting in a short bitstream and low decoding burden.

[0250] C: Run-length encoding with variable-length coding (RLE-VLC). When a field vector is found to have a significantly long run (e.g., the percentage of consecutive duplicate values ​​ρ_{runlen} ≥ T_runlen, default 0.3), run-length encoding is used to reduce duplicate storage, and then variable-length encoding is performed on the run length. Suitable for flags[] or local hotspot clustering scenarios.

[0251] D: RAW-copy. When high entropy or neither of the above two conditions is met (such as rng_seed[]), no encoding is performed to avoid wasting time on incompressible data. The worst-case latency has an upper bound.

[0252] Through the aforementioned statistical judgment mechanism, the system can achieve on-demand compression of different fields at the hardware level, ensuring a high compression ratio in high compressibility data scenarios and without introducing additional overhead in high entropy data scenarios, thereby achieving dual optimization of compression ratio and latency in real-time random walk acceleration.

[0253] Example 3 This embodiment provides a physical hardware connection example of the system of the present invention.

[0254] The FPGA-based dynamic graph random walk acceleration system of the present invention includes a main processor 101 and an FPGA accelerator 102.

[0255] The physical hardware of the main processor 101 is preferably a CPU, but it can also be a GPU, DSP (Digital Signal Processor), ASIC (Application-Specific Integrated Circuit), or other types of processors. These processors all have sufficient computing power and control logic to perform tasks such as information preprocessing, tile partitioning, directory information construction, and random walk batch processing, and can exchange data and work collaboratively with the FPGA accelerator 102 through corresponding interfaces. This embodiment uses a CPU as an example for illustration.

[0256] The physical hardware of the FPGA accelerator 102 is a field-programmable gate array (FPGA). This FPGA integrates multiple physical hardware components. Specifically, the MMIO register group 190 is a storage unit composed of register circuits and is connected to the control unit 180 via an on-chip physical connection interface. The control unit 180 includes control logic circuitry and a state machine. The control logic circuitry and the state machine are physically connected via an on-chip interconnect bus. The control logic circuitry, acting as the control input source for the state machine, receives control information from the MMIO register group 190 and sends control signals to the control port of the state machine via internal signal lines. After receiving the control signals from the control logic circuitry, the state machine performs state switching and timing control according to preset state transition rules, generating corresponding state output signals. These state output signals are then transmitted back to the control logic circuitry via an internal feedback bus, forming a closed-loop control loop, achieving precise timing management and state coordination of the entire FPGA accelerator 102's workflow.

[0257] The descriptor queue 122 is a cache circuit that stores descriptor information and is connected to the control unit 180 via an on-chip physical connection interface. It is used to provide the control unit 180 with descriptor information for tasks to be processed. The prefetch control unit 124 is a control circuit with pre-set prefetch logic. It is physically connected to the control unit 180 and, under the scheduling control of the control unit 180, establishes a control connection with the consistency DMA engine unit 120 to trigger data prefetch operations. The consistency DMA engine unit 120 includes a DMA controller and a data transfer circuit. It serves as the data transfer interface between the internal and external storage systems of the FPGA accelerator 102. It is connected to the first on-chip temporary buffer 140, the second on-chip temporary buffer 141, the compression unit 170, and the write-back assembly unit 172 via on-chip physical connections to perform data read and write-back operations. The slice parsing unit 132 is a slice parsing logic circuit that performs data parsing. It is physically connected to the decompression unit 130 and is used to transfer the corresponding data to the decompression unit 130 after completing slice parsing. The decompression unit 130 consists of a decompression algorithm circuit and is connected to the first on-chip temporary storage buffer 140 and the second on-chip temporary storage buffer 141 via an on-chip data path. It is used to write the decompressed data into the corresponding on-chip temporary storage buffer. The first on-chip temporary storage buffer 140 and the second on-chip temporary storage buffer 141 are respectively the first and second on-chip memory circuits, used to temporarily store the slice data read by the consistency DMA engine unit 120, and are physically connected to the slice parsing unit 132 via an on-chip data path. The compression unit 170 includes a compression encoding circuit, which is connected to the consistency DMA engine unit 120 and the write-back assembly unit 172 via an on-chip physical connection. It is used to compress the data to be written back. The write-back assembly unit 172 consists of a write-back assembly control circuit for data assembly and writing. It writes the compressed data back to the external storage system through the consistency DMA engine unit 120. The graph update unit 160 is a graph update logic circuit that performs graph structure updates. It establishes a data connection with the external storage system through the consistency DMA engine unit 120 and is used to perform graph data reading and update operations. The wanderer backtracking unit 165 includes a path backtracking calculation circuit, which interacts with the external storage system through the consistency DMA engine unit 120 and establishes a control connection with the control unit 180 to trigger corresponding scheduling processing when a backtracking event occurs. These hardware modules form a collaborative processing structure through on-chip physical connections, forming a complete data processing pipeline.

[0258] MMIO channel 103 is a memory-mapped input / output interface circuit used for register access and control signal transmission between the main processor 101 and the FPGA accelerator 102. The main processor 101 configures and schedules the FPGA accelerator 102 through MMIO channel 103. Coherence interconnect channel 104 is a high-speed data transmission interface circuit used to connect the FPGA accelerator 102 to the cache 205 / main memory system 200 of the main processor 101, ensuring the FPGA accelerator 102's read and write operations on graph data, walker states, and intermediate result data. Interrupt channel 105 is an interrupt signal transmission circuit that physically connects the FPGA accelerator 102 to the main processor 101, used by the FPGA accelerator 102 to send interrupt requests and status notifications to the main processor 101. These channels are hardware interfaces implemented through physical connections, buffers, protocol controllers, and other circuit components, supporting reliable communication and coordinated operation between different devices.

[0259] The physical hardware of the main memory 200 is preferably DRAM (Dynamic Random Access Memory). This is because the system needs to store a large amount of graph data, walk state slices, directory information, etc., and DRAM has the characteristics of large capacity, low cost, and moderate read / write speed, which can meet the needs of large-scale graph data storage. At the same time, the FPGA accelerator 102 interacts with the DRAM through the memory controller, supporting random access and batch read / write operations, which meets the performance requirements of the dynamic graph random walk algorithm for the storage system.

[0260] Specifically, the CPU is connected to the register circuit (MMIO register group 190) via MMIO channel 103 for writing control information and reading the wanderer's state. A control logic circuit (control unit 180) with a state machine is connected to the register circuit to receive control information written by the main processor 101 and accordingly advance the internal state machine and scheduling process. The CPU writes control instructions (such as descriptors and doorbell trigger signals) to the register circuit. The control logic circuit reads the stored data in the register circuit and generates scheduling signals to control other circuits.

[0261] The CPU writes descriptors to the cache circuit (descriptor queue 122) through MMIO channel 103. MMIO channel 103 refers to the memory-mapped input / output channel formed between the main processor 101 and the FPGA accelerator 102 based on an on-chip or board-level wired interconnect bus, which realizes the transmission of address, data signals and control signals through physical wires.

[0262] The descriptor queue 122 is a control logic circuit and a state machine. Its read / write ports are connected to the control logic circuit of the control unit 180 via on-chip physical connections. The control logic circuit reads descriptors from the cache circuit and sends scheduling signals to the control circuit (prefetch control unit 124).

[0263] The control logic circuit is connected to the cache circuit for reading descriptors. Simultaneously, the DMA controller is connected to the common packet slice 225 and two on-chip memory circuits via the consistency interconnect channel 104 to realize the reading and storage of compressed slices. The slice parsing logic circuit is physically connected to the on-chip memory circuit to receive input slices and perform integrity checks. The decompression algorithm circuit is connected to the two on-chip memory circuits respectively, storing the wanderer packet segment and the adjacent segment in the first and second on-chip memory circuits respectively. Finally, the decompressed data is written to the cache 205 of the main processor 101 or the main memory 200 via the consistency interconnect channel 104.

[0264] During the S200 prefetch / decompression / injection phase, the control logic circuit sequentially reads descriptors from the cache circuit. The DMA controller reads the corresponding compressed slice from the common packet slice 225 through the consistency interconnect channel 104 and writes it to the first on-chip memory circuit / second on-chip memory circuit. The slice resolution logic circuit performs an integrity check on the input slice. After the check passes, the decompression algorithm circuit first decodes the compressed data, restoring it to aligned adjacent segments and wanderer packet segments. The decompression algorithm circuit sends the wanderer packet segments to the first on-chip memory circuit and the adjacent segments to the second on-chip memory circuit. The decompression algorithm circuit writes the decompressed data to the cache 205 of the main processor 101 or the main memory 200, completing the data prefetching and decompression injection.

[0265] During the S300 calculation phase, the main processor 101 directly reads the decompressed data from its cache 205, performs a random walk calculation, and writes the updated walker state back to the walker buffer 223. After the calculation is complete, the CPU sends a COMPRESS doorbell signal to the compression encoding circuit via the MMIO channel 103.

[0266] During the S400 compression and write-back phase, after the compression encoding circuit receives the COMPRESS doorbell from the CPU through the MMIO channel 103, it reads the updated wanderer state from the wanderer buffer 223 in DRAM through the consistency interconnect channel 104.

[0267] The compression encoding circuit adaptively compresses the read data. The compression encoding circuit is connected to the write-back assembly control circuit. The compression encoding circuit sends the compressed data to the write-back assembly control circuit. After receiving the compressed data, the write-back assembly control circuit encapsulates the compressed data into a complete walk-through state slice 224. The write-back assembly control circuit is connected to the DRAM via the coherence interconnect channel 104. The write-back assembly control circuit writes back to the compute write-back area (write-back walker buffer 223) in the DRAM in a non-temporal / no-allocate manner via the coherence interconnect channel 104, and updates the corresponding entry in the slice directory table 211 in the DRAM.

[0268] In summary, the components of this invention, through tight interlinking within the consistency domain, integrate the originally discrete prefetching, decompression, computation, compression, and write-back steps into a highly structured and interconnected end-to-end pipeline. Specifically, by adding location constraints, relationship constraints, and version-level constraints based on VER / CRC at each key node, not only is the spatial physical relationship between the cache 205 of the main processor 101 and the main memory 200 clarified, but the time-series constraints from S200 to S400 also enhance the distinguishability of similar features, such as the shared-pack slice 225 and the wandering state slice 224, ensuring high throughput and strong consistency in random walk computation within a dynamic graph environment.

[0269] It should be noted that the specific embodiments described above are exemplary. Those skilled in the art can devise various solutions inspired by the disclosure of this invention, and these solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents. This specification contains multiple inventive concepts; terms such as "preferredly," "according to a preferred embodiment," or "optionally" indicate that the corresponding paragraph discloses an independent concept. The applicant reserves the right to file divisional applications based on each inventive concept.

Claims

1. An FPGA-based dynamic graph random walk acceleration system, characterized in that, The system comprises a host processor (101) and an FPGA accelerator (102); The host processor (101) is used for information preprocessing; wherein, the original graph is divided into blocks, and directory information for locating data slices is constructed; a walker is activated at a starting vertex, and information of a graph structure fragment and a walker grouping is compressed into a co-pack slice (225); The FPGA accelerator (102) is used for scheduling and decompression; wherein, in response to the control of the host processor (101), the co-pack slice (225) is read, a parsing checksum and decompression operation are performed, and decompression data is provided to the host processor (101); The host processor (101) receives the decompression data, processes the walkers with the same vertex to perform random walks, updates the walker states, writes the updated walker states into a walker buffer (223), and notifies the FPGA accelerator (102) to start a compression operation; The FPGA accelerator (102) reads the walker states in the walker buffer (223) in response to the notification, performs a compression operation thereon, and writes the compression result back as a walker state slice (224) and updates the corresponding directory information.

2. The system of claim 1, wherein, The FPGA accelerator (102) supports dynamic graph updating and path backtracking; Wherein, in response to the updating of the graph structure, the affected co-pack slice (225) is located and repaired, and the affected walker is located and its state is rolled back.

3. The system of claim 1 or 2, wherein, The FPGA accelerator (102) is configured to: receive a descriptor from the host processor (101) and trigger a FETCH doorbell; perform a consistent DMA slice taking, parsing checksum and decompression operation under window constraints to obtain decompression data; and when the operation is completed, immediately send a READY interrupt to the host processor (101); The host processor (101) receives the decompression data, processes the random walks according to the grouping rule of the walkers with the same vertex, writes the processing result back to the walker buffer (223), and sends a COMPRESS doorbell to the FPGA accelerator (102) to notify the start of the compression operation; The FPGA accelerator (102) performs compression and backwriting, performs a consistent reading operation on the walker states in the walker buffer (223), adopts an adaptive coding scheme, writes the coding result back to the walker state slice (224), updates the slice directory table (211), and sends a DONE interrupt to recycle the window quota.

4. The system of any one of claims 1 to 3, wherein, The FPGA accelerator (102) comprises: a consistent DMA engine unit (120) for performing a consistent DMA slice taking operation; a slice parsing unit (132) for recalculating the integrity checksum of the slice and judging whether the slice is valid; and discarding invalid slices; a decompression unit (130) for decoding the determined valid slices, restoring them into aligned adjacent segments and walker grouping segments; and storing the segments in an on-chip temporary buffer according to the type classification of the index information in the slice header; a compression unit (170) for adaptively coding the walker state vectors. The write-back assembly unit (172) segments the encoding result and attaches a self-description header according to the format constraints of the slice catalog table (211), and finally encapsulates it into a complete wandering state slice (224). Control unit (180), scheduling slice parsing unit (132), decompression unit (130), write-back assembly unit (172), graph update unit (160) and wanderer backtracking unit (165); The graph update unit (160) locates the affected co-package slice (225) according to the update log (226), performs local repair and atomic head replacement, and maintains the impact index table (212) and the affected list (227). The wanderer backtracking unit (165) backtracks the relevant wanderers to the earliest inconsistency according to the index / list, writes the backtracked wanderer state back to the wanderer buffer (223), and notifies the main processor (101) to perform priority scheduling.

5. The system of any one of claims 1 to 4, wherein, The main processor (101) is configured as follows: The original graph is organized into blocks, and vertices and edges are divided into blocks according to preset or adaptive strategies. The data within the blocks is packaged into slices for batch transfer. At each starting vertex, several walkers are activated as needed by the task, and their walker states are initialized. The local structural fragments of the same tile and the grouping states of the walkers in that tile are merged and packaged into the same slice, and then compressed and written to the main memory (200).

6. The system of any one of claims 1-5, wherein, The main processor (101) writes slices to the descriptor queue (122) of the FPGA accelerator (102) through the MMIO channel (103) and sends a FETCH doorbell; at the same time, it sets a window in transit; it only receives and processes the next batch of descriptors when the window is not full. When the FETCH doorbell arrives, the prefetch control unit (124) of the FPGA accelerator (102) triggers the coherent DMA engine unit (120) to perform a DMA read operation on the co-packet slice (225) through the coherent interconnect channel (104) to obtain the target co-packet slice (225). The slice parsing unit (132) performs a fast check on the read slice header fields FID / VER / CRC. When the check fails, it records the error in the error register and automatically retrieves it. Once the verification is successful, the decompression unit (130) decodes the adjacency information and the rovers grouped by vertex in parallel in the on-chip temporary buffer, and rearranges the layout so that the main processor (101) can directly consume it.

7. The system of any one of claims 1-6, wherein, When the FPGA accelerator (102) sends a read request to the walker buffer (223) through the coherence interconnect channel (104), if the cache line corresponding to the target data to be read is still residing in the cache (205) of the main processor (101), the FPGA accelerator (102) directly obtains the target data from the cache (205) through the coherence interconnect channel (104) and determines the latest walker state vector; if the target data is not hit, it accesses the main memory (200). The compression unit (170) in the FPGA accelerator (102) adaptively encodes the wanderer state vector and outputs a compressed stream; The write-back assembly unit (172) receives the compressed stream output by the compression unit (170), divides the wanderer state into fixed-length or variable-length segments according to the format constraints in the slice catalog table (211), adds self-description header information, encapsulates it into a complete wanderer state slice (224), and records it to the slice catalog table (211). Once compression is complete, the FPGA accelerator (102) writes the newly generated wander state slice (224) into the wanderer buffer (223) in the main memory (200) through the coherence interconnect channel (104) and updates the corresponding entry in the slice directory table (211).

8. The system of any one of claims 1-7, wherein, When the update log (226) records an operation of adding or deleting an edge, the graph update unit (160) in the FPGA accelerator (102) accurately locates the co-package slice (225) affected by this update according to the block-level directory table (210) or slice directory table (211), and then performs read, local repair, re-compression, atomic head swap and write-back operations only on the slice, and maintains the directory information synchronously; Once the patch is complete, an impact index table (212) and an affected list (227) are generated / refreshed based on the updated content. The impact index table (212) records the affected blocks / vertices / adjacent segments and their mapping to the wanderer set, while the affected list (227) provides a list of specific wanderer IDs that need to be reviewed.

9. The system of any one of claims 1-8, wherein, The wanderer backtracking unit (165) in the FPGA accelerator (102) retrieves the latest state of the relevant wanderer based on the influence index table (212) and the affected list (227): it reads first from the wanderer buffer (223), and if it has been archived, it is obtained by decompressing from the wander state slice (224).

10. A method for accelerating dynamic graph random walk based on FPGA, characterized in that, The method includes: The main processor (101) performs information preprocessing; wherein, the original graph is divided into blocks, and catalog information for locating data slices is constructed; the walker is activated at the starting vertex, and the graph structure fragments and walker grouping information are compressed into a co-packed slice (225). The FPGA accelerator (102) performs scheduling and decompression; among which, In response to the control of the main processor (101), the co-package slice (225) is read, parsing, verification and decompression operations are performed, and the decompressed data is provided to the main processor (101). The main processor (101) receives the decompressed data, processes the walkers at the same vertex to perform random walks, updates the walker state, writes the updated walker state into the walker buffer (223), and notifies the FPGA accelerator (102) to start the compression operation. In response to the notification, the FPGA accelerator (102) reads the wanderer state in the wanderer buffer (223), performs a compression operation on it, writes the compression result back as a wanderer state slice (224), and updates the corresponding directory information.

Images